Log in

View Full Version : Shrink that installer!


Larry Hastings
08-06-2003, 03:08 PM
Like everyone else here, I'm working on a indie shareware game for Windows. And like I hope everyone else here, I'm very concerned about download size. I've set a goal of 5MB for my game, and I'm more-or-less on track to make it. Y'see, the larger your installer is, the larger percentage of people that will have trouble downloading it. And failed downloads mean lost sales.

The biggest tip I can suggest is to use compressed file formats for all your media files. Unless your game is naturally light on media files, you probably have your share of images, textures, sounds, and music. Your installer will compress these, with use a generic compression method (probably zlib (http://www.gzip.org/zlib/)), but that can't compress the data nearly as well as a domain-specific compressor.

Here then are my specific tips.


Use PNG files for images/textures.

If you're shipping uncompressed BMP files, shame on you! Image compression libraries work wonders.

There are a number of file compression standards for images, and I'm sure you're familiar with all the big ones relevant to game designers. But just for laughs I'm gonna list 'em anyway. GIF is lossless, but limited to 256 colors. JPG is lossy, which means the images will be distorted--though more often than not no one will ever notice. DXT2 through DXT5 are lossy texture compression formats defined by DirectX. They're natively supported by some graphics cards, but I don't think we indie shareware game developers can depend on that. (I especially wouldn't use DXT1 any time soon, due to a hardware bug on Nvidia graphics cards (http://udn.epicgames.com/pub/Content/TextureSpecifications/#DirectX_Texture_Compression).) Personally, I think the best choice for storing images on disk, or in the installer, is to use PNG files.

PNG (pronounced "ping") is a lossless compression standard. That means that, unlike JPG, PNG will never introduce errors (noise) into your images. PNG was designed as a drop-in replacement for GIF; Unisys realized it owned a patent on GIF, and started suing everybody, and that pissed off enough people that a bunch of developers got together to write a replacement. PNG does everything GIF does, and does it better--it's an improvement in every way over GIF. The big one: PNG supports 24-bit color (GIF is 8-bit color only) and 8-bit alpha (GIF supports only on/off transparency).

I'm writing for DirectX 8, which has built-in support for PNG files. I'm not sure how far back that support goes. If you don't think your target version of DX supports PNG, you'll need to ship your own decompressor. If you're using an image processing library (Leadtools, Pegasus, and the like), I'm sure it already supports PNG. If you need one, the obvious choice is libpng (http://www.libpng.org/pub/png/libpng.html), the open-source PNG reference implementation. It's under an MIT-style license, which basically says "use it however you like, you just can't sue us".

One more tip about PNG files. Almost certainly the image editing software you're already using (Photoshop? Paint Shop Pro? The Paint program that ships with XP?) natively supports PNG files. However, these tools don't always do the best job compressing PNG files. There's an open-source utility I just discovered called Pngcrush (http://pmt.sourceforge.net/pngcrush/) that recompresses PNG files in an attempt to make them smaller. Again, it will never introduce noise or errors into your image... it'll just try different approaches to compressing your image, and pick the best one. I ran pngcrush -brute on my textures directory, and it dropped from 1.6MB to 1.3MB. Pretty good for a half-hour's work, and it's free!


Use MP3 or OGG files for sounds.

Another slam-dunk. I hope you're not shipping WAV files with your game--you can do much better.

I'm sure you're familiar with MP3 files, but just in case you aren't: MP3 is a file format that compresses digital audio using a "lossy" compression method. That means the sound output from an MP3 will be slightly different than the original. But unless you go for maximal compression, odds are that you won't notice the difference--and neither will your users. (After all, JPG is a lossy file format, and how often do you notice the difference?)

OGG files are a patent-free open-source replacement for MP3 files. My understanding is that they're technologically better than MP3 files, and provide better sound with smaller files.

If you're curious, I suggest you experiment with whether MP3 or OGG works better for you. But I bet most of you aren't that curious, so let me say that you should use OGG files if you can, and otherwise use MP3 files.

Even if you can't use OGG or MP3 files directly in your game, you might consider shipping OGG files in your installer and then decompress them back into WAV from your installer. That way you still get the benefit of the compression for the download size. You can use any of the prevalent sound libraries (FMOD, XMPlay) to do this.


Use MOD files, or even MO3 files, for music.

For downloaded games, MOD files are a far better choice for music than MP3 or OGG files. And MO3 files are even better.

If you're serious about making a 5MB installer, and you still want music in your game, you've probably already decided on using MOD files (MOD, XM, IT, S3M, etc) for music. A MOD file, for those who aren't aware, is kind of like a MIDI file with built-in WAV sounds for the instruments. MIDI files sound different on different sound cards, because the sounds for the instruments (the trumpet, the piano) are part of the sound card, and how good the sound is (or how well it works with the song) varies wildly. But a MOD file will sound exactly the same on every computer, and perhaps more importantly, MOD artists are free to use any sounds they like. MOD files range wildly in size, based on how repetitive they are, and how many unique sounds they ship, but most are under half a megabyte for several minutes of music.

The original MOD file format was created on the Commodore Amiga platform more than ten years ago, long before home computers had enough power to compress sounds. As such, it doesn't support any compression. Neither do any of the other popular MOD file formats I've already mentioned. Still, MOD files are a much better choice than MP3 or OGG files for music. A three-minute MOD file could easily be under 500KB, wheras the average three-minute MP3 file would be roughly 3MB. Clearly, MOD files are a much better choice for indie shareware downloadable games.

But there's an even better option, if you can take advantage of it. There's one MOD file format that supports compression: the MO3 (http://www.un4seen.com/mo3.html) format from Ian Luck. MO3 isn't really a new MOD file format; rather, it's a way of storing existing MOD file types in a highly compressed form. It supports compressing XM, IT, MTM, S3M, and MOD files. The digital sounds (the WAV file parts) are compressed as MP3 or OGG files, and the sequence (the MIDI file part) is also compressed, with a more conventional method. I tried the XM files I plan to ship with my game, and compressed them from 3.9MB to 1.6MB. That's a 60% savings--hooray!

The MO3 home page (http://www.un4seen.com/mo3.html) links to a lovely utility called mo3enc (http://www.un4seen.com/download.cgi?mo3enc18). The mo3enc UI is a little strange at first, but once you're used to it, it's superbly easy to compress MOD files.

One more note about MO3 and MOD files. Paradoxically, MO3 files actually compress a little better than MOD files in my experience. I ZIPped up my MO3 directory, and the resulting ZIP file was 1162366 bytes and averaged 26% compression; the equivalent MOD files zipped up to 3181789 bytes and averaged 18% compression.

MOD files are already wonderful, but MO3 files are even better. Unfortunately, MO3 is a proprietary file format, and is only natively supported by BASS and XMPlay (also Ian Luck products). If you use FMOD, you're out of luck.

If you're not using a sound library that supports MOD or MO3 files, and you ship music with your game, I strongly suggest you consider switching. Short WAV or MP3 files will loop too often and become annoying for the user. And a game that shipped without music... well, that might not feel professional enough to get many sales. I firmly believe that MOD/MO3 files are the absolute, no-question, best solution for music in downloadable games.


Use the NSIS installer.

It's a pain to use, but it's what you need.

As I've mentioned in a previous thread, I've been doing installers and automatic updaters for more than ten years now. I've seen 'em all come and go. My pick for installers under Win32, for all but the most complicated installation scenarios, is the NSIS (http://www.nullsoft.com/free/nsis/) installer from NullSoft (makers of WinAmp). It's free, and open source (another MIT-style license), and I bet it does everything you need.

Let me get this right up front: the NSIS scripting language is stinky. It doesn't have real looping constructs, it doesn't have user-definable variables, named parameters to functions, or variable scoping. It's stone knives and bearskins time. Also, while it's nice that it comes with source code, the source code itself is a mess.

But the run-time for the installer is a jaw-droppingly-small 36k! That includes everything. Zlib file compression, modifying the registry, inspecting files, and even a language extension mechanism using DLLs. Everything a game installer needs.

Currently, a major rewrite (2.0) is in late beta, spearheaded by folks in the NSIS community (nsis.sf.net) rather than the original authors. I haven't used it, but Winamp3 uses it, so I suspect it's stable enough for production code. As for me, I still use 1.98, 'cause I'm used to it.

___________________________________

Phew! Well, that took an hour or two. Time I could have spent on my wonderful little game! I sure hope that was of use to somebody... was it?

LordKronos
08-06-2003, 03:24 PM
With NSIS, try both the regular and the bz2 compression. I find bz2 works better overall, but it depends on your files. For some, its better, for others its worse (is there any option to specify compression format for individual files? I never found any but thought it would be handy for squeezing the most out of the installer.)

If you really want to crunch things down and you have extra time to play around, experiment with different formats. Although PNGs and GIFs have better compression than TGA's, I found that I often get better compression with TGA's bz2 compress than I did with GIFs or PNGs.

Does that stereo wave/MP3/Ogg file really need to be stereo? Often you can convert them to mono without noticing a difference.

Larry Hastings
08-06-2003, 03:44 PM
Music stored in MP3/OGG probably does need to be stereo. But most stereo MP3 files are encoded using "joint stereo", which means the MP3 only stores one full channel and then the difference between the first and second channels. (I imagine OGG does something similar.) It means a stereo MP3 isn't all that much larger than the equivalent mono MP3.

If you do have stereo sound files which you want to convert to mono, make sure you drop the bitrate too. Otherwise your file size probably won't change.


One more point about MP3 files: I recommend you compress them using "VBR". VBR means Variable Bit Rate; rather than using the same number of bits every frame to describe the sound, it uses more bits for complicated frames, and fewer bits for simple frames. It's another good way to bring your filesize down without losing quality. (OGG is innately VBR.)


And one more point about OGG files: they're easy to make smaller. In order to shrink an MP3, you must decompress it, then recompress it, and the process doesn't work very well. With OGG files, you can strip off information to make it smaller without decompression / recompression.


Finally, Mr, Kronos, I just went and confirmed what I suspected: in NSIS, bzip2 support is an either-or proposition. The installer stub (called exehead) only supports one at a time. And that's true of both NSIS 1 and NSIS 2. I'm not surprised, bzip support was hacked in... but then, what in NSIS was not. :(

Dr_Gonzo
08-06-2003, 04:06 PM
A couple of points, remember when making mp3 or ogg that it's the bit rate that counts, so 128kbs MP3 is 128kbs regardless of whether it's mono or stereo. Both I do believe can do single channel (ogg can do a fair few channels I think should you feel like going surround), and as Larry said mp3 can use intensity* or joint (middle & side)** stereo to avoid encoding two discrete streams. I'm certain ogg does something similar, but the compressor I use doesn't give you that options so I don't know exactly. Ogg can generally run at a lower compression rate than mp3 for a similar sound quality which is a nice thing.

Secondly, some images will compress in a fairly poor manner with PNG. I was just trying to add some backgrounds to Blob, and we've only included PNG decompressors, which means I had to be fairly ruthless with colour depths to get a reasonable size out of it. Anything photo like looks good with jpeg (as the softening effect it gives is fairly complementary). Fine detail has a good chance of being lost though, so experimentation is the key.

Incidentally, I have noticed that the png code with PSP7.0 (which I recently upgraded to thanks to it being £20 at game :D ) is a lot better than psp6's, but pngcrushing will still save 1-10% on average for me. For animations, tiling into one image can save some space, and if you really want to crunch, but still have alpha, pngquant can do palletized (8bit) images with an alpha channel, which is pretty cool.

There is an animation format called MNG which looked interesting. I think the idea is to give you gif like animation facilities, but there weren't too many tools last time I looked at it, so using it could be a bit hairy.


* encode it mono, and place it in the stereo field based on intensity. a bit hacky.
** encode one channel that is the common audio, and another that is the difference between the two channels. Effective, space saving, and not too detrimental on the sound.

Fenix Down
08-06-2003, 04:19 PM
For images I use compressed 16-bit TARGA (TGA) files. I don't know of any other formats that support saving in 16-bit color (there are probably others I just haven't bothered to check). Using 16-bit color instead of 24-bit (and then converting to 16-bit on load anyway) reduces file size up to about 30%. :)

Right now I'm using MOD (XM) for music, which can give you as much as 5 minutes of music per 500K (and that's before being zipped). FYI, using MP3 requires a license now so I'd go with OGG. http://www.mp3licensing.com/royalty/index.html

Larry Hastings
08-06-2003, 04:54 PM
Oh, can any reply escape my e'er vigilant watch?


PNG supports 16-bit color. Always has, as far as I know. On the other hand, I hadn't thought about using it--thanks for the tip!


MNG is more or less meant as a replacement for GIF animations.

elund
08-06-2003, 05:11 PM
Great post, Larry. Some comments:

MO3 isn't the only compressed Mod-style format out there. Fmod supports ogg-compressed XM files called OXM. However, I do like MO3 better and that's what I'm using now. Bass is not cross-platform like Fmod, so I can't say I'll stick with this decision long term. Anyhow, the MO3 encoder is a great tool and easy to work with. You can rapidly change the quality level of each sample and hear the difference, so you can squeeze out the greatest space reduction. Use mono files wherever possible. You can sometimes achieve good results by panning inside the Mod itself. Use the lowest sample rate the sound can endure. High pitched sounds naturally need a higher rate.

When talking about file sizes, it's important to compare zipped results, since that's probably what your files will be delivered wrapped in. i.e., if you compress a file, put the original in a zip and the compressed version in a zip, then compare zip sizes. Generally I've found that files compressed with a lossy format recompress a little more after zipping.

I use a palettized 8-bit PNG format for most of my image files. For the larger files with alpha layers, I wrote a program to save the alpha layer separately. In one case I have several animations with the same shape, so I'm able to just reuse one mask for all of them. When saving down to 8-bit png, turn off dithering, and if your image program allows it reduce the total number of colors to below 256. I find that you can trim a few more K that way. If the visual difference is small, it's worth it to me. Do this too much and you'll get banding and odd color shifts. PSP offers a couple methods for color reduction, so try both to see which gives the best bang for your buck.

I fooled around with pngcrush on several of my files and couldn't get them to get any smaller. I guess I'll take this as a good sign?

Also, consider using UPX packer to compress your EXE and DLL files.

luggage
08-06-2003, 07:23 PM
I find that ogg is much better than mp3. At 128 and above they're both quite similar but if you start dropping it ogg is far, far better. Essentially you can get a much smaller file with ogg at a reasonable quality. At the moment we have a 3 minute track that's 758k.

JackNathan
08-06-2003, 08:16 PM
Be sure not to overlook TGA. In my very limited tests the TGA's tend to zip smaller than the PNG's even though they are bigger outside of zip. Also I don't have to worry about Photoshop embedding a color profile and D3DX reading it in and messing with my texture. But the most important factor is trying to get it down to 256 colors.


Jack

hanford_lemoore
08-06-2003, 09:56 PM
Larry, great post!

I have a question about MOD MO3: How do you author these files? With MP3 you can basically record whatever you want, whether you're using a microphone, professional music euipment, or Reason. If I were to go with MOD, what do I compose my music in?

Thank,

Hanford

Fariz
08-06-2003, 10:57 PM
Correct me if I am wrong, but most installer uses same compression methods to compress all the package, which PNG uses to compress the picture. Saying it other way, after creating installation .exe I have not found difference in packaged size between files, which had graphix in .bmp and files which had graphix in .png Difference is very noticable if you use .jpg, and at some conversion ratio disturtions are neglectable. Still it's better to use jpg for static pictures, in animation they create some artifacts.

So, using png at least in my experience gives advantage only when game is unpacked to the disk (which hardly anyone cares) but not in destribution package sizes.

Morphecy
08-06-2003, 11:23 PM
"Download this software http://upx.sourceforge.net/

I've used this to compress the generations.exe file and it's now approx 500KB instead of almost 2MB. The file remains an .EXE and will run in it's compressed state. You can also compress .DLL files!!!"

papillon
08-07-2003, 12:35 AM
Flip side of the question - is there a size that is too *small* for a demo/fullgame? Is there a size below which the browser will think "Anything that small must not be much of a game" and not bother to look at it?

LordKronos
08-07-2003, 03:42 AM
Off the top of my head, I'd say 1MB is probably the sweet spot. It says "big" while not actually being big.

Raptisoft
08-07-2003, 03:45 AM
Does anyone know where you can get PNGCrush in an executable format, as opposed to the source code? (None of the sourceforge links seem to have a binary). I tried compiling it, but I got over 3,000 errors, using VC7 with some custom stuff in it. :confused:

StAn
08-07-2003, 05:17 AM
The Amiga tracker Digibooster Pro can also use MP3 samples. ModPlug and DeliPlayer can play DBM mods, but I'm not sure if the replayer source is available though.

To answer someone's question, to create mod music you need a tracker: Impulse Tracker (IT), ScreamTracker (s3m), or whatever (I'm not well versed into Windows music software :p).

ggambett
08-07-2003, 05:43 AM
PNG is not the most efficient way of storing images. I'm splitting the source PNG images into the colormap and the alpha channel, saving the alpha channel as 4-bit raw BMP, and the colormap in either JPEG(90,5) or 8-bit indexed raw BMP. Everything is compressed with ZLib, that's why I use raw BMP instead of RLE BMP.

In some cases the original PNG is smaller, though. However, I still can't find a way to save 8-bit indexed PNG with anything else than a 1-bit alpha channel.

In Betty's Beer Bar (http://www.mrio-software.com/info.php?id=bbb) I managed to stuff a complete scenario, props, the pretty complete Betty animation, and around 30 character animations (plus a couple other data files) in 600 KB. The original PNG files are around 3 MB.

elund
08-07-2003, 07:37 AM
Originally posted by hanford_lemoore
MO3: How do you author these files? With MP3 you can basically record whatever you want, whether you're using a microphone, professional music euipment, or Reason. If I were to go with MOD, what do I compose my music in? I use Modplug Tracker (http://www.modplug.com) for composing the mod (I'm using Impulse Tracker format). My actual music, however, is composed in Sonic Foundry's Acid (http://www.sonicfoundry.com/PRODUCTS/NewShowProduct.asp?PID=704). When I'm done, I render the loops at the right pitch and tempo, and then recompose them in MPT. I'm using it essentially just as a sequencer. For reference, Mod, MO3, and Midi were also previously discussed in this thread (http://www.dexterity.com/forums/showthread.php?s=&threadid=56).

Larry Hastings
08-07-2003, 10:50 AM
Next up on Oprah: Larry Hastings answers your questions!


Some notes on PNG

I just did some reading, and it's not as multipurpose as I'd thought.

I had the idea that PNG supported 16-bit palettes, which I figured would make PNG real competitive with other file formats. But it doesn't. I don't know where on earth I got that idea.

Similarly, PNG doesn't support 16-bit color (as I had stated above--my apologies, everybody).

PNG supports the following formats for color images: either 8-bit palette, or truecolor (8 or 16 bits per color). It also supports various grayscale types, and has a couple options for specifying alpha channels.


More Pngcrushers

While reading this morning, I discovered plenty more pngcrush-like utilities. All of 'em are mentioned on the PNG home page's list of image-conversion applications (http://www.libpng.org/pub/png/pngapcv.html).

The three additional ones I played with were pngrewrite (http://pobox.com/~jason1/imaging/), OptiPNG (http://www.cs.toronto.edu/~cosmin/pngtech/optipng/), and PNGOUT (http://advsys.net/ken/utils.htm).

pngrewrite will allow you to alter the image, changing the bit depth and the like. (All pngcrush does is attempt to recompress your existing image.) However, you'll still need pngcrush.

PNGOUT is a lightweight when compared with these other tools. However, it claims to have a unique approach to compression, which helps some folks realize a 5% or even 10% improvement over pngcrush. Me, I did see file sizes drop, but not by that much... say 1% or 2%. But it's somethin'. I guess.

Of the utilities I examined, OptiPNG is the big win. It's like pngcrush and pngrewrite combined together, and then some. It seems to make files at least as small as pngcrush, and maybe smaller. I think I'm gonna switch.


Fenix Down: gawrsh! Your message is what prompted the above notes on PNG. You're right, apart from images that have color gradients (where banding would make it look awful), 16-bit TGA files will often win. And you're right, they compress amazingly well. I experimented and made a 500k TGA file, and it compressed to 50k! Another trick added to my book.

Also, you make an excellent point that the actual size in the installer is what matters, and you can closely simulate that by zipping the files and comparing that. That's why I was so enthused to see that MO3 files compress better than the original MOD files, not to mention being much smaller to begin with.

However, to hoist you on your own petard :) UPX and other EXE self-compressors aren't that big a win when talking about an installer. I tried UPX on my game, then zipped up both the UPX and non-UPX executables, and the UPX version was 10% smaller (40k). Every little bit helps, and I wouldn't tell interested folks not to do it, but it's not as big a win as many of these other techniques we're discussing.


elund: I dimly recall reading about OXM. I'll give it a try sometime. However, I'm happy with MO3 for now.


luggage: I hear OGG is better than MP3 too, but I didn't have any data to back it up so I kept my trap shut. In any case OGG is not patent-encumbered, and is as free as air. So I prefer OGG too.


hanford_lemoore: Yes, as elund says, you need a "tracker". The best resource I know of for trackers for Win32/DOS is at Maz-Sound (http://www.maz-sound.com/trackers.html). He lists 'em all. I've never been able to figure it out, myself, but you can easily convert General MIDI files into MOD files (specifically, IT files) using ModPlug (http://www.modplug.com/index.php3) (which is freeware).


Fariz: You make a good point. PNG does use the exact same compressor (ZLib) as many installers, including NSIS.


papillon: I'd agree with Mr. Kronos, I'd be dubious that anything under a meg would have enough goodness to be worth my time.


Raptisoft: The links straight to the [url=http://sourceforge.net/project/showfiles.php?group_id=1689]download page (]official Pngcrush homepage[/url), which runs fine on Windows and even understands long filenames. That's what I used. However, the GnuWin32 (http://gnuwin32.sourceforge.net/) project has a Win32 native port of pngcrush on their download page (http://gnuwin32.sourceforge.net/packages.html).


ggambett: Hats off to your mastery at image compression. I doubt I'll get my media payload anywhere near that small.


Again, I say: phew!

ggambett
08-07-2003, 11:09 AM
Originally posted by Larry Hastings
ggambett:Hats off to your mastery at image compression. I doubt I'll get my media payload anywhere near that small.
Heh, thanks :) but I'm not doing anything too special, really. Just what I said. And there is room for improvement, too : by packing several small images into one rectangle, I'll be taking advantage of 1) reduced overhead in the resources file, 2) less overhead due to less image headers and palettes, 3) much better lzw-based (ie zlib) compression because of the greater redundancy of the images (for example, putting all the "arm" images in one big image makes it possible to compress them better than individually).

BTW, I've been experimenting with different rectangles-inside-rectangle packing algorithms but none gave satisfactory results. Any pointers?

Punchey
08-07-2003, 01:07 PM
I just wanted to say THANKS, LARRY! It gives a person one of those warm-fuzzy feelings inside (that most of us alpha-male mucho macho masculine types don't want to admit we have from time to time) just to see someone pour this much effort into putting out helpful information for others. It is this exact spirit that brought me to Dexterity in the first place, and you have done your part here to continue that tradition. Hats off to you are your kind!

I must say, I've never been participant of a forum before in which a good portion of the threads posted were not to do with questions, but rather to do with unsolicited answers and helpful tidbits. This is unprecedented in my experience. So hats off to Steve for starting such an amazing community! It is the generous and helpful spirit with which you founded this community that has spawned such a flood of other's helpfulness and kindness. Thanks to you all and know that your contributions are very much appreciated!

Fenix Down
08-07-2003, 04:17 PM
Originally posted by Larry Hastings
You're right, apart from images that have color gradients (where banding would make it look awful), 16-bit TGA files will often win. And you're right, they compress amazingly well. I experimented and made a 500k TGA file, and it compressed to 50k! Another trick added to my book.


Hmm I think overall you gave me credit for more things than I've actually said but I'm not going to complain. :) Just a note on TGAs -- how much it compresses depends on image data, but I've also noticed just now that uncompressed TGAs ZIP to a smaller size than compressed ones by around 10% (maybe more in some cases). Just another thing to keep in mind. And you're right, I tried UPX and it shaves only a little more (after UPXing it and Zipping) off what you get from just Zipping a DLL or EXE. Still I think it makes the EXE/DLL harder to mess with unless the cracker knows it's UPX compressed in which case they can just decompress it. :)

Karukef
08-08-2003, 12:42 AM
Although I could probably go on and on on this topic, I don't have time. I just want to point out that OGG is better than mp3. So don't even think about that anymore.

ggambett
08-08-2003, 05:13 AM
Originally posted by Fenix Down
And you're right, I tried UPX and it shaves only a little more (after UPXing it and Zipping) off what you get from just Zipping a DLL or EXE. Still I think it makes the EXE/DLL harder to mess with unless the cracker knows it's UPX compressed in which case they can just decompress it. :)

There's a general lesson here. When possible, use a compression algorithm designed specifically for the data you're compressing. A general compressor (ie ZIP) can't make any assumptions on its data. Specific compressors can. For example, JPEG knows that your eye is better at differentiating changes in the luminance than in the hue of a color, and takes advantage of it. MPEG knows that your brain interpolates images over time. MP3/OGG know about the limitations of the way we hear. Compare JPEG/MPEG/MP3/OGG with ZIP! In a similar way, UPX knows what's inside an executable and therefore what can be safely omitted. ZIP, on the other hand, must save and restore every bit exactly.

Larry Hastings
08-08-2003, 12:18 PM
ggambett: Agreed 100% about the "larger point here". I actually tried to make that point in the original posting, but it got stripped out during the long editing session. I guess I thought it diluted my other points, or something.

Two more discoveries on my road to maximal compression and minimal download size.

First of all, I emailed one of the principal PNG developers and asked why it didn't support 16-bit palette support. He said it was a question of keeping the PNG standard strictly to features they thought were necessary, "and TIFF already supported wide palettes". Hey! Unfortunately, none of the image editor programs I have handy supports 16-bit palettized TIFF files (I even downloaded a demo of DeBabelizer, and it didn't seem to hack it), so I can't test this theory out. Which reminds me--Fenix, what program do you use to convert to TGA?

Second of all, I discovered that 99% of the time, you can convert your textures to a 256-color palette and never know the difference. And that definitely will drop your image sizes down.

Jeff Evertt
08-08-2003, 01:15 PM
First off, thanks for starting off this thread Larry, I've found it quite informative. I haven't done much work with music.

I've released one shareware game thus far and it didn't include any music, just sound fx. I'm working on a second one now and have been considering paying someone to create some music.

What do you guys think is a good song length to avoid too much repetition? Any how many songs do you need to have a good variety?

One comment about your original post, Larry (and this only really applies if you're making a 3D game). I think that DXT is completely valid choice even if you are supporting older cards. Most cards made in the last 3 to 5 years do support it and format converting on load in the case of a non-support isn't a big deal. DXT3/5 are 1 byte per texel and DXT1 is half of that. The image quality lose is sometimes an issue, but rarely.

I do agree that to minimize install, you could do better with something like jpg compression. But, converting to DXT at install could be slow.

-Jeff
www.evertt.com (http://www.evertt.com)

Fenix Down
08-08-2003, 02:02 PM
Originally posted by Larry Hastings

Which reminds me--Fenix, what program do you use to convert to TGA?

I'm using Paint Shop Pro 8 right now, but previous versions should work fine as well.

oNyx
08-08-2003, 02:04 PM
Originally posted by ggambett
[...]BTW, I've been experimenting with different rectangles-inside-rectangle packing algorithms but none gave satisfactory results. Any pointers?

Have you tried to save only the differences? :)

eg: first rectangle is complete... second rectangle only contains the differences compared to #1... third rectangle only contains the differences compared to #2

That way you should get much bigger areas for LZW to run trough ;)

---

The filesize drop produced by pngcrunch depends on the image size itself. If the header itself is 20% of the filesize, the chances will be low that it can be more squished.

So throwing each set of sprites into one png (as ggambett already mentioned) can save alot. Only one header and again bigger and better places for compression algorithms.

Henrik
08-08-2003, 02:07 PM
I don't understand why you guys irritated about png or tga not supporting 16-bit palettes.. if you do want them, why not roll your own file format or extend one of them? No one's stopping you, and it won't probably take you more than an hour or two to implement if you really need it..

Hydroaxe
08-08-2003, 03:42 PM
Originally posted by Fenix Down
Hmm I think overall you gave me credit for more things than I've actually said but I'm not going to complain. :) Just a note on TGAs -- how much it compresses depends on image data, but I've also noticed just now that uncompressed TGAs ZIP to a smaller size than compressed ones by around 10% (maybe more in some cases). Just another thing to keep in mind. And you're right, I tried UPX and it shaves only a little more (after UPXing it and Zipping) off what you get from just Zipping a DLL or EXE. Still I think it makes the EXE/DLL harder to mess with unless the cracker knows it's UPX compressed in which case they can just decompress it. :)

I absolutely love this thread. There's lots of interesting information in it and it answered a question I had... I think. So... uncompressed TGA's (it's a format my programmer wants to use) will zip to a smaller size then already compressed ones. So then can you get more compression by that method than turning your graphics into PNG's and trying to zip them? Is that correct?

ggambett
08-09-2003, 01:37 PM
Originally posted by oNyx
Have you tried to save only the differences? :)

eg: first rectangle is complete... second rectangle only contains the differences compared to #1... third rectangle only contains the differences compared to #2

That way you should get much bigger areas for LZW to run trough ;)
Sorry... I don't know what are you talking about :(

Either I don't get it, or I didn't express myself correctly. I have a set of images, and I want to paste them, without overlapping, inside the smallest possible rectangle that can hold them. I really don't understand what you said :(

oNyx
08-09-2003, 04:25 PM
Originally posted by ggambett
Sorry... I don't know what are you talking about :(

Either I don't get it, or I didn't express myself correctly. I have a set of images, and I want to paste them, without overlapping, inside the smallest possible rectangle that can hold them. I really don't understand what you said :(

Oh. Well... yes, I hadn't understand you corretly. I thought you were talking about different animation frames stored in one image.

The thing you're asking for sounds usefull... but infact it isn't. The compression of png or gif doesn't care if there are 5 black pixels in a row or 100 (it get's stored like x times color y). Therefore the saving equals 0.

Larry Hastings
08-09-2003, 06:10 PM
Jeff Evertt: Well, that depends on what is the lowest-end hardware you're targeting. I don't have anything handy that says what cards do and don't support DXT3/5. (Again, nVidia cards as recent as the GeForce 3 had problems with DXT1, so be cautious.) But in other threads on this board, people are talking about DirectX 8 being too agressive in terms of requirements of the user's machine. And that's nearly three years old, and it's a free download from Microsoft, and every retail game shipped in the last 2+ years has included it on the CD. So I am making at least a vestigal attempt at old-hardware-compatibility, in hopes of having a wider audience for my game.

As far as I'm concerned, the big win with DXT is saving video memory. If I knew I could encode all my textures in DXT and it'd work on all my client's hardware, I'd do it, 'cause then I'd be in absolutely no danger of texture thrashing. As it is, I... fret.

(Speaking of: can anyone tell me a good way to tell how much texture memory I've used? What I planned to try was calling GetAvailableTextureMem() at the beginning of my game, then polling it after EndScene() calls and noting the widest delta.)

As for song length, it depends on the length of your levels. First of all, in my game I switch music every time I switch levels. And if a level lasts two minutes, no point in going over two minutes, is there? If you're going to play the same song for a long time, it's gonna get tiresome regardless. I'd definitely say at least three minutes. Five would be better. Maybe a better approach would be to just switch tracks when a song runs out?

Hydroaxe: Part of the point on uncompressed TGAs was that he also converted them to 16-bit color. And I believe he meant unpalettized 16-bit color, so it's either 565 or 555 with one bit of alpha. You should try this out and make sure this format won't goof up your images; it generally won't be noticable, but it'll cause banding in gentle color ramps.

In any case, Fenix Down was comparing his bitmaps against PNG-24 files, where every pixel was stored with full 24-bit color. And in those circumstances, 16-bit TGAs will of course win. However, 8-bit palettized PNG files would beat 16-bit TGAs any day of the week, even without pngcrush. Of course, this can cause even more noticable artifacting than even 16-bit color. But so far it's worked great for me.

The moral of the story: you have plenty of options for storing your textures. If you really want to get your file sizes way way down, you'll have to experiment.

Henrik: I think I'm the only person bemoaning the lack of 16-bit palette image formats. And you're right, I could roll my own. But sadly I've got a game to ship, and I fear it would take me a lot longer to write one than your estimate.

ggambett: Is what you're talking about similar to "take a collection of many different Tetris shapes and fit as many of them into a arbitrary-sized square as possible"? If so, that problem is equivalent to the Knapsack Problem, and it is what we Computer Scientists call "NP-Complete". In other words it is Hard.

And that's the limit of what I know. If there is such a squish-bitmaps-together tool out there, I don't know of it. On the other hand, how would you get them back out again? You'd need code to pull 'em apart, and data to tell you how to pull 'em apart, and... and you might well wind up with a larger installer than if you hadn't bothered with all that ruckus.

oNyx: Are you sure? What you're talking about is "Run-Length Encoding", or RLE compression. But PNG uses Zlib (LZ77) for its compression, and GIF uses LZW.

I did a quick web search, and found that you can compress GIF files using either RLE or LZW. However, this appears to be quite rare, as RLE-encoded GIFs aren't as small as LZW-encoded GIFS. I looked at the libpng source code, and at the PNG technical documentation, and it doesn't appear to support RLE at all.

oNyx
08-09-2003, 06:42 PM
Originally posted by Larry Hastings
oNyx: Are you sure? What you're talking about is "Run-Length Encoding", or RLE compression. But PNG uses Zlib (LZ77) for its compression, and GIF uses LZW.

I did a quick web search, and found that you can compress GIF files using either RLE or LZW. However, this appears to be quite rare, as RLE-encoded GIFs aren't as small as LZW-encoded GIFS. I looked at the libpng source code, and at the PNG technical documentation, and it doesn't appear to support RLE at all. [/B]

Yes, what I described sounds like RLE but I didn't really ment that algo.

What I intended to say is, that compression schemes usually check what's repeated. So if you have repeatative data it will get crushed very well.

Therefore it doesn't pays off if you "recycle" unused places in images.

Hydroaxe
08-10-2003, 12:42 AM
Originally posted by Larry Hastings
Hydroaxe: Part of the point on uncompressed TGAs was that he also converted them to 16-bit color. And I believe he meant unpalettized 16-bit color, so it's either 565 or 555 with one bit of alpha. You should try this out and make sure this format won't goof up your images; it generally won't be noticable, but it'll cause banding in gentle color ramps.

In any case, Fenix Down was comparing his bitmaps against PNG-24 files, where every pixel was stored with full 24-bit color. And in those circumstances, 16-bit TGAs will of course win. However, 8-bit palettized PNG files would beat 16-bit TGAs any day of the week, even without pngcrush. Of course, this can cause even more noticable artifacting than even 16-bit color. But so far it's worked great for me.

The moral of the story: you have plenty of options for storing your textures. If you really want to get your file sizes way way down, you'll have to experiment.



I just love this thread. I took a sprite frame from my upcoming game, (a 16-bit TGA) and zipped it. Then I did some basic testing with Optipng and Pngout. I got these results:

A zipped Optipng image : 23.3% smaller than zipped TGA
A zipped Pngout image: 25.7% smaller than zipped TGA

What is a palettized PNG? The original PNG that I converted from I had saved from Photoshop 6 as an 8-bit PNG with 64 colors selected for the output. It didn't need more than that.

I also saved an 8-bit PNG with 256 colors to see if I was wasting any time by initially reducing colors in Photoshop before converting with Pngout, but when I reduced it in Pngout, it came out larger than the zipped TGA. NOTE: I didn't try any of the tweaking parameters in either program though.

StAn
08-10-2003, 07:20 AM
Originally posted by ggambett
BTW, I've been experimenting with different rectangles-inside-rectangle packing algorithms but none gave satisfactory results. Any pointers?

You have probably already thought about this, but I simply sorted the small rectangles by height and then fit them into the big rectangle from left to right, top to bottom, for my font textures generator. I spent quite some time thinking about the subject without finding any better solution, and more importantly it works well enough for me as it is :p
Grab here: http://www.guru-meditation.net/images/picoftheweek/Font_Maker_03-Nov-02.png

Larry Hastings
08-10-2003, 07:25 AM
Hydroaxe: By palettized PNG, I mean a PNG file that stores color information in a palette (color lookup table) instead of with each pixel. A palettized PNG file can have no more than 256 unique colors in the whole file, but is markedly smaller than the equivalent non-palettized PNG file.

If your PNG file only has 64 unique colors, then according to the docs OptiPNG will convert it to use a palette (if your file isn't already using one).

One more tool for you to play with: TweakPNG (http://entropymine.com/jason/tweakpng/), a PNG file dumper / tweaker. This lets you see the chunks stored inside the PNG file. Using this, you'll be able to see for sure whether or not a PNG file is palettized. Drag & drop your PNG file onto the TweakPNG main window, then take a look. If there's a PLTE chunk in there, the file is using a palette.

Another way to check is to double-click on the IHDR chunk (always the first chunk). If the "color type" is 3, the file is using a palette (and there'd better be a PLTE chunk in the file too). If the file is anything else (usually either 2 or 6), the file is not using a palette.

Karukef
08-10-2003, 11:48 AM
Image compression
I looked into some alternatives to image compression, namely lossless image compressors. These programs typically compress a single bitmap. The bitmap to compress must usually be a simple type such as .bmp.

I tested two compressors, uhic and bmf against optimized png's for non-palletized images and the results were worth notizing. Both uhic and bmf gave noticably smaller files (such as 338kb vs png's 468kb for a 800x600 non-photorealistic image).

I wish I could compress all my non-256 color/non-photorealistic images with these kinds of programs, but the best ones were all under a "this program is a test, use at own risk and cannot be used for commercial purposes". I couldn't even find a homepage for either the bmf nor the uhic compressor. Too bad. I might send the authors an e-mail about their lisence restrictions, but who knows.

General Compression
It is also worth pointing out that the leading edge general compressors are a fair bit ahead of the more known ones. For example, the bzip2 testcase on http://artst.narod.ru/ gives 20% bigger files than the test winners such as Uharc. Interestingly enough, these general compressors will compress a non-photorealistic image almost just as well as the image compressors I mentioned above. So to achieve minimal filesize, all there is to do is either lossy-compress or simply store the image as a bitmap, preferrably with a pallette (regular or wide). The general purpose compressor will then do the rest. This shows how important the final compressor can be.

Hydroaxe
08-10-2003, 03:47 PM
Originally posted by Larry Hastings
Hydroaxe: By palettized PNG, I mean a PNG file that stores color information in a palette (color lookup table) instead of with each pixel. A palettized PNG file can have no more than 256 unique colors in the whole file, but is markedly smaller than the equivalent non-palettized PNG file.

If your PNG file only has 64 unique colors, then according to the docs OptiPNG will convert it to use a palette (if your file isn't already using one).

One more tool for you to play with: TweakPNG (http://entropymine.com/jason/tweakpng/), a PNG file dumper / tweaker. This lets you see the chunks stored inside the PNG file. Using this, you'll be able to see for sure whether or not a PNG file is palettized. Drag & drop your PNG file onto the TweakPNG main window, then take a look. If there's a PLTE chunk in there, the file is using a palette.

Another way to check is to double-click on the IHDR chunk (always the first chunk). If the "color type" is 3, the file is using a palette (and there'd better be a PLTE chunk in the file too). If the file is anything else (usually either 2 or 6), the file is not using a palette.

Okay, so then what I'm doing in Photoshop is considered as asigning a palette because I see both of the chunks you mentioned when using TweakPNG. I also found that I could delete two chunks I don't need, but the size turned out to be exactly the same number of bytes as when I compressed with PNGout.

ggambett
08-10-2003, 04:57 PM
Larry Hastings : Is what you're talking about similar to "take a collection of many different Tetris shapes and fit as many of them into a arbitrary-sized square as possible"? If so, that problem is equivalent to the Knapsack Problem, and it is what we Computer Scientists call "NP-Complete". In other words it is Hard.
I Computer Scientist know what does NP mean, but I never heard of the Knapsack Problem. That's the info I needed, thanks :)

Larry Hastings : On the other hand, how would you get them back out again? You'd need code to pull 'em apart, and data to tell you how to pull 'em apart, and... and you might well wind up with a larger installer than if you hadn't bothered with all that ruckus.
The code to pull them apart is done already, in the form of Image::grab(). I just need the data, which yes, is an additional data file. At 8 bytes per image (2 bytes x 4 coordinates) I doubt it would increase the download size.

I've made some informal tests (ie taking all these images and pasting them together with GIMP) and I could save around 40% of the size. So I still think it's worth trying.


oNyx : What I intended to say is, that compression schemes usually check what's repeated. So if you have repeatative data it will get crushed very well.
Therefore it doesn't pays off if you "recycle" unused places in images.

That's true for RLE, which scans for repetitions of one value. LZW and other dictionary based algorithms check for repeating patterns. The more repeating a pattern is, the better it will compress. Therefore my savings would come from having several similar images compressed using the same dictionary and thus exploiting the repetitions of some patterns across several images, not from reusing empty space.


StAn : You have probably already thought about this, but I simply sorted the small rectangles by height and then fit them into the big rectangle from left to right, top to bottom, for my font textures generator. I spent quite some time thinking about the subject without finding any better solution, and more importantly it works well enough for me as it is
That's a workable solution and gives pretty good results when the images are similar in size, but the wasted space is still too high if the images aren't so regular. I also tried constructing a BSP tree (or BPP, as it's a Binary Plane Partition) and I didn't like the results.

Now I think about it, the wasted space would be wasted if encoded as JPEG, but not if encoded with a dictionary based algorithm. So this one might be a good solution after all.

Larry Hastings
08-10-2003, 08:43 PM
Originally posted by Hydroaxe
Okay, so then what I'm doing in Photoshop is considered as asigning a palette because I see both of the chunks you mentioned when using TweakPNG.That means the original file you're editing has a palette. You can explicitly control the color mode in Photoshop 6 under Image| Mode | .... All the different modes are there; the ones you're most likely to be interested in are "Grayscale", "Indexed Color", and "RGB Color".

By the way, I was wrong about OptiPNG possibly palettizing your image if it had < 256 unique colors. OptiPNG doesn't do that. I was confused by the documentation saying "palette reduction"; apparently this means removing redundant / unused colors from the palette and nothing more. I have yet to see a command-line utility that will convert PNG-24 files into PNG-8 files; I don't doubt there is one (and maybe someone could steer me to one that runs under Windows?). For now I'll have to do it by hand in Photoshop.I also found that I could delete two chunks I don't need, but the size turned out to be exactly the same number of bytes as when I compressed with PNGout. Did you save the file after you deleted the two chunks? You need to press Ctrl-S to save your changes in TweakPNG. Half the time I forget to save, and it doesn't remind me to either.

You should be able to do both. PNGOut only affects how well the IDAT chunk is compressed. It doesn't have anything to do with those unnecessary chunks Photoshop likes to add in.

On the other hand, you're gonna save next-to-nothing by removing those chunks. I bet both of 'em together are less than 50 bytes, and the overhead for a chunk is only 12 bytes (4-byte ID, 4-byte length, 4-byte CRC). So don't kill yourself with trying to expunge them from your assets. But hey, if you're having fun, go crazy-go-nuts.

Hydroaxe
08-11-2003, 01:42 PM
Yes, I did save the file after removing the Photoshop chunks and was still able to view it after. PNGout does compress the IDAT chunk like you say, but it also appears to remove the two Photoshop chunks "gama" and "text" because they are not shown when I drag the image into TweakPNG. I verified it again today just to make sure. It will be easier compressing images with PNGout because of this.

Henrik
08-12-2003, 02:19 AM
Here's a useful algorithm for packing many images into a big one (the example uses lightmaps, but of course any images could be packed). This is useful for packing sprites into textures or packing several images into one to improve compression.

http://www.blackpawn.com/texts/lightmaps/default.html

ggambett
08-12-2003, 06:08 AM
Originally posted by Henrik
Here's a useful algorithm for packing many images into a big one (the example uses lightmaps, but of course any images could be packed). This is useful for packing sprites into textures or packing several images into one to improve compression.

http://www.blackpawn.com/texts/lightmaps/default.html
The BSP technique I was talking about was an implementation of that article, and I didn't like the results. Maybe I did something wrong :confused:

Larry Hastings
08-12-2003, 01:21 PM
Originally posted by Hydroaxe
PNGout does compress the IDAT chunk like you say, but it also appears to remove the two Photoshop chunks "gama" and "text" because they are not shown when I drag the image into TweakPNG. You're right! PNGOUT also strips out gAMA and tEXt chunks.

You might be interested to know that the funny capitalization in those chunks is relevant. The PNG Specification (http://www.libpng.org/pub/png/spec/1.2/) has explicit rules (http://www.libpng.org/pub/png/spec/1.2/PNG-Structure.html#Chunk-naming-conventions) for upper- and lower-case in chunk names, and they are quite significant. For instance, if the first letter is upper-case, that means it is a "critical" chunk; if it's lower-case, it's an "ancillary" chunk. Ancillary chunks are "not strictly necessary in order to meaningfully display the contents of the file".

At a guess, PNGOUT simply removes all "ancillary" chunks it doesn't recognize. This could theoretically be trouble, if there was some chunk that really was important to you but was marked ancillary. For instance, the tRNS chunk specifies transparency information for palettized images. It is specified as "ancillary", but they are necessary for my game. Now, PNGOUT explicitly passes through tRNS chunks, so this isn't a problem. But hypothetically, there could be some other chunk marked as "ancillary" that I relied upon, and PNGOUT might automatically delete it.

However, in practice it sounds like what PNGOUT does is fine. The PNG format is very stable these days; the specification cited above is from 1999. I wouldn't worry about being dependent on exotic new chunks.


Oh, and ggambett: My apologies, but I now think I was wrong when I said your problem was equivalent to the Knapsack Problem. In Knapsack, the objects are weighted--some are more important than others. In your problem all the objects are of equal weight. This makes the problem a lot easier to solve.

How did you pick which textures to insert? If you didn't try it already, let me suggest a "greedy" algorithm. Sort all your images by the number of pixels they contain (x times y). Then walk the whole list from largest to smallest, adding each image that fits.

Also, what didn't you like about your results? Images compress quite well, particularly areas of wasted space. So I expect even a simple attempt at packing would be very nearly as good as some hypothetically optimal algorithm. If it was me, I would make a minimal effort at efficient packing and call it a day.

ggambett
08-12-2003, 01:47 PM
Originally posted by Larry Hastings
ggambett: My apologies, but I now think I was wrong when I said your problem was equivalent to the Knapsack Problem. In Knapsack, the objects are weighted--some are more important than others. In your problem all the objects are of equal weight. This makes the problem a lot easier to solve.

There is hope :)


How did you pick which textures to insert? If you didn't try it already, let me suggest a "greedy" algorithm. Sort all your images by the number of pixels they contain (x times y). Then walk the whole list from largest to smallest, adding each image that fits.

Yup, I did that :(


Also, what didn't you like about your results? Images compress quite well, particularly areas of wasted space. So I expect even a simple attempt at packing would be very nearly as good as some hypothetically optimal algorithm. If it was me, I would make a minimal effort at efficient packing and call it a day. [/B]

The results were clearly sub-optimal, I could produce a better packing by hand. But, as you say, even a sub-optimal packing will be an improvement.

Hydroaxe
08-12-2003, 08:51 PM
Originally posted by Larry Hastings
You're right! PNGOUT also strips out gAMA and tEXt chunks.

You might be interested to know that the funny capitalization in those chunks is relevant. The PNG Specification (http://www.libpng.org/pub/png/spec/1.2/) has explicit rules (http://www.libpng.org/pub/png/spec/1.2/PNG-Structure.html#Chunk-naming-conventions) for upper- and lower-case in chunk names, and they are quite significant. For instance, if the first letter is upper-case, that means it is a "critical" chunk; if it's lower-case, it's an "ancillary" chunk. Ancillary chunks are "not strictly necessary in order to meaningfully display the contents of the file".

At a guess, PNGOUT simply removes all "ancillary" chunks it doesn't recognize. This could theoretically be trouble, if there was some chunk that really was important to you but was marked ancillary. For instance, the tRNS chunk specifies transparency information for palettized images. It is specified as "ancillary", but they are necessary for my game. Now, PNGOUT explicitly passes through tRNS chunks, so this isn't a problem. But hypothetically, there could be some other chunk marked as "ancillary" that I relied upon, and PNGOUT might automatically delete it.

However, in practice it sounds like what PNGOUT does is fine. The PNG format is very stable these days; the specification cited above is from 1999. I wouldn't worry about being dependent on exotic new chunks.



Yeah, that is interesting. I was wondering what the wierd captalization meant. Thanks for all the info in this thread Larry. :)

damocles
08-15-2003, 05:52 AM
Having made the decision to switch from RLE TGAs to PNG files, I spent the last few days having major headaches trying to get the damn things to load. I now have that sorted (thanks to someone else's wrapper library :) ) only to find that I don't have any software able to write PNG files with alpha channels!

I've tried both Photoshop 7.0 and Paint Shop Pro and neither seem to want to store any alpha channel data. I downloaded a free plug-in for Photoshop called SuperPNG which claimed to do the job, but doesn't.

Can someone tell me how I can save 32-bit images, preferrably in photoshop, in PNG format?

Larry Hastings
08-15-2003, 02:27 PM
damocles: I've never had any trouble. If the image mode is RGB Color, and there genuinely is transparency poking through (that Photoshop white/light-gray checkerboard), then those pixels show up as transparent when DirectX loads up the image.

If you wanted to be 100% certain, use the "Save For Web" function, pick PNG-24, and make sure the "Transparency" box is checked.

That any help?

damocles
08-15-2003, 02:49 PM
I have already tried that. And I'm sure it's not the code because I'm using someone else's code and they say it loads alpha channels no problem.

When I load it up in photoshop after saving, there is no alpha channel data at all. In game, no alpha channel.

Does it have to be on a layer so that transparency is showing through? I use the actual alpha channel to make more detailed transparencies. Works fine for TGAs but not for PNG.

Larry Hastings
08-15-2003, 03:13 PM
I don't know about TGA files. But when saving PNG files from Photoshop, the final file will be exactly the same as if you Flatten Layers (discarding invisible layers).

Try this. Make a new image, create a new layer, then delete the "Background" layer. You should now have a PSD containing no pixels of any kind.

Now draw a blob in the middle of the current layer, and set the layer's Opacity to 50%. You should now see the checkboard peeking through a little bit. Use "Save To Web", PNG-24, with Transparency checked. Bingo.

I did that, and the results are here:
http://www.midwinter.com/~lch/transparency.png
Those aren't gray pixels, those are black pixels that are 50% transparent.

You can download the PSD file here:
transparency.psd (http://www.midwinter.com/~lch/transparency.psd)
(right-click and choose "Save As...")

Got the hang of it now?

damocles
08-15-2003, 04:49 PM
It seems that PNG doesn't have alpha like I thought it did. The file you provided has an alpha value that affects the whole image. That's not an alpha channel, that's simply a transparency setting. That's no use for masking techniques using alpha data.

I think I'm going to stick with RLE TGAs until I can find a better option :/

Larry Hastings
08-15-2003, 05:05 PM
I don't see how you jumped to that conclusion, but you're incorrect. PNG supports per-pixel transparency. The file I provided is a "32-bit image" as you put it. Take a closer look--the edges around the main blob are soft, with many transparency values between 0 and .5.

For your edification, here is another image, where I doodled on two more layers: pure red at 36% and a dark green (#00A651) at 74%.

http://www.midwinter.com/~lch/transparency.2.png

damocles
08-16-2003, 02:26 AM
In which case the trouble must lie somewhere in Photoshop or the image loading code. When I load that image into photoshop, I don't get alpha channel data, I get an image that is the doodle on a layer with varying degrees of transparency. In a way this is alpha data, but it doesn't seem to translate into 32-bit alpha channel texture data for some reason.

Has anyone here successfully loaded a PNG file into a 32-bit openGL texture?

Larry Hastings
08-16-2003, 03:07 AM
I'm not using OpenGL, so I can't help you there. But I can assure you that Photoshop is working fine... at least, it is over here.

I'm not sure what you're expecting. In a PNG-24 image specifying transparency, each pixel has its own red, green, blue, and alpha values. How is this different from "alpha channel data"? Can you post a counterexample, which I could look at in Photoshop, that would illustrate the difference?

damocles
08-16-2003, 03:41 AM
Well, I found the problem - if it is a problem. More of a quirk of photoshop.

When producing images for TGAs (and most other fiel formats), you use the alpha _channel_ to set the transparency data that will be saved in the file. When using PNGs, you have to construct the images on a layer with an empty space underneath showing through to store the alpha data. Seems like a funny way of doing it to me, but I was always brought up to use alpha channels :/

So now I am going to have to manually go through every transparent image in my game and convert it to using layer transparency instead of alpha channel....unless I can find a command to convert alpha channels to layer transparency....

Thanks for your help.

Edit: managed to construct a rather fiddly PS action sequence to automate most of the process, which basically involved selecting channels, copying layers, selection modification, deleting layers, etc. But yay, I have PNG teeny tiny files - that should save me about 700k :D