combining texture compression with palettes

To achieve even higher levels of compression, perhaps it is possible to compress already palettized texture data.

The data in question is 16 colors or less, often 8-11. A thought I have is to expand to a 256 entree palette which spaces the original entries to a palette of smooth gradient so that the lossy compression does not give undesirable artifacts when the resulting value is close but not exact.

The trouble is, all of the currently available texture compression schemes (dxt* etc etc…) are optimized for both luminance as well as color data. I am not sure how to represent the palette indexes appropriately. Maybe the EAC format would be a good bet, but is not yet widely supported.

Any thoughts on how I can apply this to dxt1 or etc1 formats? Any other ideas?

The DXT1 format use 64 bits for 4x4 = 16 pixels
(C0 and C1 rgb565 colors followed by 4x4 x2 bits = 64 bits)
(the bloc size is 4x4 pixels and use 2x16bits palette colors C0 and C1 + 4x4x2bits indices = 64 bits)

The rgb565 16bits extrems colors C1 and C0 can to be stored on a separates textures than the 32 bits chunk of 4x4 indices

We can use 3 separates textures texC0, texC1, texIDX instead only one “big” texture that store 64 bits chunks of 2x 16bits extrems colors palette followed by a 4x4 array of 2 bits indices

C0 and C1 colors can to be stored into two separates texC0 and texC1 atlas textures that can to be on lowers definitions than the fixed (with/4) x (height/4) definition used into the DXT1 sheme
(and they can perhaps to be compressed/decompressed in real time using something similar to JPEG for to be even smaller)

The texIDX texture can now independantly to be reduced from 32 bits to 8 bits with a different bloc size of 2x2 instead 4x4
(the 32bits bloc of 4x4 indices is only transformed into 4x 8 bits blocs of 2x2 indices)

Alls combinaisons of 2x2 indices can to be precomputed into a common palette colormap that have 256 entries
Into each color entry are stored the four indices of one combinaison of possibles 2x2 indices blocs
(on each palette entry, the red component is the topleft index, the green component is the top-right index, the green component is the bottom-left index and the alpha component is the bottom-right index of the 2x2 bloc indices)
=> we can in this way directly compute the reductions of 4x4 or 2x2 blocs sizes (and others bloc size such as 2x1, 1x2 or 1x1) into the fragment shader program with only the use of a common specific colormap for each handled blocs size and only one global/shared “indexmap” for to handle the color association between the top-left, the top-right, the bottom-left or the bottom right indice on this bloc
(I think that the 4x4 bloc size handling is possible via the use of 4x blocs of 2x2 pixels that are retrieved from the original 4x4 = 32 bits bloc with only the use of simples masks operations on the 4 individuals red, green, blue and alpha components at the fragment shader program level)

And we can too transform the RGB 4:4:4 colorspace into a YCbCr 4:2:2 colorspace (or a AYCbCr colorsapce if we want ot handle the alpha component) for to bypass the problem of the (C0, 2/3 * C0 + 1/3 * C1, 1/3 * C0 + 2/3 * C1, C1) linear ramp than cannot handle a color line with very differents Red, Green and Blue values on each texels on the same bloc

With the YCbCR colospace, alls interpolations can directly and independantly to be make at the Y, Cb and Cr components values
So, they are not limited to a simple r,g,b line between two colors because we use now differents lines interpolation for each of the Y=luminance, Cb=blue chroma, Cr=red chroma
(we can use an YCbCrA 4:1:1:1 colorspace if we want too can handle the alpha channel with no impact of the final compressed texture size)

=> with all this, I think that this can handle textures with something like only about 2 bits per pixel instead the 4 bits per texel needed with the DXT1 compression, so a 2x more effective compression level than the DXT1 compression scheme

==> I test to implement somes ideas and give you a report on some days if this seem to give goods results on the compression level and/or the quality/speed on textures that use this new compressed format