FSAA implemented as a fragment shader

Hi,

I would like to know if it would be theseable to write a fragmentshader that performs fullscreen antialiasing.

Performance is not an issue. I am writing a comparison about antialiasing methods and I would like to have actual code for the different methods. If this is not really doable in opengl, I’ll move along to supersampling using the accumulation buffer.

My basic idea is to render into a bigger sized sperate framebuffer. Then let the fragmentshader fill my geometry with color. Then have a second fragment shader read the fragment’s colors and compare it to its neighbours, calculation intermediate colors. And lastly blitting the sperate framebuffer to the smaller windowmanager’s framebuffer.

Is there a way to access the fragment’s colors? Or do I have to completly render my scene and access the pixels through gl_readPixels()?

First of all, full-screen anti-aliasing and supersample anti-aliasing are the same thing.

Second, the only proper way to do true supersample anti-aliasing is (as far as I know) to turn it on in your driver settings. You COULD emulate it by rendering to a texture and then doing a second render pass that renders a square over the entire screen, and then samples the texture in the fragment shader. If you are on OpenGL 4.2, you could use Image Load Store in your second pass to directly access texels without going through the sampling hardware.

No, not really. The latter is one possible implementation of the former. Along with MSAA, FXAA, MLAA, GPAA, GBAA, SDAA, etcetc.

Second, the only proper way to do true supersample anti-aliasing is (as far as I know) to turn it on in your driver settings.

For the system (window) framebuffer possibly, but for off-screen multisample render targets you can request through the OpenGL API that SSAA-based rendering be enabled. The GL lingo for this is per-sample shading (see ARB_sample_shading), as opposed to per-pixel shading (which is what you get with MSAA or when the render target is single sampled).

I may well be wrong, but I was under the impression that MSAA, SSAA, FXAA, etc are all distinct. Of course Wikipedia isn’t always the best source, but here’s where I got the idea that FSAA and SSAA are synonymous.

For the system (window) framebuffer possibly, but for off-screen multisample render targets you can request through the OpenGL API that SSAA-based rendering be enabled. The GL lingo for this is per-sample shading (see ARB_sample_shading), as opposed to per-pixel shading (which is what you get with MSAA or when the render target is single sampled).

I wasn’t aware of that, I stand corrected. For the OP, here is a link to more info on multisample framebuffers.

In light of that, would the best way to do this be to render to a multisample FBO and then blit it to the main framebuffer?

You can do either. If you render to an offscreen multisample framebuffer (with SSAA, MSAA, or whatever), then you can read back the individual subsamples with texelFetch() in a fragment shader to get a single sample texture you can easily read back.

Yes, in terms of algorithms. Though you can combine these algorithms together when performing FSAA .

Of course Wikipedia isn’t always the best source, but here’s where I got the idea that FSAA and SSAA are synonymous.

Yeah, I think that article is slightly confused. FSAA (unlike the rest) isn’t a specific algorithm. It stands for “full screen antialiasing”, so it encompasses all algorithms which implement some form of full-screen antialiasing.

For instance, here’s the output of NVidia’s FSAA mode list dump, which can be applied to GLX windows:


> nvidia-settings --query=fsaa --verbose
...
    Valid 'FSAA' Values
      value - description
        0   -   Off
        1   -   2x (2xMS)
        5   -   4x (4xMS)
        7   -   8x (4xMS, 4xCS)
        8   -   16x (4xMS, 12xCS)
        9   -   8x (4xSS, 2xMS)
       10   -   8x (8xMS)
       11   -   16x (4xSS, 4xMS)
       12   -   16x (8xMS, 8xCS)
       14   -   32x (8xMS, 24xCS)

Here you can see that some of these FSAA modes use MSAA, and some use a combination of it with CSAA or SSAA

That makes sense, thank-you for the correction.

Well i did mean “traditional” supersampling (rendering at higher resolutions), when i wrote FSAA. So i wasnt using the terms correctly.
I wanted to distinguish between higher resolution super sampling and “shifted scene” supersampling. So far I havent found a different name for the later.

thanks for the hint. this was what i was looking for!
so far i managed to render to and read back from a texture, but I came across two questions:

  1. How do texelFetch() and texture() differ? The official description says the later is for texels and the former is for a single texel. But both methods return gvec4. So how does texture() return multiple texels? Or what exactly does it return?
  2. For my prove of concept I fed gl_FragCoord to texelFetch, to get an exact copy of a texture. This works, but isn’t texelFetch ought to get absolute coordinates, instead of texelFetch’s relative coordinates?

ok, so the next thing will be to actually implement a supersampling algorithm.

[QUOTE=sektion31;1257701]Well i did mean “traditional” supersampling (rendering at higher resolutions), when i wrote FSAA. So i wasnt using the terms correctly.
I wanted to distinguish between higher resolution super sampling and “shifted scene” supersampling. So far I havent found a different name for the later.[/QUOTE]

Ok. Not sure if there’s a standard name, but when I’ve tripped over it it’s called MultiPass Accumulation Buffer AA. Conceptually very similar, but you don’t take the hit of storing all the supersamples, and filtering is done progressively as-you-go. But you do take the hit of passing the geometry down the pipe multiple times.

How do texelFetch() and texture() differ? … how does texture() return multiple texels? Or what exactly does it return?

Think of texelFetch() as bypassing texture filtering. You address a specific sample in a specific texel, and that’s what you get. texture() honors texture filtering and wrapping modes, potentially sampling a number of texels, and filters (averages) them down to give you an average texel value.

isn’t texelFetch ought to get absolute coordinates, instead of texelFetch’s relative coordinates

Not sure I understand. For a screen-size texture, you can address individual pixels/texels directly through gl_FragCoord, which are absolute relative ot the resolution of the window (presuming your viewport fills your window): (0,0)…(xres,yres):

texelFetch( tex, ivec2( gl_FragCoord.xy ), … )

It sounds interesting, but I havent heard of games using it. I wonder why. Mabybe its more difficult with higher AA modes than 2x. Haven’t had a closer look yet.

[/QUOTE]

eh. bad typo… I meant: isn’t texelFetch ought to get absolute coordinates, instead of gl_FragCoord’s relative coordinates. Thats how I understand the opengl reference:

gl_FragCoord — contains the window-relative coordinates of the current fragment

I don’t know what you mean by absolute coordinates. Window coordinates are always relative to your window, not something large like the desktop.

Also re Multipass Accumulation Buffer AA, … it’s a very old technique – may even predate MSAA. It’s expensive, relative to the alternatives.

I was confused about two things. First I thought relative coordinates mean that they are stored as a real number between 0 and 1. And because of that I wondered why its ok to pass this as an ivec2 to texelFetch().

I get now, that relative is referring to the coordinates within a window. And I also understand that the following is allowed and can be used as a form of typecast:


int a;
a = int(1.7); // a equals 1

Ok so by now I have implemented a simple supersampling algorithm. It works fine for me, but maybe someone would like to comment on it. Especially if you think I didn’t implemented the anti-aliasing correctly.

How my code works:

  1. I draw to a texture that has a x-times higher resolution than my final windowFramebuffer.
    1b. FragmentShaderA creates the colors for this texture.
  2. I draw the same geometry with the same resolution to the windowFramebuffer, that has the final (smaller) resolution.
    2b. FragmentShaderB reads x2 texels from the texture and mixes them into one color per fragment of the windowFramebuffer.

[CODE=FragmentShaderB]#version 430 core
uniform sampler2D tex;
uniform int ssaa;
layout (location = 0) out vec4 color;
layout(pixel_center_integer) in vec4 gl_FragCoord;

vec4 samples[4][4];

void main(void) {

color = vec4(0,0,0,1);

if (ssaa == 2) {

	//samples[col][row];
	for (int i = 0; i < 2; ++i) {
		for (int j = 0; j < 2; ++j) {
	  		samples[i][j] = texelFetch(tex, ivec2((gl_FragCoord.x*ssaa)-i, (gl_FragCoord.y*ssaa)-j), 0);
	  		color += samples[i][j];
		}
	}
	color /= 4;

} else if (ssaa == 4 ){

	for (int i = 0; i < 4; ++i) {
		for (int j = 0; j < 4; ++j) {
	  		samples[i][j] = texelFetch(tex, ivec2((gl_FragCoord.x*ssaa)-i, (gl_FragCoord.y*ssaa)-j), 0);		  		
	  		color += samples[i][j];
		}
	}
	color /= 16;

} else {
	color = texelFetch(tex, ivec2(gl_FragCoord.x, gl_FragCoord.y), 0);
}

}



Pictures, because that is what graphical programming is all about, right? :)

.........NoAA..........|........2xSSAA.........|.........4xSSAA
[[img]http://abload.de/thumb/noxj3uru.png[/img]](http://abload.de/image.php?img=noxj3uru.png) [[img]http://abload.de/thumb/2xolu5t.png[/img]](http://abload.de/image.php?img=2xolu5t.png) [[img]http://abload.de/thumb/4xifu85.png[/img]](http://abload.de/image.php?img=4xifu85.png)

I'll need a few days to document this. After that I'll look into writing a Multipass Accumulation Buffer implementation.

Looks like a good start. A few comments. First, doesn’t look like you’re computing a filtered alpha value properly. Also, your “-i, -j” texcoord math makes it appear you might be passing in negative texcoords which might not be good, but depends on what quad you’re launching for the filtering.

One other thought. While you can implement this with a texture that’s scaled Nx larger and then downsample to a smaller texture, you can also render to a multisample texture using supersample rasterization (ARB_sample_shading), and then let either do the downsample yourself as you’re doing above (the 3rd texelFetch parameter ends up being the sample index within each texel) or just let the GL driver do it by using glBlitFramebuffer() to copy/downsample from this multisample texture to a single sample texture of the same resolution.