Minimum-error-boundary-cut : Need to "go from pixel to pixel"

Hey Guys,

I found myself interested in texture synthesis - especially texture quilting.
Therefore I’m overlapping random patches of the original texture to create a new texture. With the overlapping part of two neighboring patches I already got the error surface texture (it’s the square of the difference of overlapping pixels).

Now I want to make the Minimum-error-boundary-cut but I cant find anything that would help me go through the rows and coloumns of pixel in the overlapping regions.
Someone told me that it may not be possible :frowning:

I’m programming with Java on OS X with GLSL 3.3
I would appreciate your help!
Regards :slight_smile:

Now I want to make the Minimum-error-boundary-cut but I cant find anything that would help me go through the rows and coloumns of pixel in the overlapping regions.

You want to do this in a fragment shader, yes? If you declare your texture sampler uniforms as type ‘sampler2DRect’ you can (well, must) access them with texture coordinates in texels (instead of the usual normalized texture coordinates) - or you can use texelFetch to fetch a single texel. Not sure if that’s what you are looking for, you are not saying a lot about how you go about accessing the textures now :wink:

Thanks for your answer!

I forgot to mention that I’m using JOGL3 as well, maybe that is important…

Because it’s one of my first GLSL projects I’m rather unexperienced. Thats why I’m experimenting a lot :smiley:

So far i declared my textures as “uniform sampler2D” in the beginning of the fragment shader.
Shall i change that to “uniform sampler2DRect”? I’ve read in a few forums about texelFetch and other stuff but it isn’t quite understandable. With REct i get the non-normalized texel coords, right? Do you know an easy to understand example of how that works?
What else can i say to make clear what i am doing? :slight_smile:
Thanks a lot again!!! <3

Yes, change ‘sampler2D’ to ‘sampler2DRect’ and adjust the values you pass into the texture() call:

[code=before]
uniform sampler2D texA;
in vec2 texcoords;

void main()
{
vec4 sample = texture(texA, texcoords);
}



[code=after]
uniform sampler2DRect texA;
in vec2 texcoords;

void main()
{
    ivec2 sizeA = textureSize(texA);
    vec4 sample = texture(texA, texcoords * sizeA);
}

Instead of using textureSize and converting from normalized to texel-based texture coordinates you can also just change the tex-coords you pass to OpenGL. Oh, and the above is totally untested, so the syntax may not be 100% right :wink:

You could post your fragment shader (ideally with anything that is not relevant removed, so it is easier to focus on the concrete problem), it could also help if you include some information on what texture coordinates you pass to OpenGL.


out vec4 glFragColor;
smooth in vec4 texCoord;

uniform sampler2D background;   //With every Slice background will be updated (coloured)
uniform sampler2D muster;         //Original texture (Patches are taken from here)
uniform int slice;                         //Should be named Iteration, in every slice one more patch is added to background
uniform int numSlices;                //maximum number of patches 

void main(void) {

//these are the edges for every patch
    float leftBound = ((slice%numSlices)+0.0)/numSlices; 
    float rightBound = leftBound + (1.0/numSlices);
    upper = (numSlices - (slice/numSlices));   
    float upperBound = (upper+0.0)/numSlices;
    float lowerBound = upperBound - 1.0/numSlices;

//if we are in the right area == here belongs the next patch
    if (slice <(numSlices*numSlices)){
         if(texCoord.s <= rightBound && texCoord.s > leftBound && 
               texCoord.t >= lowerBound  && texCoord.t <=upperBound) {

//the actual pixels from one patch of background are transformed to a random patch of the original textur "muster"           
               float normalizedS = (texCoord.s-leftBound)*numSlices;
               float normalizedT = (texCoord.t-lowerBound)*numSlices;
               vec2 leftLowerCorner = (1.0-1.0/numSlices)*vec2(randomValue, randomValue2);
               vec2 c =  leftLowerCorner + (1.0/numSlices)*vec2(normalizedS,1.0-normalizedT);
               glFragColor=texture(muster,c);

Okay that’s what I got so far.
When I’m changing it to Rect the textur is all black. Do I have to multiply the Coords just when using texture(texA, texCoords)? I guess I need to update my Edges(Bounds) too, right?

Accessing rectangle textures with regular texture() function is almost the same as using texelFetch (coordinate-wise): if the tex size is 256x256 pixels, then the coordinates must be in range [0…255].
But unless the source coordinates are the direct integer values you need to correct the calculation error:

ivec2 sizeA = textureSize(texA);
vec4 sample = texelFetch(texA, ivec2(texcoords * sizeA + 0.5));

Notice that I use texelFetch() here, not texture().

Putting [noparse]


[/noparse] around source code should do the trick.
I think you should just pick one kind of units (normalized or texels), do all calculations in those units, and at the end use the appropriate type of sampler to access your texture (normalized: sampler2D, texels: sampler2DRect). Oh, for using sampler2DRect you might have to change the texture target in your Java code as well, from GL_TEXTURE_2D to GL_TEXTURE_RECTANGLE for all calls like glTexImage2D, glBindTexture.

On a more general note: if you only want to render to a rectangular part of your target framebuffer, why don’t you render a rectangle that only overlaps that part instead of rendering a large rectangle covering all of the target framebuffer and not doing anything for certain parts?

  • your shader seems to only write to glFragColor under some conditions, I don’t think that is allowed, you always need to write to it OR call discard to indicate that you don’t have any output.
  • the ‘background’ texture makes me wonder if that is also the texture you have bound as a render target? In general it is not possible to read from and write to the same texture - the usual workaround is to use two textures (one for reading the other for writing) and after one pass exchange their use, i.e. read from the just written texture and overwrite the other one.

When i do that i get this Error: No matching function for call to textureSize(sampler2D)
in this line:

ivec2 sizeMuster=textureSize(muster);

Do i have to import something to be able to use that method?

On a more general note: if you only want to render to a rectangular part of your target framebuffer, why don’t you render a rectangle that only overlaps that part instead of rendering a large rectangle covering all of the target framebuffer and not doing anything for certain parts?
In the end i need to render L & U -shaped overlapping regions. So rectangular will not be a solution.

your shader seems to only write to glFragColor under some conditions, I don’t think that is allowed, you always need to write to it OR call discard to indicate that you don’t have any output.
I took the unimportant parts out to focus on the relevant part. The construction of all the patches works fine.

the ‘background’ texture makes me wonder if that is also the texture you have bound as a render target? In general it is not possible to read from and write to the same texture - the usual workaround is to use two textures (one for reading the other for writing) and after one pass exchange their use, i.e. read from the just written texture and overwrite the other one.
yes background is my render target. hmmm… i guess i’ll try to make a new one then.

What i wanna do should kinda look like this:


if(we are in an overlapping region){
      for(every x Coord){                  //for every coloumn 
               for(every y coord){         //for every row 
                     if this pixel has the minimal color in this row/coloumn remember it
              }
      }
}

Do i have to import something to be able to use that method?

You probably have to declare the GLSL version you want to use, i.e. put a #version 330 on the first line of the shader, otherwise you only get GLSL 1.10 and IIRC that did not have support for textureSize.

In the end i need to render L & U -shaped overlapping regions. So rectangular will not be a solution.

You could still render sets of triangles that match those shapes though - but okay it is somewhat of an optimization so can be left for later.

Yay it works, the problem was that i forgot the argument int lod in the method textureSize :slight_smile:
So now im gonna play a little with it and look where it leads me.

BTW I’m not using Xcode. A few people mentioned it in forums would you suggest to use it? Or is there even a better program to help with GLSL on OS X?

It has to look like this:


#version 330

Note, the ‘#’ before version (and possibly has to be at the beginning of the line) not before the number. Sorry if that’s what you are already using, it’s not entirely clear from what you’ve posted.

Also, sorry no idea about Xcode, I don’t use OS X - I would assume it is a reasonable choice given how tightly OpenGL is built into the OS.

Okay, textureSize works fine.
Now im dealing with texelFetch:

  1. writing
vec4 sample = texelFetch(muster, ivec2(texCoord.st * sizeMuster + 0.5), 0);

this will paint the whole texture but horizontal flipped. I’d usually do *(-1) in the t Coord like this:

vec4 sample = texelFetch(muster, texCoord.s * sizeMuster + 0.5, texCoord.t * *(-1)*sizeMuster + 0.5, 0);

but that are not legal arguments. Is there a way to use the ivec2 and multipliing just the t coord?

2.just to make sure i understand texelFetch:

vec4 sample = texelFetch(muster, ivec2((0.1,0.1) * (800,600) + 0.5), 0);

gives me the vec4 color of the pixel at position (0.1,0.1), doesn’t it?
2.1 Why do I have to add 0.5?
2.2 I guess

ivec2((0.1,0.1) * (800,600) + 0.5)

this calculates all pixel from interval [0,1]x[0,1] to (in this case) [0,800]x[0,600] right? In order to get the color of the neighbor pixel to the right in question 2. i have to write:

vec4 sample = texelFetch(muster, ivec2((0.1,0.1) * (800,600) + 0.5)+(1,0), 0);

right?

Thanks in advance!
This is so much fun! :smiley:

Demo:
float x = 1.0/3.0; //x=0.333333
float y = x*3; //x=0.99999… != 1.0
Unless you deal with integer numbers or numbers that could be represented as x/(2^n) you will have calculation errors.

Hello, me again :slight_smile:

i am still not on a good foot with texelFetch.
Is it correct that

vec4 sample = texelFetch(muster, ivec2((0.1,0.1) * (800,600) + 0.5), 0);

gives me the vec4 color of the pixel at position (0.1,0.1), doesn’t it?

And in generell

ivec2(texCoord.st * (800,600) + 0.5)

calculates all pixel from interval [0,1]x[0,1] to (in this case) [0,800]x[0,600] right? In order to get the color of the neighbor pixel to the right i’d have to write:

vec4 sample = texelFetch(muster, ivec2((0.1,0.1) * (800,600) + 0.5)+(1,0), 0);

right?

Cheers,
Textursynthesizer

This topic was automatically closed 183 days after the last reply. New replies are no longer allowed.