sampler2D question/problem

[Feline]

I've been trying to chase down an oddity with sampling textures and the attached is the simplest state to which I can abstract it all - and now I figure I'll put it out here to see whether people can confirm or deny that they see the same things that I do and maybe help me understand what I'm doing here.

The attached fragment uses the generic DE-raytracer, the distance estimator is simply the distance to the xy plane
return pos.z;

The coloring samples a tiny (attached) "test.png" that has only 4 pixels (2x2, red, yellow, blue green). To run the fragment this needs to go somewhere into your include path.

There is a slider (Selector) that allows three choices how to sample:

• Selector=0 : p.xy - simply the coordinates in the xy plane. This rolls over for x or y > 1 so it makes a periodic tiling of the png. Everything is seamless and exactly how I would expect it to be.
• Selector=1 : fract(p.xy) - as far as I understand it, this should do exactly the same thing - except that I'm clipping coordinates greater than 1 "by hand" back into the [01) interval. Except that this creates visible lines between the tiles. At first I thought this might be because the precise value "1.000" is never sampled, but instead this should sample the precise value "0.000" which (due to the linear interpolation that is obviously happening) should be the exact same thing as "1.000" (except for a machine epsilon).
• Selector=2 : mod(p.xy,1) - this should be numerically identical to fract() if I understand the GLSL docs right

Both of the latter two produce those same visible lines.

The width of the lines appear to be independent of resolution - they're always TWO pixels wide (and indeed appear to be drawn with units 2x2 pixels in size). And the weirdest part:

The lines only appear in one coordinate (I think it's the x-coordinate).

What is going on here? Any kind of rounding problem (or whatever this might be) should affect the x-direction and the y-direction in the same way, no?

Is this something any of you folks have seen before? How do you deal with this?

[3dickulus]

I tried your frag but no lines here... but I have a couple of observations

notes
texture2D is a deprecated builtin for ES
texture is a builtin for 1.3
you used reserved word texture as a local variable name

try this...
add as the first line of your frag...

Code:

#version 130

then make from line 5 to line 20 look something like this...

Code:

uniform sampler2D mytexture; file[test.png]
// 0= bare roll-over, 1=fract, 2=mod(...,1)
uniform int Selector; slider[0,0,2]

vec3 baseColor(vec3 p, vec3 q) {
	vec2 Pcoord;

	switch (Selector) {
		case 0: {Pcoord = p.xy; break;}
		case 1: {Pcoord = fract(p.xy); break;}
		case 2: Pcoord = mod(p.xy,1);
	}

     	return texture(mytexture,Pcoord).xyz;
}

and in the preset...

Code:

mytexture = test.png

the test.png file only needs to be in the same folder as the frag and it seems fine with the github version and my own hacked version of Fragmentarium

[Feline]

This is interesting in a variety of ways.

First off: changing "texture" to "mytexture" (and texture2D to texture) doesn't make a difference - the lines are there in the frac() case but not in the first case.

If you don't see the lines, can I inquire about your hardware - are you perchance using an AMD graphics card? I'm asking because I get these lines on three different computers with three different graphics cards that have only one thing in common: they're all NVIDIA. If we had behavior here that depends on either the HW or the GL/drivier implementation that would be really interesting...

Lines that are visible to some people but not others tweaks my memory: in this thread here http://www.fractalforums.com/fragmentarium/odd-diagonal-feature/ there was an issue with a line appearing for some people (in particular myself) that seems to have been related to sampling buffers with integer layout. Hum. That sure sounds related...

[3dickulus]

nVidia GTX 760 runs your frag with no odd lines, driver v346.46, Qt v4.8.6
I tried the different settings 0,1,2 and the image seems the same, smooth.

your frag runs smooth as you posted it w/o changes, yes, tweak, yes, Win or Lin ?

[3dickulus]

ok so my suggestive notes tell me it's not syntax or version related.
hard to debug if it only shows up on your machine has anyone else tried this frag?

[knighty]

AFAIK, these artifacts are caused by the discontnuity of mod() and fract(). The derivatives of these function are constant but they are not defined at the discontinuities.
When invoking texture2D(), the derivatives of the argument coordinates wrt the screen coordiantes are computed in order to determine the mip level to use. the derivatives are computed in 2x2 pixel blocks which explains why the artifact are 2 pixels wide.
Why the artifact appear in one direction only is a mistery? 
Why 3diculus doesn't get those artifact? maybe the compiler "knows" that the derivatives are constant so it simplified things by not generating the dFdx and dFdy instructions... maybe... The only way to know for sure is to see the assembly generated code.

[Syntopia]

Knighty is right, this is a typical derivative problem. This was fixed in Fragmentarium 1.0 (where mipmapping is disabled), and I don't see the artifacts on my machine. Are you running the latest version?

[knighty]

A little correction to my previous post:
(mod(f(x),a))'=f'(x) if x is not equal to n*a, (n is any integer); otherwise it is not defined. This means that it is almost safe to ignore fract() and mod() in derivative computations. Maybe that's what modern shader compilers do.

[3dickulus]

The only way to know for sure is to see the assembly generated code.

I've hacked in a NVobjdump routine

Code:

AntiAliasScale
Aperture
BackgroundColor
BaseColor
CamLight
CamLightMin
Detail
DetailAO
Dither
EquiRectangular
Eye
FOV
FocalPlane
Fog
FudgeFactor
Gamma
GaussianWeight
Glow
GlowMax
GradientBackground
MaxRaySteps
OrbitStrength
Selector
Specular
SpecularExp
SpecularMax
SpotLight
SpotLightDir
Target
backbuffer
pixelSize
subframe
texture
gl_ProjectionMatrix
gl_Vertex
Dir
PixelScale
Right
UpOrtho
coord
from
viewCoord
viewCoord2
gl_Position
gl_FragColor
gl_FragDepth
gl_Vertex
Dir
PixelScale
Right
UpOrtho
coord
from
viewCoord
viewCoord2
gl_FragColor
gl_FragDepth

Fragment info
-------------
0(614) : warning C7011: implicit cast from "int" to "float"

!!NVvp5.0
OPTION NV_bindless_texture;
OPTION NV_shader_atomic_float;
 PARAM c[9] = { state.matrix.projection.transpose.row[0..3],
 program.local[4..8] };
ATTRIB vertex_attrib[] = { vertex.attrib[0..0] };
OUTPUT result_attrib[] = { result.attrib[0..7] };
TEMP R0, R1, R2;
ADD.F R0.xyz, -c[5], c[6];
DP3.F R0.w, R0, R0;
RSQ.F R0.w, R0.w;
MUL.F R1.xyz, R0.w, R0;
DP3.F R0.x, R1, c[7];
MAD.F R0.xyz, R1, -R0.x, c[7];
DP3.F R0.w, R0, R0;
RSQ.F R0.w, R0.w;
MUL.F R0.xyz, R0.w, R0;
MUL.F R2.xyz, R1.zxyw, R0.yzxw;
MAD.F R2.xyz, R1.yzxw, R0.zxyw, -R2;
DP3.F R0.w, R2, R2;
RSQ.F R0.w, R0.w;
MUL.F result.attrib[6].xyz, R0.w, R2;
MOV.F result.attrib[5].xyz, R0;
MUL.F R2.xy, vertex.attrib[0].y, c[1];
MAD.F R0.xy, vertex.attrib[0].x, c[0], R2;
MAD.F R0.xy, vertex.attrib[0].z, c[2], R0;
MAD.F R0.zw, vertex.attrib[0].w, c[3].xyxy, R0.xyxy;
MOV.F result.position, vertex.attrib[0];
DIV.F R0.x, c[8].y, c[8].x;
MOV.F result.attrib[4].xyz, R1;
MOV.F R0.y, R0.w;
MUL.F R0.x, R0.z, R0;
MUL.F result.attrib[1].xy, R0, c[4].x;
MOV.F result.attrib[0].xyz, c[5];
MOV.F result.attrib[2].xy, vertex.attrib[0];
MOV.F result.attrib[3].xy, R0.zwzw;
MUL.F result.attrib[7].x, c[0], c[8];
MUL.F result.attrib[7].y, c[1], c[8];
END

!!NVfp5.0

OPTION NV_gpu_program_fp64;
OPTION NV_bindless_texture;
OPTION NV_shader_atomic_float;
 PARAM c[31] = { program.local[0..30] };
ATTRIB fragment_attrib[] = { fragment.attrib[0..7] };
TEMP R0, R1, R2, R3, R4, R5, R6, R7;
LONG TEMP D0;
TEMP RC, HC;
OUTPUT result_color0 = result.color;
ADD.S R0.x, c[1], {1, 0, 0, 0};
I2F.S R0.z, R0.x;
MUL.F R0.xy, fragment.attrib[2], R0.z;
DP2.F R0.w, R0, {12.9898, 78.233002, 0, 0};
DP2.F R0.y, R0, {4.8979998, 7.23, 0, 0};
SIN.F R0.w, R0.w;
MUL.F R0.x, R0.w, {43758.547, 0, 0, 0};
FRC.F R0.x, R0;
MUL.F R1.x, R0, {6.2800002, 0, 0, 0};
COS.F R0.y, R0.y;
MUL.F R0.y, R0, {23421.631, 0, 0, 0}.x;
FRC.F R0.y, R0;
RSQ.F R0.w, R0.y;
COS.F R0.x, R1.x;
SIN.F R0.y, R1.x;
DIV.F R0.xy, R0, R0.w;
MUL.F R0.xy, R0, c[4].x;
MUL.F R1.xyz, R0.y, fragment.attrib[5];
MUL.F R0.zw, fragment.attrib[1].xyxy, R0.z;
DP2.F R0.y, R0.zwzw, {12.9898, 78.233002, 0, 0};
MAD.F R2.xyz, R0.x, fragment.attrib[6], R1;
SIN.F R0.x, R0.y;
DP2.F R0.y, R0.zwzw, {4.8979998, 7.23, 0, 0};
MUL.F R0.x, R0, {43758.547, 0, 0, 0};
FRC.F R0.x, R0;
MUL.F R0.w, R0.x, {6.2800002, 0, 0, 0}.x;
COS.F R0.y, R0.y;
MUL.F R0.y, R0, {23421.631, 0, 0, 0}.x;
FRC.F R0.y, R0;
RSQ.F R0.z, R0.y;
COS.F R0.x, R0.w;
SIN.F R0.y, R0.w;
DIV.F R7.xy, R0, R0.z;
SNE.U R0.x, c[0], {0, 0, 0, 0};
MOV.U.CC RC.x, -R0;
POW.F R3.w, {10, 0, 0, 0}.x, c[9].x;
POW.F R2.w, {10, 0, 0, 0}.x, c[10].x;
MOV.F R1.w, {0, 0, 0, 0}.x;
IF NE.x;
MUL.F R0.xy, fragment.attrib[7], c[7].x;
MUL.F R0.zw, R0.xyxy, R7.xyxy;
ELSE;
MUL.F R0.xy, fragment.attrib[7], c[7].x;
MUL.F R0.xy, R0, c[8].x;
MAD.F R0.zw, R0.xyxy, R7.xyxy, fragment.attrib[1].xyxy;
ENDIF;
MAD.F R0.xyz, R0.z, fragment.attrib[6], fragment.attrib[4];
MAD.F R0.xyz, R0.w, fragment.attrib[5], R0;
MAD.F R0.xyz, R0, c[3].x, -R2;
DP3.F R0.w, R0, R0;
RSQ.F R0.w, R0.w;
MUL.F R1.xyz, R0.w, R0;
SNE.U R0.w, c[0].x, {0, 0, 0, 0}.x;
MOV.U.CC RC.x, -R0.w;
MOV.F R0.xyz, R1;
IF NE.x;
MAD.F R0.y, -fragment.attrib[3], {0.5, 0, 0, 0}.x, {0.5, 0, 0, 0}.x;
MOV.F R0.x, fragment.attrib[3];
MUL.F R0.zw, R0.xyxy, {3.1415927, 0, 0, 0}.x;
SIN.F R3.y, R0.w;
SIN.F R0.x, R0.z;
COS.F R3.x, R0.z;
MUL.F R0.x, R3.y, R0;
MUL.F R0.xyz, R0.x, fragment.attrib[6];
MUL.F R3.x, R3, R3.y;
MAD.F R0.xyz, R3.x, R1, R0;
COS.F R0.w, R0.w;
MAD.F R0.xyz, R0.w, fragment.attrib[5], R0;
ENDIF;
DP3.F R0.w, R0, R0;
RSQ.F R0.w, R0.w;
MUL.F R1.xyz, R0.w, R0;
ADD.F R3.xyz, fragment.attrib[0], R2;
MOV.F R4.z, {0, 0, 0, 0}.x;
MOV.F R0.w, {0, 0, 0, 0}.x;
MOV.F R4.w, {0, 0, 0, 0}.x;
MOV.S R2.x, {0, 0, 0, 0};
REP.S ;
SGE.S.CC HC.x, R2, c[12];
BRK (NE.x);
MAD.F R2.y, R0.w, R1.z, R3.z;
MUL.F R4.z, R2.y, c[11].x;
SEQ.S R2.y, R2.x, {0, 0, 0, 0}.x;
MOV.U.CC RC.x, -R2.y;
ADD.F R1.w, R1, {1, 0, 0, 0}.x;
IF NE.x;
DP2.F R2.y, R1, {12.9898, 78.233002, 0, 0};
SIN.F R2.y, R2.y;
MUL.F R2.y, R2, {43758.547, 0, 0, 0}.x;
FRC.F R2.y, R2;
MAD.F R2.y, R2, c[13].x, -c[13].x;
MAD.F R4.z, R2.y, R4, R4;
ENDIF;
ADD.F R0.w, R0, R4.z;
MUL.F R4.w, R0, R3;
SLT.F R2.y, R4.z, R4.w;
TRUNC.U.CC HC.x, R2.y;
IF NE.x;
ADD.F R2.y, R4.w, -R4.z;
MOV.U.CC RC.x, {1, 0, 0, 0};
ADD.F R0.w, R0, -R2.y;
BRK (NE.x);
ENDIF;
SGT.F R2.y, R0.w, {100000, 0, 0, 0}.x;
TRUNC.U.CC HC.x, R2.y;
IF NE.x;
MOV.U.CC RC.x, {1, 0, 0, 0};
RCP.F R2.y, R4.z;
ADD.F R2.z, R0.w, {-100000, 0, 0, 0}.x;
MAD.F R1.w, -R2.z, R2.y, R1;
BRK (NE.x);
ENDIF;
ADD.S R2.x, R2, {1, 0, 0, 0};
ENDREP;
I2F.S R3.w, c[23].x;
SGT.F R5.x, c[28], {0, 0, 0, 0};
TRUNC.U.CC HC.x, R5;
MOV.F R2.xyz, c[27];
DIV.F.SAT R1.w, R1, R3.w;
IF NE.x;
DP2.F R2.x, fragment.attrib[1], fragment.attrib[1];
RSQ.F R2.x, R2.x;
DIV.F R2.x, c[28], R2.x;
MAD.F R2.xyz, R2.x, -c[27], c[27];
ENDIF;
SLT.F R3.w, R4.z, R4;
TRUNC.U.CC HC.x, R3.w;
IF NE.x;
MAD.F R5.xyz, R0.w, R1, R3;
MUL.F R3.w, R4, {0.5, 0, 0, 0}.x;
MAX.F R3.y, R3.w, {1e-07, 0, 0, 0}.x;
MAD.F R3.x, R1.z, -R4.w, R5.z;
ADD.F R3.z, R3.x, -R3.y;
ADD.F R3.x, R3, R3.y;
ADD.F R3.z, R3.x, -R3;
MOV.F R3.xy, {0, 0, 0, 0}.x;
DP3.F R3.w, R3, R3;
RSQ.F R3.w, R3.w;
MUL.F R3.xyz, R3.w, R3;
SEQ.S R3.w, c[30].x, {0, 0, 0, 0}.x;
MOV.U.CC RC.x, -R3.w;
MUL.F R1.w, R1, c[14];
IF NE.x;
MOV.F R4.xy, R5;
ELSE;
SEQ.S R3.w, c[30].x, {1, 0, 0, 0}.x;
MOV.U.CC RC.x, -R3.w;
IF NE.x;
FRC.F R4.xy, R5;
ELSE;
SEQ.S R3.w, c[30].x, {2, 0, 0, 0}.x;
MOV.U.CC RC.x, -R3.w;
IF NE.x;
FLR.F R4.xy, R5;
ADD.F R4.xy, R5, -R4;
ENDIF;
ENDIF;
ENDIF;
PK64.U D0.x, c[29];
TEX.F R4.xyz, R4, handle(D0.x), 2D;
ADD.F R4.xyz, R4, -c[25];
MAD.F.SAT R4.xyz, R4, c[26].x, c[25];
SLT.F R3.w, c[10].x, {0, 0, 0, 0}.x;
TRUNC.U.CC HC.x, R3.w;
POW.F R4.x, R4.x, c[5].x;
POW.F R4.y, R4.y, c[5].x;
POW.F R4.z, R4.z, c[5].x;
IF NE.x;
DP2.F R1.w, R5, {12.9898, 78.233002, 0, 0};
SIN.F R1.w, R1.w;
MUL.F R1.w, R1, {43758.547, 0, 0, 0}.x;
FRC.F R1.w, R1;
MAD.F R5.x, -R1.w, c[13], {1, 0, 0, 0};
MOV.F R4.w, {0, 0, 0, 0}.x;
MOV.F R1.w, {0, 0, 0, 0}.x;
MOV.F R5.y, {1, 0, 0, 0}.x;
MOV.F R3.w, {1, 0, 0, 0}.x;
REP.S ;
SGE.F.CC HC.x, R3.w, {6, 0, 0, 0};
BRK (NE.x);
MUL.F R6.x, R5, R3.w;
MUL.F R6.x, R3.w, R6;
MUL.F R5.y, R5, {0.60000002, 0, 0, 0}.x;
MUL.F R5.w, R5.x, R3.z;
MUL.F R5.w, R5, R3;
MUL.F R5.w, R5, R3;
MUL.F R6.x, R2.w, R6;
MAD.F R5.w, R5, R2, R5.z;
RCP.F R6.x, R6.x;
ADD.F R5.w, R5, -R5.z;
MAD.F.SAT R5.w, -R5, R6.x, {1, 0, 0, 0}.x;
MAD.F R4.w, R5.y, R5, R4;
ADD.F R1.w, R5.y, R1;
ADD.F R3.w, R3, {1, 0, 0, 0}.x;
ENDREP;
MUL.F R2.w, R4, c[14];
DIV.F.SAT R1.w, R2, R1.w;
ENDIF;
MUL.F R2.w, c[19].x, {3.1415, 0, 0, 0}.x;
MUL.F R4.w, c[19].y, {1.57075, 0, 0, 0}.x;
COS.F R5.x, R4.w;
SIN.F R3.w, R2.w;
SIN.F R4.w, R4.w;
MUL.F R5.y, R3.w, R4.w;
MUL.F R5.x, R3.w, R5;
COS.F R5.z, R2.w;
DP3.F R2.w, R5, R5;
RSQ.F R2.w, R2.w;
MUL.F R5.xyz, R2.w, R5;
ADD.F R6.xyz, -R1, R5;
DP3.F R1.y, -R3, R1;
DP3.F R2.w, R6, R6;
RSQ.F R2.w, R2.w;
MUL.F R6.xyz, R2.w, R6;
DP3.F R2.w, R3, R6;
MAX.F R2.w, R2, {0, 0, 0, 0}.x;
ADD.F R3.w, -R2, {1, 0, 0, 0}.x;
POW.F R4.w, R3.w, {5, 0, 0, 0}.x;
POW.F R3.w, R2.w, c[16].x;
ADD.F R2.w, c[16].x, {2, 0, 0, 0}.x;
MUL.F R3.w, R2, R3;
MAD.F R2.w, R4, -c[16].x, R4;
ADD.F R4.w, R2, c[16].x;
DP3.F R5.x, R3, R5;
MAX.F R1.y, R1, c[21].x;
MAX.F R2.w, R5.x, {0, 0, 0, 0}.x;
MUL.F R3.w, R3, R4;
MUL.F R3.w, R3, R2;
MUL.F R1.x, R3.w, c[15];
MUL.F R1.x, R1, {0.125, 0, 0, 0};
MIN.F R3.x, R1, c[17];
MUL.F R1.y, R1, c[20].w;
MUL.F R2.w, R2, c[18];
MUL.F R1.xyz, R1.y, c[20];
MAD.F R1.xyz, R2.w, c[18], R1;
MAD.F R1.xyz, R3.x, c[18], R1;
POW.F R2.w, c[24].x, {4, 0, 0, 0}.x;
MUL.F R2.w, -R2, R0;
ADD.F R3.xyz, -R4, c[14];
MAD.F R3.xyz, R1.w, R3, R4;
MUL.F R2.w, R2, R0;
MUL.F R1.w, R2, {1.442695, 0, 0, 0}.x;
MAD.F R2.xyz, -R1, R3, R2;
EX2.F R1.w, R1.w;
MAD.F R2.xyz, -R1.w, R2, R2;
MAD.F R1.xyz, R1, R3, R2;
ELSE;
MUL.F R1.xyz, R1.w, c[22];
MAD.F R1.xyz, R1, c[22].w, R2;
ENDIF;
DP3.F R0.x, R0, R0;
RSQ.F R0.x, R0.x;
MUL.F R0.x, R0, R0.w;
MIN.F R1.w, R0.x, {1000, 0, 0, 0}.x;
MAX.F R1.w, R1, {9.9999997e-06, 0, 0, 0}.x;
RCP.F R1.w, R1.w;
DP2.F R2.x, R7, R7;
RSQ.F R2.x, R2.x;
MAD.F result.depth.z, R1.w, {-9.9999997e-06, 1, 0, 0}.x, {-9.9999997e-06, 1, 0, 0}.y;
RCP.F R1.w, R2.x;
SGT.F R2.x, c[6], {0, 0, 0, 0};
TRUNC.U.CC HC.x, R2;
MAD.F R0.xy, fragment.attrib[2], {0.5, 0, 0, 0}.x, {0.5, 0, 0, 0}.x;
PK64.U D0.x, c[2];
TEX.F R0, R0, handle(D0.x), 2D;
ADD.F R1.w, -R1, {1, 0, 0, 0}.x;
IF NE.x;
DP2.F R1.w, R7, R7;
RCP.F R2.x, c[6].x;
DIV.F R1.w, -R1, c[6].x;
MUL.F R2.x, -R2, {1.442695, 0, 0, 0};
MUL.F R1.w, R1, {1.442695, 0, 0, 0}.x;
EX2.F R2.x, R2.x;
EX2.F R1.w, R1.w;
ADD.F R1.w, R1, -R2.x;
ENDIF;
MOV.F R2.w, R1;
MUL.F R2.xyz, R1, R1.w;
ADD.F result_color0, R0, R2;
END

I did notice this err during compilation...
0(614) : warning C7011: implicit cast from "int" to "float"

[Syntopia]

The Sampler2D read is in the first line beginning with TEX.F. All of the derivate stuff and mipmapping is done inside the TEX.F function, so the assembly does not help much (see Section 2.X.8.Z in the assembly specs: http://developer.download.nvidia.com/opengl/specs/GL_NV_gpu_program5.txt).

Still, I believe this correct way to fix this is to disable mipmapping: e.g. use glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR).

[knighty]

Thank you for the link. I was wandering why there were no ddx/ddy instructions and thought it was because mipmapping was disabled.

[Feline]

This is a good topic, because I learn something in the process... 

First off, I do in fact still mostly use 0.912b - my excuse is that I like the "preview" mode but in reality my thoroughly (and terribly) hacked version of the raytracer doesn't really work in 1.0 and I've been too lazy to sit down and fix the myriad atrocities I committed in it. Indeed, when I fire up 1.0 (the 'triangle' version) the lines are not there any more. Cool. That solves this particular problem.

I also scared up an oldish ATI RagePro3700 card, which has the added data point that I get the lines with fract() but not with mod(...,1). For whatever reason.

Clearly I need to learn more here - I figured that access to the map at some location simply takes the nearest samples and interpolates between them somehow (linearly from the look of it) - but that wouldn't require knowledge of the gradient, no? Why is that needed?