5

I have a problem with precision on AMD in shaders (hlsl). In vertex shader I calculate UVs for particle using modulus operator on float. With simple % there is a problem on Nvidia and on AMD (e.g. 7 % 7 = 7) so I have written modulus as:

float Mod( float x, float y )    
{       
     return x - floor(x / y) * y;    
}

This have fixed the problem on Nvidia but AMD still returns wrong values.

Any idea what is going on? And how to fixed it?

I know I can use:

float( int( x ) % int( y ) )

but I think it is slower and I wonder if there is any better way.

Derag
  • 596
  • 2
  • 11

1 Answers1

1

In HLSL, you can use the built-in fmod function for this.

I'm not sure what's going wrong with your implementation, though. It looks mathematically correct. It's conceivable you're encountering a precision issue with the subtraction, depending on the inputs.

BTW, an alternative implementation which avoids the subtraction is:

float Mod(float x, float y)
{
    return frac(x / y) * y;
}
Nathan Reed
  • 24,302
  • 2
  • 63
  • 103
  • But `frac(x / y) * y` is exactly what normal modulus operator compiles to (on GCN at least) and it gives incorrect results on both Nvidia and AMD. I'm not sure about `fmod`, though. It seems to give slightly different assembly code. I will check it, thanks. – Derag Apr 19 '18 at 19:45
  • So I have chacked it and `frac(x / y) * y`, operator `%` and `fmod` give incorrect results on Nvidia and AMD. `x - floor(x / y) * y` works correct on Nvidia but AMD not so much. – Derag Apr 20 '18 at 07:57