pouët.net

Go to bottom

Fixed point problem

category: general [glöplog]
I'm trying to convert my perspective-correct texture mapper to fixed-point. The mapper is working fine in floating-point.

The problem is the calculation of the texture-coordinate deltas that I don't seem to get right. There seems to be a general problem with overflow or whatever.
Code: int DUDX = ((-(((long long)(UZ3 - UZ1) * (long long)DY23) - ((long long)(UZ2 - UZ1) * (long long)-DY31))) / (long long)C);

The UZs are in 8.24 format and are correct, the DYs are in 28.4 format and are correct, C is in 24.8 format and correct (read: same as in float mapper). They are all stored in int variables.
The formula is not pretty, but I reverted to the exact same order as in the float mapper and used parenthesises just to be sure...

I look at the disassembly gave me the impression that this should be working slow, but correct, but it isn't.

I'm not aiming for speed at this point. I just want a C-implementation that is working.

Obvious errors? Ideas?
added on the 2010-03-11 17:24:27 by raer raer
Haven't tried to do this or thought about it before, but doing fixed point calculations between different formats, will that work? And which of the three types will be the format we assume in the final result?
added on the 2010-03-11 17:34:34 by Optimus Optimus
You can read this.
added on the 2010-03-11 17:52:52 by raer raer
Basically if you divide two fixed-point numbers with the same format you'll end up with an integer result. So usually you prescale the nominator to preserve precision.
The result of the multiplication should be in 36.28 format and C is in 24.8 format, so there should be enough precision even after the division.
added on the 2010-03-11 18:42:53 by raer raer
8.24 * 28.4 = 37.28 format because you're doing a signed multiply. Section 5.8 of your link.
added on the 2010-03-11 19:48:22 by trc_wm trc_wm
Ah.. That's easy for me.. I still do fixed point programming.

In what format do you want your result? 24.8 or 8.24? Or maybe 16.16?



added on the 2010-03-11 21:17:29 by torus torus
You are using the full 32 bits, so I suppose you are using unsigned ints. Thinking that your UZs and DYs are extrictly positive numbers, it would only work if you do it this way:

int DUDX = ((-((((long long)UZ3 - (long long)UZ1) * (long long)DY23) - (((long long)UZ2 - (long long)UZ1) * (long long)-DY31))) / (long long)C);

Because else you are doing the UZ3-UZ1 as unsigned ints.

Then, do a calc about the format of the final result. 36.28 / 24.8 should give you a number with 20 bits of decimals.
added on the 2010-03-11 22:12:30 by texel texel
In any case, I suggest you use signed ints and 31 bits, for example: 7.24 (leaving this way 1 bit for the sign)
added on the 2010-03-11 22:13:59 by texel texel
Ummm... also, the problem that torus said, 36.28 with signed needs a bit more or there would be overflow problems...
added on the 2010-03-11 22:17:44 by texel texel
texel: There's no explicit sign-bit in normal signed fixed point numbers, just as there is no explicit sign-bit in signed integers. It's not 31 bits just because negative values are encoded as two's compliment.
added on the 2010-03-11 22:47:34 by kusma kusma
Fun fact: In the DSP-world, where fixed-point is still used a lot the sign-bit is always specified.

If they talk about Q15 they talk about a 16 bit integer with one sign-bit and 15 bit fraction. Q15.16 is the good old fixed point value with 16 bit whole part and 16 bit fraction and so on..

added on the 2010-03-11 23:28:51 by torus torus
Did I ever mention that TI DSPs suck? Well, they do...
added on the 2010-03-11 23:34:11 by trc_wm trc_wm
Thanks guys...

Sorry, I didn't make it clear that all variables are signed ints, so the formats are 7.24 for the UZs, 27.4 for the DYs and 23.8 for C.
I don't know really what precision is necessary for the interpolants. I think a 8.24 format would be more than enough...

I'll try some stuff tomorrow.
added on the 2010-03-12 00:27:58 by raer raer
uhm 7.24, so signed int. whatever... :)
added on the 2010-03-12 00:29:42 by raer raer
the easiest way to do perspective projection in integer is not to use fixed point. start out with manually doing floating point. a good first step is doing just a int for both mantissa and exponent. work that out. you need only add (shift to same exponent, add), multiply (mul mantissa, add exponent) and reciprocal (tabel and 2 iterations). then you will see where you actually need to normalize if you want to optimize.
added on the 2010-03-12 05:56:20 by shiva shiva
Yeah, one could maybe work out precision and overflow problems better that way...
added on the 2010-03-12 09:44:52 by raer raer
instead of stabbing in the dark, why don't you just do the fixed and floating point calculations side-by-side (and term-by-term) and then assert that they agree, up to some tolerance? then you'll see where exactly it goes wrong. usually makes such bugs easy to find :)
added on the 2010-03-12 10:13:22 by ryg ryg
The multiplications are screwing up:
Code: (long long)(UZ3 - UZ1) * (long long)DY23

gives wrong results. When I multiply the numbers by hand, I get correct results. "UZ3 - UZ1" is still correct though.

The generated code is:
Code: long long bla3 = (long long)(UZ3 - UZ1) * (long long)DY23; 004026D5 mov eax,dword ptr [UZ3] 004026DB sub eax,dword ptr [UZ1] 004026E1 cdq 004026E2 mov ecx,eax 004026E4 mov esi,edx 004026E6 mov eax,dword ptr [DY23] 004026EC cdq 004026ED push edx 004026EE push eax 004026EF push esi 004026F0 push ecx 004026F1 call @ILT+1490(__allmul) (4395D7h) 004026F6 mov dword ptr [bla3],eax 004026FC mov dword ptr [ebp-104h],edx


ebp-104h is NOT pointing to the upper dword of bla3. that's why the number in bla3 is missing bits. WTF?
added on the 2010-03-12 12:01:35 by raer raer
Plus sizeof(long long) == 4. WTF?

What is the portable way to get a 64Bit integer in C/++?
added on the 2010-03-12 12:07:08 by raer raer
your 'long long' is not a 'signed 64 bit value' ? :)
added on the 2010-03-12 12:12:27 by the_Ye-Ti the_Ye-Ti
what compiler are you using?
added on the 2010-03-12 12:13:14 by the_Ye-Ti the_Ye-Ti
The compiler included in VS2008 Express.
added on the 2010-03-12 12:14:10 by raer raer
http://msdn.microsoft.com/en-us/library/cc953fe1.aspx

'long long' equivalent to __int64

added on the 2010-03-12 12:18:08 by the_Ye-Ti the_Ye-Ti
if you use vs2008 you can directly use _int64 _emul(long, long) and remove the ugly casts. Gives faster code as well.

added on the 2010-03-12 12:22:41 by torus torus
Is it still portable then?

I was reading about stdint.h. Though I'm not sure if the type "int64_t" is portable either...
added on the 2010-03-12 12:25:46 by raer raer

login

Go to top