Author Topic: fld results in nan  (Read 16646 times)

nobody

  • Guest
fld results in nan
« on: September 08, 2009, 07:13:26 PM »
How is it possible that when I use the instruction "fld qword [eax]" the nan-value is stored in st0? The concrete values from my debugger:

before execution:
st0 = 2
st1 = 1.7320508075688772953084487715713635
st2 = 1.7320508075688772953084487715713635
st3 = 3
st4 = 2
st5 = 1.7320508075688772953084487715713635
st6 = 1.7320508075688772953084487715713635
st7 = 3
eax = 0xff988f48
qword at eax = 0x4008000000000000 (3 in 64 bit float format)

execute: fld qword [eax]

after execution:
st0 = -nan(0xc000000000000000)   (raw 0xffffc000000000000000)
st2 = 1.7320508075688772953084487715713635
st3 = 1.7320508075688772953084487715713635
st4 = 3
st5 = 2
st6 = 1.7320508075688772953084487715713635
st7 = 1.7320508075688772953084487715713635

I expected st0 to hold the value 3... I am not experienced in nasm, if you need information please ask.

nobody

  • Guest
Re: fld results in nan
« Reply #1 on: September 08, 2009, 08:56:57 PM »
I've managed to get it working by using FFREE ST(i) for every register at the beginning of my procedure. It seems not to be a very elegant way to me but it does the job.
Also, something I've found that describes my problem quite exactly is the following (just in case someone finds this thread): http://software.intel.com/en-us/articles/intel-c-compiler-floating-point-calculations-produce-nans-with-c-code/
If someone knows a better solution to avoid the stack overflow I'd be happy to hear it.

Offline Frank Kotler

  • NASM Developer
  • Hero Member
  • *****
  • Posts: 2667
  • Country: us
Re: fld results in nan
« Reply #2 on: September 08, 2009, 11:03:17 PM »
There were two things that crossed my mind when I read your post. The first was that the value in eax doesn't "look like" valid memory to me - apparently that's okay. The second was that you're overflowing the FPU stack.

The article you cite mentions "rotating registers". Ray Filiatreault's tutorial:

http://www.website.masmforum.com/tutorials/fptute/index.html

describes FPU registers as a "revolver cylinder" - as in "six-shooter" (although it's an eight-shooter). This explains why it doesn't just "push the top value off into the bit-bucket", as you might expect.

Instead of freeing the registers individually, you could use "finit" (or "fninit"?) to clear all FPU registers before starting - IF you're done with them. As Intel's article implies, the right way to do it is not leave "junk" on the stack to begin with. Not being too experienced with the FPU, I find it quite "tricky" to keep track of what's where on the stack, and leave it "clean" at the end of my calculation (in an "elegant" way, preferably). I suspect it gets easier with practice. Now that you understand what the problem is, you can probably find a "neater" way to solve it - depends where that "junk" on the stack came from. If it's not "junk", you may need to arrange some temporary storage for intermediate values. (FPU instructions ending in "p" pop the stack - there are a few besides "fstp" that may be useful) I can't decide if this is a "royal PITA" or an "interesting puzzle". Depends on which day I'm doing it, I think. :)

Best,
Frank

nobody

  • Guest
Re: fld results in nan
« Reply #3 on: September 08, 2009, 11:04:00 PM »
Are you sure that the value held in eax actually points to a valid memory address??