41
Programming with NASM / Re: Problem with calculating root
« Last post by fredericopissarra on January 30, 2025, 07:07:16 PM »I'll assume you just want to extract the square root of an integer value and get an integer result using `sqrtsd` instruction...
First: double type has 53 bits of precision and 11 bits in the expoent of the scale factor plus the signal bit. This means a double is 64 bits long (but with less precision than an long long int)... Let's say your function is defined to obey SysV ABI for x86-64 (not for Windows):
If you want to use unsigned int, instead, you need to zero the upper 32 bits of RDI:
First: double type has 53 bits of precision and 11 bits in the expoent of the scale factor plus the signal bit. This means a double is 64 bits long (but with less precision than an long long int)... Let's say your function is defined to obey SysV ABI for x86-64 (not for Windows):
Code: [Select]
; unsigned long long int sqrtroot( unsigned long long int x );
; Entry: RDI = x
; Exit: RAX
global sqrtroot
align 4
sqrtroot:
cvtsi2sd xmm0,rdi ; Convert RDI to double scalar in XMM0.
sqrtsd xmm0,xmm0 ; xmm0 = sqrt(xmm0)
cvtsd2si rax,xmm0 ; Convert XMM0 double scalar to RAX (this convertion applies rounding).
ret
For Windows change RDI to RCX.If you want to use unsigned int, instead, you need to zero the upper 32 bits of RDI:
Code: [Select]
; unsigned int sqrtroot( unsigned int x );
; Entry: EDI = x
; Exit: EAX
global sqrtroot
align 4
sqrtroot:
mov edi,edi
cvtsi2sd xmm0,rdi ; Convert RDI to double scalar in XMM0.
sqrtsd xmm0,xmm0 ; xmm0 = sqrt(xmm0)
cvtsd2si rax,xmm0 ; Convert XMM0 double scalar to RAX (this convertion applies rounding).
ret
PS: GCC makes sure the upper double from xmm0 is zero beginning with pxor xmm0,xmm0, but we don't need to do this since we're dealing only with an scalar...