x64 stack alignment [Need help]

[coldflame]

Hello everyone,

I need a little advice. There's not a lot of documentation for x64 programming (in comaprison to 32b). Can anyone explain me stack alignment? To be more clear, I have following code:

Code: [Select]

[BITS 64]
%include 'win32n.inc'

extern MessageBoxA
extern ExitProcess

section .data
headline          db "Greetings",0
my_text        db "How are you?",0

section .code
start:
    sub rsp,40         ; shadow space, aligns stack
        mov rcx,0               ; hWnd = HWND_DESKTOP
        mov rdx,my_text        ; LPCSTR lpText
        mov r8,headline          ; LPCSTR lpCaption
        mov r9,MB_OK | MB_ICONINFORMATION              ; uType = MB_OK
        call MessageBoxA        ; call MessageBox API function

        mov rcx,0               ; uExitCode = MessageBox(...)
        call [ExitProcess]

As I read somewhere

In addition to registers, each function has a frame on the run-time stack. This stack grows downwards from high addresses. (...) The end of the input argument area shall be aligned on a 16 (...) byte boundary. In other words, the value (%rsp + 8 ) is always a multiple of 16 when control is transferred to the function entry point. The stack pointer, %rsp, always points to the end of the latest allocated stack frame.

What's not clear to me is why it works with sub rsp,40 while it doesn't work with any other number. As stated above, it should be aligned on a 16 byte boundary. That means 16, 32, 48 etc. Why sub rsp,32 does not work? Why rsp,48 does not work and why does 40 work? I'm little bit confused.

What's even more strange to me is that on a different PC (I use Broadwell while the other PC has Haswell inside), it works with numbers 8, 24, 40 etc. That means 16 byte alignment +8 (like constant). After I call the first function, the stack is in the middle of 16 byte boundary, that's why the constant +8. BUT how should I know that the stack is going to be there? There are so many "why" for me.

Can someone help me and explain me this? Thank you!

[coldflame]

UPDATE:

It works with ANY 16 byte boundary +8 with ecx,0 before the ExitProcess call.

When I use rcx,0, it works ONLY with rsp,40.

When I use ecx,0, it works as mentioned above.

[coldflame]

I'm using NASM: nasm -fwin64 call.asm

And GoLink linker: golink call.obj

When I run the call.exe, it displays dialog window.

[dreamCoder]

On entry to _start, the address of the stack is not aligned to 16.

When the OS allocates a stack for your code, it gives you a stack aligned to 16. But there's a prior "CALL" to _start that modifies the RSP a.k.a breaking the alignment - probably by the linkers. Thus you need to re-align it back again to at least SUB RSP,8

[dreamCoder]

To make it more clear;

1. It's not your code that needs stack alignment. It's the Win API functions such as MessageBox.
2. When a Win API function is called, it must see the current stack (Top Of Stack) be aligned to 16 byte boundary. Thus, to avoid bugs, scared programmers rarely touch the stack for other purposes or at least avoiding odd pushes when they have to play with the stack. They strive to maintain aligned stack at all times in their code.
3. So whatever you do, make sure the current TOS is aligned to 16-byte boundary prior to calling a WinAPI function.

The stack breakdown should be something like this;

Code: [Select]

sub rsp,8  ;align the stack
sub rsp,32  ;shadow space allocation
Although this seems tedious, but it is more readable and much more clearer to readers than

Code: [Select]

sub rsp,40

[coldflame]

Thank you for the reply! However, I'm not sure about the functionality of it. I mean, should I ALWAYS "test" SUB RSP,8 and if it works, then I don't have to bother with it anymore? Because I get this error not only with WinAPI function.

In other words: if some code does not work, should I "test/add" SUB RSP,8 as something like "universal" solution and see if it works? Or is it needed to solve the alignment separately according to the function I call (let's say one of the WinAPI ones)?

Thank you!

[dreamCoder]

No universal solution for it. It depends on many things such as the linkers, compilers and how the OS allocates the stack for your code. You should start with a small use case / test code just to check for the stack alignment (for example, calling a MessageBox) at the start of any of your code. And of course there is the debugger.

My general strategy is to get the stack aligned at the start of the code and then maintain such alignment throughout the code. This is safe enough for my use. If u have other errors, then at least you know it didn't come from the stack misalignment.

Btw, I'm not sure how you used GoLink but I think it should be something like

Code: [Select]

golink call.obj user32.dll kernel32.dll

[dreamCoder]

I found that using "call" as the file name will show nothing as the output. Try changing the file name to something else.

[coldflame]

That was just an example. Sorry for that. My filename is "okno1.asm" and "okno1.obj". And yes, I'm using GoLink with kernel32 and user 32 when linking.

I have found here: https://blogs.msdn.microsoft.com/oldnewthing/20040114-00/?p=41053 this:

The stack must be kept 16-byte aligned. Since the "call" instruction pushes an 8-byte return address, this means that every non-leaf function is going to adjust the stack by a value of the form 16n+8 in order to restore 16-byte alignment.

So it might be caused by call instruction if I got it right.

[dreamCoder]

Yes, that's exactly my first reply regarding the "call" (that is, the SUB RSP,8 is required to re-align the stack)

Can't blame you for such confusions. I was once like you
 

[coldflame]

Yeah I'm not so skilled in assembly language, just did some codes for my thesis, but they were made within x86 architecture. Now I need to work with x64 architecture so that's why I'm a little bit confused of this everything Thank you for your time (: