x86-64 mode and "the red zone"

[fredericopissarra]

When you need to reserve space in the stack for local variables (if needed) you simply subtract the amount of memory from RSP (or ESP, if in i386 mode), respecting the QWORD (or DWORD) alignment. But, in x86-64 mode, when you are dealing with a "pure" function (a function which don't call any other functions) there is something called "the red zone".

The "red zone" is space of 128 bytes bellow the address pointed by RSP that will not be touched by signals and this region is available to programs running in userspace (ring 3) -- usually, not in kernelspace because interrupts are managed by the kernel and can occur any time, and the kernel has its own stack. You can disable high level language compiler to use the red zone, like using `-mno-red-zone` on GCC, for example. Indeed, this is important for UEFI applications...

This way, if your function is pure and uses up to 128 bytes of stack space below RSP, you don't need to reserve this space subtracting the amount from RSP. But beware: If your function isn't "pure" (calls any other function) you DO NEED to reserve space the traditional way.

This is a feature of the operating system. Linux, FreeBSD and Windows uses the red zone. Make sure YOUR OS has this as well, before using.

The "red zone" don't exist in real or i386 protected mode.

[fredericopissarra]

BTW... I strongly recommend, if you are dealing with real or protected mode, that you create a structure mirroring the state of the stack for your functions instead of calculating, by head, the corresponding offsets. For example:

Code: [Select]

; int f( int a, int b ) { return a + b; }
_f:
  mov eax,[esp+4]
  add eax,[esp+8]
  retHere I get the a and b arguments from the stack using the offsets 4 and 8, but it is easier to use structures to let the compiler calculate the offsets:

Code: [Select]

struc fstk
     resd 1    ; return address put here by call _f.
.a:  resd 1
.b:  resd 1
endstruc

_f:
  mov eax,[esp + fstk.a]  ; let NASM calculate the offset!
  add eax,[esp + fstk.b]
  retIn cases where you need to use local vars, you can do:

Code: [Select]

struc fstk
.local_t: resd 1
.local_size:
     resd 1    ; return address put here by call _f.
.a:  resd 1
.b:  resd 1
endstruc

_f:
  sub  esp,fstk.local_size  ; let NASM calculate the local vars reservation size.
  mov eax,[esp + fstk.a]  ; let NASM calculate the offset!
  add eax,[esp + fstk.b]
  ...
  add  esp,fstk.local_size
  ret
Notice: No prologue/epilogue needed! But for 8086 we need to use `bp` to access the stack... in real mode for 8086/80286 processors (taking int as word) we could do:

Code: [Select]

struc fstk
     resw 1    ; old_bp.
     resw 1    ; return address put here by call _f.
.a:  resw 1
.b:  resw 1
endstruc

_f:
  push bp
  mov bp,sp   ; prologue needed!

  mov ax,[bp + fstk.a]  ; let NASM calculate the offset!
  add ax,[bp + fstk.b]

  pop bp      ; epilogue needed!

  retAnd the same trick could be used for local vars.

Code: [Select]

struc fstk
.local_t: resw 1
.local_size:
     resw 1    ; old bp.
     resw 1    ; return address put here by call _f.
.a:  resw 1
.b:  resw 1
endstruc

_f:
  push bp

  sub   sp,stk.local_size

  mov bp,sp   ; BP needed to access the stack.

  mov ax,[bp + fstk.a]  ; let NASM calculate the offset!
  add ax,[bp + fstk.b]
  ...
  add sp,stk.local_size

  pop bp      ; epilogue needed!

  ret