Well, these high level examples are too complicated for my simple brain. I'd rather start at the bottom and work up. Since the question asks for "help with stack", maybe we should start with the fact that the "call" instruction puts the return address on the stack (and "ret" removes it - It had better be there!). Our function is going to need a parameter, so we can push that on the stack first...
push 15
call func
; XXX
func:
; do nothing
ret
Encryptor256 showed you a slightly different example, which ended in "ret 4". The difference is in calling conventions. The "stdcall" convention expects the called function to clean up the stack (used by Windows APIs). The "cdecl" convention (usually used by C) expects the caller to clean up the stack. The parameter(s) has to be "removed" from the stack. It isn't really "removed" we just move esp above it and reuse the space. Where the "XXX" comment is, we need to "add esp, 4". If you're using "stdcall", do nothing there.
We need to return some value - that just goes in eax - and we need to access our parameter. We can do that by means of a "stack frame". First, we push the caller's ebp. In this case, the caller isn't using ebp but generally it will be. Then we load ebp with the current value of esp. From there on, leave ebp alone! We can alter esp, however. For example, eventually this monster is going to call itself - twice! We might want a temporary variable - unique to this invocation of the function - to store the first result while we call it again. We can make such a temporary variable on the stack...
push 15
call func
add esp, 4
; display result
; exit cleanly
func:
push ebp ; save caller's ebp
mov ebp, esp ; set our stack frame pointer
sub esp, 4 ; make room for a temporary variable
; at this point the stack looks like:
; our parameter
; return address
; caller's ebp <- ebp points here
; room for temp <- esp points here
mov eax, [ebp + 8] ; fetch our parameter
; do some logic here
; we'll just return the parameter we were passed
; first, we'll save it in our temporary variable
mov [ebp - 4], eax
mov eax, [ebp - 4] ; for no good reason
mov esp, ebp ; undo our stack frame - this "frees" our temporary variable
pop ebp ; restore caller's ebp
ret
As I understand the "logic", if our parameter is 0 or 1, we want to return 1. If the parameter is 2, we want to return 7. If it is 3 or more, things get complicated. We need to prepare a new parameter, "n - 3", push it, call ourself, clean up stack, save the result in our temporary variable, prepare another parameter, "n - 2" (if we've lost track of "n", we can get it back from the stack "mov eax, [ebp + 8]"), push it, call ourself, clean up stack, add the first result from the temporary variable, and return that. Whew!
If I've got it right, f(15) should be 145, right? I'll post mine (very simple minded one for 32-bit Linux), but let's see what you can come up with first...
Best,
Frank