Sys_brk effects on stack

[appina]

Hi,

I'm currently writing a little piece of code that extends the current bss segment of the program to make room for dynamically allocated data. I'm running 32-bit x86 Ubuntu. I use system call sys_brk to make room for the heap. Now my question is that should I modify stack base pointer and stack pointer in registers ebp and esp after the system call. According to my understanding sys_brk extends the bss segment towards stack meaning that if heap size is high enough my stack pointers end up pointing to some memory locations inside my heap area. This means I should move my stack pointers to point a memory locations outside the heap but in this part I bumped into a problem. I keep getting a segmentation fault and I have tracked the problem to the instructions in which I set stack pointer to point same location as stack base pointer. Should I even be doing this "stack adjustment" or is my functionality faulty?

Please don't pummel me to death if my source looks horrible. I started programming with assembly four days ago and there is quite a lot of stuff I don't know about assembly. Healthy criticism and improvement ideas are always welcome. I tried to use as much comments as I could to make the code even remotely understandable (at least I tend to forget what I was doing without them).

Code: [Select]

sys_brk: equ 45

; initializes heap of size specified in register ecx
; stack will be corrupted during this operation
global setupheap

; cleanups previously initialized heap
; stack will be corrupted during this operation
global cleanupheap

section .text
setupheap:
mov edx, ebp ; set stack base pointer to edx

; get brk location to eax
xor ebx, ebx
mov eax, sys_brk
int 0x80

sub edx, eax ; calculate difference of brk location and
; stack base pointer to edx = stack size

; extend bss section
mov [obrk], eax ; store original brk location to obrk
mov ebx, ecx ; put heap size to ebx from ecx
add ebx, eax ; calculate new brk location
mov eax, sys_brk
int 0x80

mov [cbrk], eax ; store new brk location to cbrk

pop eax ; store old value of eip to eax

; calculate new stack location
mov ebp, edx ; put stack size to ebp
add ebp, [cbrk] ; calculate new stack base pointer
mov esp, ebp ; set stack pointer to point to base of stack

push eax ; push old value of eip from eax to stack to
; allow corrent return

ret

cleanupheap:
mov edx, ebp ; set stack base pointer to edx
sub edx, [cbrk] ; calculate stack size

; shrink bss section back to it's original size
mov ebx, [obrk]
mov eax, sys_brk
int 0x80

mov [cbrk], eax ; store new brk location to cbrk

pop eax ; store old value of eip to eax

mov ebp, edx ; put stack size to ebp
add ebp, [cbrk] ; calculate new stack base pointer
mov esp, ebp ; set stack pointer to point to base of stack

push eax ; push old value of eip from eax to stack to
; allow corrent return
ret

section .bss
obrk: resb 4
cbrk: resb 4



[appina]

Okay, I have done some research and talked to a more experienced programmer and it turns out that I have tried to fix a non-existing issue. I misunderstood how program layouts in memory and my worries about overwriting stack area with the heap have been pointless. Also the concept of extending bss segment is incorrect since brk location is not equal to end of bss segment. I have rewritten my source and this should do the job (without any effect on stack ). I wonder how many facepalms I just caused.

Code: [Select]

sys_brk: equ 45

; initializes heap of size specified in register eax
; stack will be corrupted during this operation
global setupheap

; cleanups previously initialized heap
; stack will be corrupted during this operation
global cleanupheap

section .text
setupheap:
mov edx, eax ; store heap size into edx

; get current brk location
xor ebx, ebx
mov eax, sys_brk
int 0x80

mov [obrk], eax ; store original brk location to obrk
mov ebx, edx ; put heap size into ebx
add ebx, eax ; add original brk location to ebx
mov eax, sys_brk
int 0x80

mov [cbrk], eax ; store new brk location to cbrk

ret

cleanupheap:
; move brk back to it's original position
mov ebx, [obrk]
mov eax, sys_brk
int 0x80

mov [cbrk], eax

ret

section .bss
obrk: resb 4
cbrk: resb 4



[Frank Kotler]

Hi Appina,

I'm glad to hear you got it sorted out. I haven't tried either version, but I can see that the first one isn't going to work well. You pop the return address, move esp someplace else, push the return address back, and "ret". That "ret" will work, but when we get back to the caller, the caller's stack is gone forever. Also you alter the caller's ebp, which is probably not a good idea. The second version looks good to me.

I agree with you about the wisdom of commenting code. It's amazing how quickly I forget "What was I thinking?" It might be a good idea to comment each subroutine with what it "expects", what it "returns", and what it "trashes"... and other side-effects of note. (I don't always do it, but I always think it's a good idea) Almost as bad as not enough comments is obsolete comments that apply to code that no longer exits. Just sayin'...

You confuse me when you say that the "break" isn't the end of the .bss section. I thought it was (first byte after the .bss section actually). What have you learned? A question that has just come to my mind... As an alternative to allocating memory with sys_brk, we can allocate memory with sys_mmap (mmap2 actually, I think). This memory is not contiguous with .bss, but starts up around 0x60000000 I think. Where's my "break" now?

Another "good idea" is to check for error after int 0x80 (or anything else that could return an error). I've just learned ("man 2 brk" under "Linux notes") that sys_brk doesn't return an error number if it fails, but just returns the current "break". Glib C wraps this in a check to see if the return was less than we asked for, and sets ENOMEM if so. We should probably do the same.

Lots to learn! Hope to see you around!

Best,
Frank

[appina]

Hi Frank,

good to hear that the second version looks good. About error checking and such, is there a standard way to return values from function or should I just swing it? I have been using eax register as return register. So is it a good practice if I put -1 to eax when error occurs and 0 or new brk location when everything goes okay?

According to my current understanding due the virtual memory and address space randomization the operating system applies small random offset to starting addresses of stack, memory mapping segment and heap. I was given link to this website (http://duartes.org/gustavo/blog/post/anatomy-of-a-program-in-memory) which describes pretty well how programs layout in memory. Of course it is possible that I have misunderstood something again but it sure was good reading.

It is highly probable that I'm around here asking questions in near future I'm planning to implement memory management without using c lib.

[encryptor256]

Hi!

I'm planning to implement memory management without using c lib.

There, might be, one more thing, to think about.
If i'm not mistaken, for example, c function named "malloc", on win64,
always returns a memory address, that is aligned on 16 byte boundary.

But of course it is aligned on some boundaries, then all structure padding garbage would be pointless.

I think, memory management, should be considered when building an os.

If you are using os and it already have a memory management, then,
Why do you need a memory management at all? You don't trust malloc?

EDIT0:
Sci-Fi linux assembly call: sys_berserker.
Programmer: "Call Berserker, Call Berserker!!!" 

[appina]

Hi encryptor256!

It is not that I don't trust malloc or anything like that Reason for my unwillingness to use malloc is the opportunity to learn by doing. It is extremely intriguing to try to figure out how to solve issues with memory management for example external and internal fragmentation of the heap.

[munair]

Hi encryptor256!

It is not that I don't trust malloc or anything like that Reason for my unwillingness to use malloc is the opportunity to learn by doing. It is extremely intriguing to try to figure out how to solve issues with memory management for example external and internal fragmentation of the heap.

While this is an old topic, I would like to add that I perfectly understand why someone prefers not to use C libraries in asm. I am currently developing asm routines for a compiler and mixing asm with C just doesn't feel right somehow. Of course, using C libs would be a quick short cut, but there is less/no satisfaction in doing that. In my particular case, I want a pure language (the compiler) with nasm / ld as backend without having to lean on C or any other high level language. I suppose it is really a matter of taste.

[munair]

Here is a nice link that demonstrates memory management in asm: https://github.com/TstevetsT/ListMalloc