Author Topic: How C compilers do their magic?  (Read 14972 times)

Offline fredericopissarra

  • Full Member
  • **
  • Posts: 373
  • Country: br
How C compilers do their magic?
« on: January 09, 2023, 05:21:50 PM »
To sumarize my argument against the use of C functions in an assembly "standalone" code, here's how C does its magic:

The main() function isn't the "main function". There is a lot of preparation before main() is called. There is no "dynamic allocable" space in an standalone assembly program, the structure and how it works must be done before main() starts. The same thing happens when you try to use functions as printf() or scanf(): C is locale aware and default locale information must be loaded before main(). Notice that printf() and scanf() uses, implicitly, the streams sdtout and stdin, respectively, and I'm not talking about the file descriptors 0 and 1, but pointers to FILE opaque structure. They must be initialized before main(). I think you got the idea: A lot of initialization is made before main() is called and a lot of finalization code runs after main() returns.

Take that and some more "excentric" things C functions must be aware of: signals, for instance and you'll begin to understand how slow are some "useful" C functions.

Am I saying that printf() is statically linked to your code? NO! Nowadays the standard C library is dynamically linked, even if early bound... BUT, all the state kept by the library is allocated in YOUR process address space, and all the library initialization and finalization is done there as well.

This means YOUR code is always called by the "C Runtime Library", usually a simple object file (like crt0.o in UNIX systems) or some code inside imported static libraries like msvcrt.lib.

There's no way an assembly code using a C library function could not use these initialization/finalization hidden codes... and, to use them, you must obey the operating system, and C libraries, ABI (Application Binary Interface). This means, for example, obeying stack pointer aligment before calling a function, pushing arguments in backward order (i386, for example), restoring the stack pointer after the function call (cdecl calling convention), etc... But also, to preserve EBX, ESI, EDI and EBP (i386 ABIs for Windows and Unixes) inside your functions (the caller assumes these registers aren't changed)... More: EAX (and, maybe EDX) is always used to return integer values from functions (including pointers), but EAX, EDX and ECX are completly free to be changed (you cannot rely on they previous values before calling a function, except if they are result values).

Take printf() as an example: The function's prototype is:
Code: [Select]
int printf( const char *fmt, ... );It means EAX is always changed by the routine, and, probably, ECX and EDX too. Since EBX, ESI, EDI and EBP are always preserved, you can rely on them, but bot EAX, ECX and EDX...

The entire point to do an assembly "pure" program is to do things run FASTER. The second point, less important, is to create SMALL routines. IMHO, if you use C functions because it is an easy thing to do, you are missing the point of what assembly programming is. A function like printf() is generic enough to be real slow... To print a simple 5 characters string takes, more or less, 200000 (200 thousands) cycles of clock. Compare this to a simple use of a system call on Linux, which is 20 times faster (still slow, but faster than printf).

BTW, C compilers tend to create better code than you do by hand. Take a simple function as an example:
Code: [Select]
_Bool is_divisible_by_4( int x ) { return ( x % 4 ) == 0; }A less experienced programmer can thing about converting this to:
Code: [Select]
_is_divisible_by_4:
  push ebp
  mov  ebp,esp

  mov  eax,[ebp+8]
  xor  edx,edx
  mov  ecx,4
  idiv  ecx
  mov  eax,0
  test edx,edx
  setz al

  pop  ebp
  ret
There are several things wrong with this:

1) Prolog/Epilog aren't necessary since the 80386;
2) IDIV is SLOW... On old processors (before Nehalem microarchiteture, at least) the instruction takes 100~200 clock cycles. On processors up to Tiger Lake microarchitecture it takes 30~40 cycles; And on modern processors, more or less, 17 cycles.
3) Division isn't necessary to get the remainder here...

Here's an "pure" assembly way of thinking in action:
Code: [Select]
; Input: EAX = signed int
; Output EAX=1 (divisible) or 0 (not divisible)
; OBS: Any integer divisible by 4 has the first 2 bits zeroed!
_is_divisible_by_4:
  and  eax,3  ; keep only bits 0 and 1 and affect ZF.
  setz al        ; set AL to 1 if ZF=1, 0 if ZF=0.
  ret
If you insist on using cdecl for i386 just add a mov eax,[esp+4] before the and instruction.

My argument is: There's no advantage using standard library C functions in your assembly code. The entire "runtime" must run for them to work properly. This is completely different from create a C function that don't use standard C library functions and calling them from an assembly routine. But this is usually interesting done with code compiled as freestanding, not hosted.

So, what you can do if converting an integer to a string (or a floating point) is what you need? Do your conversion routines... They are easy to do (specially integers) and uses way less space then the standard, generic, routines - You just have to think as an assembly programmer, not C.
« Last Edit: January 09, 2023, 05:26:34 PM by fredericopissarra »

Offline alCoPaUL

  • Jr. Member
  • *
  • Posts: 74
  • Country: ph
    • Webpage
Re: How C compilers do their magic?
« Reply #1 on: January 13, 2023, 05:00:42 PM »
puts() is a C Standard Library function and is used in almost Console Examples of Assembly Language Books.

and you're singling out printf().

like bro, printf() is only for displaying TEXTS on the CONSOLE.

you can do the string manipulations in pure assembly (even without repne scasb or whatnot) and display the result using printf(), after passing the address of the TEXT result to the functions (+switches).

and even with the +switches, at the end of the day, printf() just displays letters on the console..

it's not about the size. it's not about the speed.

your assembly linux source with SYSCALL won't run on windows. with mods, asm for windows that has puts(), printf(), scanf(), etc can be ported to macos coz almost all the OS has C Runtime Library installed.


Offline fredericopissarra

  • Full Member
  • **
  • Posts: 373
  • Country: br
Re: How C compilers do their magic?
« Reply #2 on: January 13, 2023, 08:15:58 PM »
puts() is a C Standard Library function...
Really?! You don't say!

like bro, printf() is only for displaying TEXTS on the CONSOLE.
Nope... printf() is for writing a formated string to stdout stream (that uses a file descriptor).

you can do the string manipulations in pure assembly (even without repne scasb or whatnot) and display the result using printf(), after passing the address of the TEXT result to the functions (+switches).

and even with the +switches, at the end of the day, printf() just displays letters on the console..

it's not about the size. it's not about the speed.
Of course it is about speed and size. Otherwise you wouldn't trying to use assembly and do the entire code in C.

You are missing the point... printf() is SLOW (20 times slower than using a system call or a direct operating system call [WriteConsoleA from Win32 Console API, for example) and it is big, inneficient...

your assembly linux source with SYSCALL won't run on windows. with mods, asm for windows that has puts(), printf(), scanf(), etc can be ported to macos coz almost all the OS has C Runtime Library installed.
Of course a Linux code cannot work on Windows. Assembly isn't portable at all, not even between different modes of operation of the same processor (real, i386 and x86-64 modes are examples). Or are you saying that because you use C Standard Library your i386 code will, by magic, run on a Raspberry PI?
« Last Edit: January 13, 2023, 08:17:47 PM by fredericopissarra »

Offline alCoPaUL

  • Jr. Member
  • *
  • Posts: 74
  • Country: ph
    • Webpage
Re: How C compilers do their magic?
« Reply #3 on: January 15, 2023, 10:19:03 PM »
64 bit winasm and 32 bit winasm can coexist in windows subsystem and you can make each of those do the same thing. however, you cannot run your  asm bloated with syscalls program natively on windows and vise versa if your winasm code uses messageboxa and exitprocess.

you can make it appear that winasm and opensource asm share portability by them both using the c standard library functions AND THIS  PORTABILITY IS AT SOURCE LEVEL since in both subsystems, the c standard lib functions are exposed but you still have to assemble or link them separately and observe each of the subsystems' intricacies.
« Last Edit: January 15, 2023, 10:24:04 PM by alCoPaUL »

Offline fredericopissarra

  • Full Member
  • **
  • Posts: 373
  • Country: br
Re: How C compilers do their magic?
« Reply #4 on: January 16, 2023, 11:45:06 AM »
64 bit winasm and 32 bit winasm can coexist in windows subsystem and you can make each of those do the same thing. however, you cannot run your  asm bloated with syscalls program natively on windows and vise versa if your winasm code uses messageboxa and exitprocess.
Yes... Windows (and LINUX) can run 32 bits applications in a 64 bits environment, but NO, they are NOT the same. They don't share the same calling convention, the same set of registers, ...

you can make it appear that winasm and opensource asm share portability by them both using the c standard library functions AND THIS  PORTABILITY IS AT SOURCE LEVEL since in both subsystems, the c standard lib functions are exposed but you still have to assemble or link them separately and observe each of the subsystems' intricacies.
Again, NO, you can't, here's why (ON WINDOWS):
Code: [Select]
; 32 bits call of puts().
  push msg
  call puts
  add esp,4

; 64 bits call of puts
  lea  rcx,[msg]
  call puts
...
msg:
  db `Bleargh!\n`,0
NOT the same... NOT portable...
Assembly ISN'T portable.
« Last Edit: January 26, 2023, 10:26:05 AM by fredericopissarra »

Offline alCoPaUL

  • Jr. Member
  • *
  • Posts: 74
  • Country: ph
    • Webpage
Re: How C compilers do their magic?
« Reply #5 on: January 26, 2023, 12:46:39 AM »
bro

; by alCoPaUL [GIMO][As][aBrA][NPA][b8][BCVG][rRlf]
; January 25, 2023 NYC
; nasm <dash>f macho <dash>o GosuIchiban.o SnowLeopard.asm
; ld <dash>arch i386 <dash>macosx_version_min 10.4 <dash>lc /usr/lib/crt1.o <dash>o GesuIchIvan GosuIchiban.o
global _main
extern _printf
section .text
_main:
enter 0,0
mov al,10
mov bl,45
z:lea edx,[a]
mov cx,596
r:cmp byte [edx],bl
je s
jmp u
s:mov byte [edx],al
u:inc edx
dec cx
cmp cx,0
jnz r
push a
push i
call _printf
mov al,10
cmp bl,45
xchg al,bl
je z
push x
push i
call _printf
leave
ret
section .data
x:db 2Ch,32h,37h,68h,2Ch,30h,0
i:db 25h,73h,0
a:db '; by alCoPaUL [GIMO][As][aBrA][NPA][b8][BCVG][rRlf]-; January 25, 2023 NYC-; nasm <dash>f macho <dash>o GosuIchiban.o SnowLeopard.asm-; ld <dash>arch i386 <dash>macosx_version_min 10.4 <dash>lc /usr/lib/crt1.o <dash>o GesuIchIvan GosuIchiban.o-global _main-extern _printf-section .text-_main:-enter 0,0-mov al,10-mov bl,45-z:lea edx,[a]-mov cx,596-r:cmp byte [edx],bl-je s-jmp u-s:mov byte [edx],al-u:inc edx-dec cx-cmp cx,0-jnz r-push a-push i-call _printf-mov al,10-cmp bl,45-xchg al,bl-je z-push x-push i-call _printf-leave-ret-section .data-x:db 2Ch,32h,37h,68h,2Ch,30h,0-i:db 25h,73h,0-a:db ',27h,0


---------------

Use the preinstalled NASM in Snow Leopard coz the NASM posted online have lelz meme.

https://forum.nasm.us/index.php?topic=3597.0
« Last Edit: January 26, 2023, 02:11:52 AM by alCoPaUL »

Offline fredericopissarra

  • Full Member
  • **
  • Posts: 373
  • Country: br
Re: How C compilers do their magic?
« Reply #6 on: January 26, 2023, 12:45:03 PM »
Suggestion: LEARN how to program in assembly first.

Offline alCoPaUL

  • Jr. Member
  • *
  • Posts: 74
  • Country: ph
    • Webpage
Re: How C compilers do their magic?
« Reply #7 on: January 26, 2023, 04:19:30 PM »
you get the REL32 error msg?? try using the first/earliest NASM that was available for MacOS...

and what i posted was assembled in NASM and not linked in GCC but LD, right?

we don't target free software. we target windows and macosx, which are still closed source and macosx is certified UNIX-like by the Standard.

Offline fredericopissarra

  • Full Member
  • **
  • Posts: 373
  • Country: br
Re: How C compilers do their magic?
« Reply #8 on: January 26, 2023, 05:09:23 PM »
you get the REL32 error msg?? try using the first/earliest NASM that was available for MacOS...

and what i posted was assembled in NASM and not linked in GCC but LD, right?

we don't target free software. we target windows and macosx, which are still closed source and macosx is certified UNIX-like by the Standard.
Nope... because this is a shitty code... here, improved:
Code: [Select]
; ShiTTy alCoPaul cODe dOnE RIGHT.
; For i386 (Linux AND Windows)
;
;   Compile and link:
;
;     WINDOWS (using mingw32):
;       $ nasm -DWINDOWS -fwin32 -o test.o test.asm
;       $ i686-w64-mingw32-ld -s -o test.exe test.o -lkernel32
;
;     LINUX:
;       Don't define WINDOWS and use the 32 bits ld.
;       
  bits    32

%ifdef WINDOWS
  %define STD_OUTPUT_HANDLE -11
%else
  %define SYS_EXIT  1
  %define SYS_WRITE 4
%endif

  section .data

str:
  db  `A lot of bull**** separated by \"<dash>\" is substituted-`
  db  `by '\\n'--`
str_len equ $ - str

  section .text

  global  _start

  %ifdef WINDOWS
    extern __imp__GetStdHandle@4
    extern __imp__WriteConsoleA@20
    extern __imp__ExitProcess@4
  %endif

_start:
  ; scans and change all '-' for '\n'.
  lea   edi,[str]
  mov   ecx,str_len
  mov   al,'-'          ; char to scan for.
  jmp   .test

  align 4
.loop:
  repne scasb
  cmp   byte [edi-1],al
  jne   .test
  mov   byte [edi-1],`\n`
.test:
  test  ecx,ecx
  jnz   .loop

  ; finally prints the shitty string.
  %ifdef WINDOWS

    push  STD_OUTPUT_HANDLE
    call  [__imp__GetStdHandle@4]

    xor   ebx,ebx
    push  ebx
    push  ebx
    push  str_len
    push  str
    push  eax
    call  [__imp__WriteConsoleA@20]

    push  ebx
    jmp   [__imp__ExitProcess@4]

  %else

    mov   eax,SYS_WRITE
    mov   ebx,1           ; stdout
    lea   ecx,[str]
    mov   edx,str_len
    int   0x80
 
    mov   eax,SYS_EXIT
    xor   ebx,ebx
    int   0x80

  %endif

And works with any NASM and i386 ld, FOR WINDOWS AND LINUX.

MacOS is crap as well...

Offline fredericopissarra

  • Full Member
  • **
  • Posts: 373
  • Country: br
Re: How C compilers do their magic?
« Reply #9 on: January 26, 2023, 05:41:22 PM »
bro
I'm NOT your "bro"...

Quote
; by alCoPaUL [GIMO][As][aBrA][NPA][b8][BCVG][rRlf]
; January 25, 2023 NYC
; nasm <dash>f macho <dash>o GosuIchiban.o SnowLeopard.asm
; ld <dash>arch i386 <dash>macosx_version_min 10.4 <dash>lc /usr/lib/crt1.o <dash>o GesuIchIvan GosuIchiban.o
And you are missing completely the point... This unformated code don't work on x86-64 mode for THE SAME PROCESSOR, for example.

SAME 80x86 processor for the SAME operating system, but different mode of operation. Different assembly code... too much for "portability", isn't it?

On i386 mode you have to push the arguments to the variadic function printf()... for x86-64 mode you have to use registers (DIFFERENT registers on WINDOWS & LINUX, for example)! Worse even: On i386 mode the identifier must have that ugly _ prefix, but not on x86-64 mode!
« Last Edit: January 26, 2023, 05:58:09 PM by fredericopissarra »

Offline alCoPaUL

  • Jr. Member
  • *
  • Posts: 74
  • Country: ph
    • Webpage
Re: How C compilers do their magic?
« Reply #10 on: January 26, 2023, 06:01:07 PM »
you tell me to learn Assembly and attached is the 5th Source Code specifically for you.

like have you seen similar codes that i output on your Open Source Forums or elsewhere on the internet? or was it like COVID that appeared from nowhere like printf~wtf bro lelz..

; by alCoPaUL [GIMO][As][aBrA][NPA][b8][BCVG][rRlf]
; January 26, 2023 NYC
; ./nasm <dash>f macho64 <dash>o 6GosuIchiban4.o SnowLeopard64.asm
; ld <dash>arch x86_64  <dash>macosx_version_min 10.4 <dash>lc /usr/lib/crt1.o <dash>o 6GesuIchIvan4 6GosuIchiban4.o
global _main
extern _printf
section .text
_main:push rbx
mov cl,10
mov bl,45
z:lea r9,[a]
mov dx,685
r:cmp byte[r9],bl
je s
jmp u
s:mov byte[r9],cl
u:inc r9
dec dx
cmp dx,0
jnz r
...
[REST OF THE CODE IS ATTACHED]

YO, BOTH OF THE MACOS CODES (32 & 64 BITS) will Assemble successfully on NASM 2.01RCs under MacOSX Snow Leopard. and linked with what? LD?

so do i need to install the runtimes in MacOS for the quines to do their thing?

eat sh1t bro, ahahahha.

and now you're butthurt calling my previous code shitty but who thought of that first? you reply and i slap you with the 64 bit version?

ahahaha eat sh1t bro..

well my 5 codes (16, 32, 64 bit windows and 32,64 bit MacOS) run natively on their respective subsystem.

and you're using mingw 32 bit to prove my natively running codes sh1t? like the f***..

35 years of assembly language programming got fAcked with 5 quines.
« Last Edit: January 26, 2023, 07:41:52 PM by alCoPaUL »

Offline fredericopissarra

  • Full Member
  • **
  • Posts: 373
  • Country: br
Re: How C compilers do their magic?
« Reply #11 on: January 26, 2023, 06:21:39 PM »
you tell me to learn Assembly and attached is the 5th Source Code specifically for you.
I see you learn nothing at all...

Are you saying this x86-64 crap (still unformated) code runs in 32 bits as well? If not, why you call it "portable"?

Quote
35 years of assembly language programming got fAcked with 5 quines.
That's hilarious comming from someone who don't know how to use an instruction that exists since the first 8086!

NASM 2.01? BWAHAHAHAHA!

Offline debs3759

  • Global Moderator
  • Full Member
  • *****
  • Posts: 224
  • Country: gb
    • GPUZoo
Re: How C compilers do their magic?
« Reply #12 on: January 26, 2023, 06:51:54 PM »
Guys, if all you can do is argue, please ignore each other.

Fred is one of our most helpful members, sharing good programming code and tips with a lot of people.

Insults help nobody. I don't want to delete posts that are trying to be helpful, or offering/requesting help, but I will if I keep seeing insults thrown around.
My graphics card database: www.gpuzoo.com

Offline alCoPaUL

  • Jr. Member
  • *
  • Posts: 74
  • Country: ph
    • Webpage
Re: How C compilers do their magic?
« Reply #13 on: January 26, 2023, 07:06:47 PM »
debs, ok. ill be civil from now on but try to examine his attitude to the things i post. it's like he's getting that anger tick everytime that i post something..

sorry bout that verbal wrestling.

and thanks for reminding..

anyway, yes NASM 2.01 coz like i only have a Snow Leopard box and the built -in NASM available there which is inhouse Compiled by Apple Computers is version 0.98 (NASM.us even started on NASM 0.99 as base download)...

NASM 2.01 already supports MACHO64 assembly and natively run on MacOSX and try to assemble the one that is attached...

then LD it and ./6GesuIchIvan4 it..
« Last Edit: January 26, 2023, 07:39:23 PM by alCoPaUL »

Offline fredericopissarra

  • Full Member
  • **
  • Posts: 373
  • Country: br
Re: How C compilers do their magic?
« Reply #14 on: January 26, 2023, 07:19:46 PM »
What real "portability" looks like using assembly:
Code: [Select]
; test.asm
;
;   Compile and link:
;
;     WINDOWS:
;       $ nasm -f win32|win64 -o test.o test.asm
;       $ i686-w64-mingw32-ld -s -o test.exe test.o -lkernel32 #win32
;       % x86_64-w64-mingw32-ld -s -o test.exe test.o -lkernel32 #win64
;
;     LINUX:
;       $ nasm -f elf32|elf64 -o test.o test.asm
;       $ ld -s -o test test.o
;       

%ifidn __?OUTPUT_FORMAT?__, win32
  bits    32
  %define OK2CONTINUE
%elifidn __?OUTPUT_FORMAT?__, elf32
  bits    32
  %define OK2CONTINUE
%endif

%ifidn __?OUTPUT_FORMAT?__, win64
  bits    64
  default rel
  %define OK2CONTINUE
%elifidn __?OUTPUT_FORMAT?__, elf64
  bits    64
  default rel
  %define OK2CONTINUE
%endif

%ifndef OK2CONTINUE
  %error "Only win32, win64, elf32 and elf64 allowed."
%endif

  section .data

str:
  db  `A lot of bull**** separated by \"<dash>\" is substituted-`
  db  `by '\\n'--`
str_len equ $ - str

  section .text

  ; Win32 API functions from kernel32.dll (different identifiers
  ; for different modes of operation).
  %ifidn __?OUTPUT_FORMAT?__, win32
    extern __imp__GetStdHandle@4
    extern __imp__WriteConsoleA@20
    extern __imp__ExitProcess@4
  %elifidn __?OUTPUT_FORMAT?__, win64
    extern __imp_GetStdHandle
    extern __imp_WriteConsoleA
    extern __imp_ExitProcess
  %endif

  global  _start

  align 4
_start:
  ; Different pointer sizes.
  %if __?BITS?__ == 32
    lea   edi,[str]
  %else
    lea   rdi,[str]
  %endif

  mov   ecx,str_len
  mov   al,'-'          ; char to scan for.
  jmp   .test

  align 4
.loop:
  repne scasb
  %if __?BITS?__ == 32
    cmp   byte [edi-1],al
  %else
    cmp   byte [rdi-1],al
  %endif

  jne   .test

  %if __?BITS?__ == 32
    mov   byte [edi-1],`\n`
  %else
    mov   byte [rdi-1],`\n`
  %endif

.test:
  test  ecx,ecx
  jnz   .loop

  ; finally prints the shitty string.
  ; With different calling conventions from Win32, Win64, elf32 and elf64!
  %ifidn __?OUTPUT_FORMAT?__, win32
    push  -11                       ; STD_OUTPUT_HANDLE
    call  [__imp__GetStdHandle@4]

    xor   ebx,ebx
    push  ebx
    push  ebx
    push  str_len
    push  str
    push  eax
    call  [__imp__WriteConsoleA@20]

    push  ebx
    jmp   [__imp__ExitProcess@4]
  %elifidn __?OUTPUT_FORMAT?__, win64
    mov   ecx,-11               ; STD_OUTPUT_HANDLE
    call  [__imp_GetStdHandle]

    mov   rcx,rax
    lea   rdx,[str]
    mov   r8d,str_len
    xor   r9d,r9d
    push  r9
    call  [__imp_WriteConsoleA]

    xor   ecx,ecx
    jmp   [__imp_ExitProcess]   
  %endif

  %ifidn __?OUTPUT_FORMAT?__, elf32
    mov   eax,4           ; sys_write
    mov   ebx,1           ; stdout
    lea   ecx,[str]
    mov   edx,str_len
    int   0x80
 
    mov   eax,1           ; sys_exit
    xor   ebx,ebx
    int   0x80
  %elifidn __?OUTPUT_FORMAT?__, elf64
    mov   eax,1           ; sys_write
    mov   edi,eax         ; stdout (reusing from eax).
    lea   rsi,[str]
    mov   edx,str_len
    syscall

    mov   eax,60          ; sys_exit
    xor   edi,edi
    syscall
  %endif
I suspect MacOS follows SysV ABI as well, but cannot be sure (don't have how to test).

The point is: For the same processor (80x86) we have:

1 - Different calling conventions between different operating systems and, with the same, between different modes of operation;
2 - Different imported identifier names (even with C functions, i386 prefixes with _, x86-64 don't).

So, no... assembly isn't portable, unless you fill your code with conditionals...

Even for the above, supposed "portable" code, there is a problem... If '-' is changed to '\n', this don't work very well on Windows Console, which expects "\r\n". Yep, printf() or fputs() deal with this issue, but at a huge cost (a bloated final code) and with a different calling convention on each mode/operating system. And if someone is using libc functions, it is better to develop the entire program in C (nothing to gain using asssembly, since, like I said before, the C compiler probably will do a way better job than a manually created routine).

THOSE are my points...

If one wants to use libc or not it is NOT my business, but to say that practice is "portable" is misleading.