Win64 Structure Alignment (Win API)

[encryptor256]

Progress!  Structure and union definitions ( including nested ) and natural size alignment are now fully implemented!

I stink think:

• I found an Easter Egg.
• NASMX_STRUC macro, forgot to define, a structure size.

In this thread, i posted, some where above, file named: "win64_demo1.asm",
it show's that CREATESTRUCT_size variable is and was recognized.

But current test, on "Commit [r202]" (Latest snapshot), shows that something went\gone wrong,
OR at least, there is some, new "structure size" variable.

Just look, code file: "nasmx-code-202-trunk\demos\win64\DEMO1\demo1.asm", was updated to:

Code: [Select]

;// DEMO1.ASM
;//
;// Copyright (C)2005-2011 The NASMX Project
;//
;// This is a fully UNICODE aware, typedefined demo that demonstrates
;// using NASMX typedef system to make your code truly portable between
;// 32 and 64-bit systems using either ASCII or UNICODE
;//
;// Contributors:
;//    Bryant Keller
;//    Rob Neff
;//
%include 'windemos.inc'

entry    demo1

[section .code]

proc   demo1, ptrdiff_t argcount, ptrdiff_t cmdline
locals none

    ; CREATESTRUCT
    ; -----------------------------------------------------------------------------------
    invoke wsprintfA,buffer,txt_number,txt_CREATESTRUCT,CREATESTRUCT.lpCreateParams
    invoke wsprintfA,buffer,txt_number,txt_CREATESTRUCT,CREATESTRUCT_size
    invoke wsprintfA,buffer,txt_number,txt_CREATESTRUCT,CREATESTRUCT_Size
    invoke wsprintfA,buffer,txt_number,txt_CREATESTRUCT,CREATESTRUCT_SIZE
    invoke MessageBoxA, NULL, buffer, NULL, MB_OK
    invoke ExitProcess, NULL

endproc

[section .data]
    buffer: times 250 db 0
    txt_number:  declare(NASMX_TCHAR) NASMX_TEXT('%s, %d'), 0x0
    txt_CREATESTRUCT:  declare(NASMX_TCHAR) NASMX_TEXT('CREATESTRUCT'), 0x0

NASMX_STRUC CREATESTRUCT
    NASMX_RESERVE lpCreateParams, ptrdiff_t, 1
    NASMX_RESERVE hInstance,  ptrdiff_t, 1
    NASMX_RESERVE hMenu,      ptrdiff_t, 1
    NASMX_RESERVE hWndParent, ptrdiff_t, 1
    NASMX_RESERVE cy,         int32_t, 1
    NASMX_RESERVE cx,         int32_t, 1
    NASMX_RESERVE y,          int32_t, 1
    NASMX_RESERVE x,          int32_t, 1
    NASMX_RESERVE style,      int32_t, 1
    NASMX_RESERVE lpszName,   ptrdiff_t, 1
    NASMX_RESERVE lpszClass,  ptrdiff_t, 1
    NASMX_RESERVE ExStyle,    int32_t, 1
NASMX_ENDSTRUC

AND,
It produces, compile output:

Code: [Select]

DEMO1.asm:25: error: symbol `CREATESTRUCT_size' undefined
DEMO1.asm:26: error: symbol `CREATESTRUCT_Size' undefined
DEMO1.asm:27: error: symbol `CREATESTRUCT_SIZE' undefined

Usage of variable "CREATESTRUCT.lpCreateParams", proves, that CREATESTRUCT is defined.
So, i tried at least three common possible "struct size" variable definitions and none of them was\were recognized.

Might be caused by...
OLD code, nasmx.inc, NASMX_STRUCT, here we could define CREATESTRUCT_size (%$__nx_size):

Code: [Select]

        ....
        ;// create new ctx and initialize
%push %3
%xdefine %$__nx_name %%__tname
%assign %$__nx_size %%__tsize
        ...
NEW code, nasmx.inc, NASMX_STRUCT, here is no such thing (%$__nx_size):

Code: [Select]

        ...
        ;// create new ctx and initialize
%push %3
%xdefine %$__nx_name %%__tname
%define %$__nx_isnested %%__is_nested
        ...

So, i got no clue, how to get "structure size".

[Rob Neff]

Ahh, the difference between nasm's %define and equ!

Compare this small snippet:

Code: [Select]

    mov eax, CREATESTRUCT_size
    %define CREATESTRUCT_size 48

which generates 'error: symbol `CREATESTRUCT_size' undefined'

to this:

Code: [Select]

    mov eax, CREATESTRUCT_size
    CREATESTRUCT_size equ 48

which assembles correctly.

CREATESTRUCT_size should probably be an equate.  But in the current revision it was %define'd.  You brought that to light since you placed the definition of the CREATESTRUCT in the data segment rather than at the top of the file in an include-like fashion.  Subsequently, the use of the CREATESTRUCT_size was unknown at the point you attempted to invoke.  So I'll fix that for sure.

A nice feature to add would be a sizeof() operator.  One is used internally in NASMX but with extreme caution.
Consider a structure member offset - CREATESTRUCT.lpCreateParams - which is an offset of zero (0) from the beginning of the structure.  It gets equated to zero and becomes unmodifiable at that point.  Regardless of whether it is defined or equated we can't simply do a sizeof(CREATESTRUCT.lpCreateParams) in order to find the size of that member variable.  It would be the same as calling sizeof(0)!  That's why nasm, and subsequently NASMX, append _size to the structure names and members before any equating occurs.  It's sort of like a reversed C sizeof() macro.

[encryptor256]

Ahh, the difference between nasm's %define and equ!

Ahh, yes.
Okay, okay, now it's clear, now it's works.

A nice feature to add would be a sizeof() operator.

Yes, it would eliminate, the need of typing/appending with "_size".
But, now, it's possible to live with out this operator.
That sizeof(X) operator could be like a new feature, in case, it get's verified and is not needed to be use with caution.

EDIT0:

Hey, look at this:

Code: [Select]

;// DEMO1.ASM
;//
;// Copyright (C)2005-2011 The NASMX Project
;//
;// This is a fully UNICODE aware, typedefined demo that demonstrates
;// using NASMX typedef system to make your code truly portable between
;// 32 and 64-bit systems using either ASCII or UNICODE
;//
;// Contributors:
;//    Bryant Keller
;//    Rob Neff
;//
%include 'windemos.inc'

entry    demo1

[section .code]

proc   demo1, ptrdiff_t argcount, ptrdiff_t cmdline
locals none

   
    ; SOCKADDR_IN
    ; -----------------------------------------------------------------------------------
invoke wsprintfA,buffer,txt_format_offset,buffer,SOCKADDR_IN_TEST.sin_family
    invoke wsprintfA,buffer,txt_format_offset,buffer,SOCKADDR_IN_TEST.sin_port
    invoke wsprintfA,buffer,txt_format_offset,buffer,SOCKADDR_IN_TEST.sin_addr
    invoke wsprintfA,buffer,txt_format_offset,buffer,SOCKADDR_IN_TEST.sin_addr.S_un
    invoke wsprintfA,buffer,txt_format_offset,buffer,SOCKADDR_IN_TEST.sin_addr.S_un.S_addr
    invoke wsprintfA,buffer,txt_format_offset,buffer,SOCKADDR_IN_TEST.sin_addr.S_un.S_un_b
    invoke wsprintfA,buffer,txt_format_offset,buffer,SOCKADDR_IN_TEST.sin_addr.S_un.S_un_w
    invoke wsprintfA,buffer,txt_format_offset,buffer,SOCKADDR_IN_TEST.sin_zero
    invoke MessageBoxA, NULL, buffer, NULL, MB_OK

endproc

[section .data]
    buffer: times 250 db 0
    txt_format_offset: db '%s',13,10,'offset: %d', 0x0
    txt_format_init:  declare(NASMX_TCHAR) NASMX_TEXT('%s, %d'), 0x0
    txt_CREATESTRUCT:  declare(NASMX_TCHAR) NASMX_TEXT('CREATESTRUCT'), 0x0



NASMX_STRUC SOCKADDR_IN_TEST
    NASMX_RESERVE sin_family, int16_t, 1
    NASMX_RESERVE sin_port, uint16_t, 1
    NASMX_STRUC sin_addr, IN_ADDR
        NASMX_UNION S_un
            NASMX_STRUC S_un_b
                NASMX_RESERVE s_b1, uint8_t, 1
                NASMX_RESERVE s_b2, uint8_t, 1
                NASMX_RESERVE s_b3, uint8_t, 1
                NASMX_RESERVE s_b4, uint8_t, 1
            NASMX_ENDSTRUC
            NASMX_STRUC S_un_w
                NASMX_RESERVE s_w1, uint16_t, 1
                NASMX_RESERVE s_w2, uint16_t, 1
            NASMX_ENDSTRUC
    NASMX_RESERVE S_addr,   uint32_t, 1
        NASMX_ENDUNION
    NASMX_ENDSTRUC
    NASMX_RESERVE sin_zero, int8_t, 8
NASMX_ENDSTRUC

Output is a message box:

Code: [Select]

offset: 0
offset: 2
offset: 4
offset: 4
offset: 4
offset: 4
offset: 4
offset: 8


Subsequently, the use of the CREATESTRUCT_size was unknown at the point you attempted to invoke.

Here is difference, define vs equ :
1. Structure member offsets is\are know at the time they are invoked.
2. But structure size is not.

So I'll fix that for sure.

Okay.

I think the "fix" will be:
back to definition with EQU
OR
requirement, that structures must have to be defined before programmer access them.

[Rob Neff]

We could make it work but with a required syntax change.

Let's use the following definition as an example:

Code: [Select]

NASMX_STRUC POINT
    NASMX_RESERVE x, uint16_t, 1
    NASMX_RESERVE y, uint16_t, 1
NASMX_ENDSTRUC

Let's create two macros:

Code: [Select]

    %define sizeof(x) ( x %+ _size )
    %define offsetf(x) ( x %+ _offset )

The key to this even working properly is that we now, instead of:

Code: [Select]

    mov eax, dword [ pt + POINT.y ];
    mov ecx, POINT.y_size

we now must use those macros as follows:

Code: [Select]

    mov eax, dword [ pt + offset( POINT.y ) ]
    mov ecx, sizeof( POINT.y )

The structure member names ( such as POINT.y ) would have to no longer be equated when creating structure definitions.  Instead, we would create an equate named POINT.y_offset.  Unfortunately, this is even more typing, but even worse is that the consequence of changing this would break any existing code terribly. 

[encryptor256]

Unfortunately, this is even more typing, but even worse is that the consequence of changing this would break any existing code terribly.

I agree, more typing would not be good, and also break existing codes would be catastrophe.

[Rob Neff]

Well, that's why compilers and assemblers have pragmas and command line switches, right? 

We can either extend the syntax which is only enabled if switched on (thus the programmer WANTS to use it), otherwise we keep the existing.

I'm wondering what other C-style compiler functionality we can add in since it's only worth doing if we gain a quite a few new features and capabilities.  We'll have to ponder on that ...

[encryptor256]

Well, that's why compilers and assemblers have pragmas and command line switches, right? 

We can either extend the syntax which is only enabled if switched on (thus the programmer WANTS to use it), otherwise we keep the existing.

Yes, you are right!
But command line switch implementation requires
a new official source code release of NASM itself, so, others could use it,
while things for NASMX can be managed just only in a single NASMX.INC file.

The thing i wanted to say, is, that - command line switches might take a little bit more time to implement.

I'm wondering what other C-style compiler functionality we can add

compiler functionality

I read, somewhere, in the past and could partly agree, someone said, it would be nice: C / C++ style comments.
Well it doesn't matter for me, i'm satisfied with current ';' - comments.

I'm wondering what other C-style compiler functionality we can add

It might end up having a NASM compiler which fully supports C.

[Rob Neff]

But command line switch implementation requires a new official source code release of NASM itself

No, I wouldn't even think of asking the nasm devs to put in a switch.  I'd use either the existing nasm -d command line switch, eg: -d NASMX_NEW_STUFF, since nasm would define that for us or make it a NASMX_PRAGMA that can be set only once.
For example, to use NASMX on FreeBSD, you have to define -d FREEBSD in order to assemble on that platform properly.

I read, somewhere, in the past and could partly agree, someone said, it would be nice: C / C++ style comments.

Unfortunately, the nasm token sequence of '//' is already used for signed division.  That immediately rules out that style of comment.

It might end up having a NASM compiler which fully supports C.

LOL! I'm pretty sure that the nasm devs do NOT want to go down that rabbit hole!

[Rob Neff]

More progress!  Added pragma pack!  Let's download the snapshot and check it out.

Here is sample code you can play with:

Code: [Select]


%define __NX_DEBUG_STRUC
%include 'nasmx.inc'

; Usage:
;     NASMX_PRAGMA PACK, PUSH, value
;           where value is either 1 or a small power of 2 ( eg: 2,4,8,16,32 )

NASMX_PRAGMA PACK, PUSH, 1

NASMX_STRUC PACKTEST
NASMX_RESERVE w,       uint16_t, 1
NASMX_RESERVE width,   uint32_t, 1
NASMX_RESERVE height,  uint32_t, 1
NASMX_RESERVE c,       uint8_t, 1
NASMX_RESERVE x,       uint16_t, 1
NASMX_ENDSTRUC

; restore natural alignment
NASMX_PRAGMA PACK, POP


Notice the line %define __NX_DEBUG_STRUC at the top.  That will be removed before release but currently allows you to see the output generated by the parser.  It's definitely enlightening to study.  Play around with different packing sizes to see the adjustments occur.  To see what would normally be generated without packing simply comment out the pragma lines.

Enjoy. 

ps: struc sizes are now back to being equated once again.

[Bryant Keller]

I read, somewhere, in the past and could partly agree, someone said, it would be nice: C / C++ style comments.

If you have a copy of the GNU C/C++ compiler, you _can_ have C/C++ comments in your NASM code. Here is an example:

Code: (demo.asm) [Select]

/**
 * Filename: demo.asm
 * Developer: S. Bryant Keller <bkeller@about.me>
 * Created: 26 Jan 2014 10:25 PM
 * Reason: To show encryptor256 how to use C/C++ comments in
 * NASM source.
 * Build: cpp demo.asm -o demo.temp
 * nasm -f elf demo.temp -o demo.o
 * gcc demo.o -o demo
 */

BITS 32

GLOBAL main
EXTERN printf

%define NewLine 10

SECTION .text

main: ; (int argc, char **argv) :int {{{
%define main.argc (ebp + 8)
%define main.argv (ebp + 12)
enter 0, 0
mov eax, [main.argv]
mov eax, [eax] ;; eax := argv[0]

push eax
push sMyNameIs
call printf
add esp, (4 * 2)

xor eax, eax ;; return (0)
leave
ret
;}}}

SECTION .data

sMyNameIs: db "This program is called: %s", NewLine, 0

;  vim: set fdm=marker ft=nasm ts=4 sw=4 tw=0 noet :

[encryptor256]

More progress!  Added pragma pack!

Okay, i will check this out.
I have never, ever used "packing" practically.

If you have a copy of the GNU C/C++ compiler, you _can_ have C/C++ comments in your NASM code. Here is an example

Thanks for example, this is something new, didn't knew that it's, even possible, here is win64 example:

Code: [Select]

/**
 * Filename: demo.asm
 * Developer: S. Bryant Keller <bkeller@about.me>
 * Created: 26 Jan 2014 10:25 PM
 * Reason: To show encryptor256 how to use C/C++ comments in
 * NASM source.
 * Build: cpp demo.asm -o demo.temp
 * nasm -f elf demo.temp -o demo.o
 * gcc demo.o -o demo
 */
/*
 * Original demo was modified to fit 64 bits.
 * Filename: (unchanged)
 * Translator: J.K. Encryptor256
 * Translated: 27 Jan 2014 12:09 PM
 * Reason: To say thanks and provide with Win64 example,
 * because didn't managed to run or compile 32 bit version correctly on Win64.
 * Build: cpp de... (unchanged)
 * nasm -f win64 demo.temp -o demo.o
 * gcc de... (unchanged)
*/

BITS 64

GLOBAL main
EXTERN printf

SECTION .text

align 16
main:
; Save all default arguments
mov [rsp+8*1],rcx
mov [rsp+8*2],rdx
mov [rsp+8*3],r8
mov [rsp+8*4],r9

; Define something
%define main.argc (rbp+8*2)
%define main.argv (rbp+8*3)

; Create stack
push rbp
mov rbp,rsp
lea rsp,[rsp-8*4]

; Print
; - assign RDX -> [[main.argv]] : Program name
; - assign R8 -> [[main.argv+8]] : First argument
mov rax,[main.argv]
mov r8,[rax+8]
mov rdx,[rax]
mov rcx,txt_format_arguments
call printf

; Clear stack n quit
lea rsp,[rsp+8*4]
pop rbp
ret

SECTION .data

txt_format_arguments: db "This program name is '%s' and is called by '%s'!",0


So, when i run, using command line: "demo.exe Computer", it gives\prints result:

Code: [Select]

This program name is 'demo.exe' and is called by 'Computer'!