Direct output to PE32, PE64, DLL32 and DLL64 with Macro Sets

[nalsakas]

Hmm, thank you for your contribution. I will update 64-bit sample applications in the project directory according to your corrections.

Can you write a brief tutorial on win64 calling convention if you don't mind? About registers, shadow space, must preserved registers etc.

Did you experiment with FLAT mode?

[encryptor256]

X:

+ There is anoter problem: DialogBoxParam requires five arguments.

So, last one of them should be initialized and stored in stack space.

Changed START procedure:

Code: [Select]

       ; ...
       ; shadow stack area
        sub rsp, 48

       ; ...
       ; ...

       ; DialogBox
        xor rax,rax
        mov [rsp+32],rax ; dwInitParam
mov r9, VA(DlgProc) ; lpDialogFunc
xor r8, r8 ; hWndParent
mov rdx, ID_DIALOG ; lpTemplateName
mov rcx, [VA(hIns)] ; hInstance
call [VA(DialogBoxParamA)]

       ; ...

I would recommend to change source code to this, 64_dialogbox_new_new.asm:

Code: [Select]

%include '../pe.inc'
%include '../windows.inc'

; Resource IDs
%define ID_DIALOG 10
%define ID_BUTTON 11
%define ID_EDIT 20

PE64

; Data declerations
BYTE title, "NASM PE MACROS",0
QWORD hIns
BYTE buffer[256]

START

push rbp
mov rbp, rsp

; shadow stack area
sub rsp, 48   

; Get module handle
mov rcx, 0
call [VA(GetModuleHandleA)]
mov [VA(hIns)], rax

; DialogBox
        xor rax,rax
        mov [rsp+32],rax ; dwInitParam
mov r9, VA(DlgProc) ; lpDialogFunc
xor r8, r8 ; hWndParent
mov rdx, ID_DIALOG ; lpTemplateName
mov rcx, [VA(hIns)] ; hInstance
call [VA(DialogBoxParamA)]
   
; // Begin: ###################
; // {
mov r9, 0
mov r8, VA(title)
mov rdx, VA(title)
mov rcx, 0
call [VA(MessageBoxA)]
; // };

.return:
mov rsp, rbp
pop rbp
ret

; Dialog Procedure
; [r9] = lParam
; [r8] = wParam
; [rdx] = uMsg
; [rcx] = hDlg
DlgProc:
; save parameters on shadow
mov qword [rsp + 8], rcx
mov qword [rsp + 16], rdx
mov qword [rsp + 24], r8
mov qword [rsp + 32], r9

push rbp
mov rbp, rsp

; shadow stack area
sub rsp, 32

; switch msg
cmp qword [rbp + 24], WM_CLOSE
je .close
cmp qword [rbp + 24], WM_COMMAND
je .command
cmp qword [rbp + 24], WM_INITDIALOG
je .initdialog

.default:
xor rax, rax

.return:
mov rsp, rbp
pop rbp
ret

.initdialog:
mov eax, 1
jmp .return

.close:
mov rdx, 1
mov rcx, [rbp + 16]
call [VA(EndDialog)]

mov rax, 1
jmp .return

.command:
mov rax, [rbp + 32]
cmp ax, ID_BUTTON
je .command_button
jmp .default

.command_button:
; GetDlgItemText
mov r9, 255
mov r8, VA(buffer)
mov rdx, ID_EDIT
mov rcx, [rbp + 16]
call [VA(GetDlgItemTextA)]

; MessageBox
mov r9, 0
mov r8, VA(title)
mov rdx, VA(buffer)
mov rcx, 0
call [VA(MessageBoxA)]

mov rax, 1
jmp .return


; Data Directories
IMPORT
LIB user32.dll
FUNC MessageBoxA
FUNC DialogBoxParamA
FUNC EndDialog
FUNC SendMessageA
FUNC GetDlgItemTextA
ENDLIB
LIB kernel32.dll
FUNC GetModuleHandleA
FUNC LoadLibraryA
FUNC FreeLibrary
ENDLIB
ENDIMPORT

RESOURCE
TYPE RT_DIALOG
ID ID_DIALOG
LANG
LEAF RVA(dialog), SIZEOF(dialog)
ENDLANG
ENDID
ENDTYPE
ENDRESOURCE

DIALOG dialog, 0, 0, 210, 142
STYLE DS_CENTER
CAPTION "NASM PE MACROS"

DEFPUSHBUTTON "OK", ID_BUTTON, 2, 122, 208, 18
EDITTEXT ID_EDIT, 2, 2, 208, 120, ES_LEFT | ES_MULTILINE | ES_AUTOVSCROLL
ENDDIALOG

END

; Assemble
; nasm -f bin -o 64_dialogbox_new_new.exe 64_dialogbox_new_new.asm


Y:

Can you write a brief tutorial on win64 calling convention if you don't mind? About registers, shadow space, must preserved registers etc.

It can be easily found on google, like x64 calling convention or x64 stack alignment.

Did you experiment with FLAT mode?

No.

Z:

Suggestion:

If you want to clear-zero out-initialize a register use, like examples here:

Code: [Select]

xor rax,rax
xor r8,r8


[nalsakas]

What is the reason behind allocating 48 bytes for shadow stack space? DialogBoxParamA needs 5 parameters x 8 bytes = 40 bytes in total for shadow stack space.   

[encryptor256]

DialogBoxParamA needs 5 parameters x 8 bytes = 40 bytes in total for shadow stack space.

Yes.

What is the reason behind allocating 48 bytes for shadow stack space?

+8 is for alignment.

[nalsakas]

So you are saying subtracting extra 8 bytes for alignment shouldn't hurt.

sub rsp, 48 --> OK, works


But why subtracting 8 bytes more crashes application?

sub rsp, 56 ---> NOT OK, crashes

Moreover putting extra alignment in Dlgproc also craches application.


DlgProc:
sub rsp, 32 ---> OK

sub rsp, 40 --> NOT OK

[encryptor256]

But why subtracting 8 bytes more crashes application?

sub rsp, 56 ---> NOT OK, crashes

Stack is not aligned and crashes.

Hey check this out: Stack alignment in x64 assembly

There is one: "vandale" individual who complains and one: "lurker" who explains.

EDIT:

Moreover putting extra alignment in Dlgproc also craches application.

DlgProc:
sub rsp, 32 ---> OK

sub rsp, 40 --> NOT OK

Yes! According to your prolog routine and later function calls, the number 32 is appropriate and stack will be aligned correctly, BUT 32+8 = 40 would ruin proper alignment.

[Bryant Keller]

There are way too many things going on in this thread for me to go over every post, so I thought I would focus on the primary topic.

Request 1:

First is I need to test if a label is defined before or after the current test  location. I need a preprocessor directive,  something like %iflabel  /  %endif.

That one already exists. %ifid will tell you if an identifier is already defined.

Request 2:

I need to test if a single line macro defined after current line? %ifdef doesn't work in these situations.

This also, already exists. %ifmacro will tell you if a macro is currently defined.

Request 3:
I could get the last value of a single line macro (a %define statement). In order for me to fill in section table of pe file I need to know in advance how many sections are defined in pe file. I need to get last declared value of section number counter, each section increments the value by one.

To my knowledge, this one hasn't be implemented. But if you need to know how many sections are going to be used, why not ask the user. Add an operand to your PE macro that asks for the section count.

PE [type], [bits], [sections]
Usage:
PE EXE, 64, 5
PE DLL, 32, 2

HtH,
Bryant

[nalsakas]

Hi Bryant,

Actually Request 2 isn't supported by nasm.

Below code block produces "something is defined". Because macro is defined before test.

Code: [Select]

%define something 1

%ifdef something
%warning "something is defined"
%else
%warning "something isn't defined"
%endif

Whereas following code block produces "something isn't defined". Because macro isn't defined before test.

Code: [Select]

%ifdef something
%warning "something is defined"
%else
%warning "something isn't defined"
%endif

%define something 1

Actually right now I don't need anything new from nasm. I made it work without future requests. PE macros works with current nasm and successfully produces PE32, PE64, DLL32 and DLL64 files using only nasm.

When I start to build PE macros I thought It would be fine to support multiple sections. But later I changed my mind. One section is enough for an executable. That way they become smaller.


regards