Nasm64: Return multiple 64 bit registers to C++ in Windows10

[prdas31]

Hi,

What I want to know are the following two things:

1) Under fastcall, how can I return multiple values to C++ using registers? (as under fastcall C++ can officially take only RAX I think. And I do not wish to use a struct or class).

In the following trial code I used std::pair just to make my point clearer.

Code: [Select]

#include <iostream>
using namespace std;

extern "C" std::pair<string, string> testCall(char* a, char* b, char**);

int main() {
    char* somebuf =(char*) malloc(256);

    std::pair<string, string>res = testCall( "Hello", "World", &somebuf );

    cout << res.first <<rec.second<<endl;
    cout<<somebuf <<endl;
    return 0;
}

2) If I pass a third argument in my function to get some output value from the assembler, how to load that value/pointer in the assembly routine? (like the somebuf  vars in my example ).

A very simple asm code I tried:

Code: [Select]

segment .text

    global testCall

testCall:
    pushfq
   
    mov rax, rdx

    mov rdx, rcx ; How to get the value of RDX in C++?
   
    mov r8, rdx ; How to send back the R8 register/pointer to &somebuf?

   
    popfq
    ret

Appreciate your help.
Thanks.

[prdas31]

I have figured out both of these issues.

Thanks, and request to ignore it.

[PatrickDes]

Can you share the solutions with us, Prdas31? I'm really curious and it might help someone else as well.

[prdas31]

Can you share the solutions with us, Prdas31? I'm really curious and it might help someone else as well.

I've noticed this request long after it's posted...sorry for that. In case it helps anybody here's an example of doing it. Kindly note that there could be other ways to passing parameters as well from CPP to ASM and vice versa:

The CPP:

Code: [Select]

//============================================================================
// Name        : CallConvTest.cpp

//============================================================================

#include <iostream>
using namespace std;

#ifdef __cplusplus
extern "C" {
#endif

char* testCall(const char*, const char*, char**);

char* testCall2(const char*, const char*, int, int, const char*);

int testCall3(const char*, const char*, int, int, const char*, int);

int testCall4(const char*, const char*, int, int, char**);

int testCaller(const char*, const char*, int, int, const char*, int, int);

int writeToConsoleFromAsm(const char*);

#ifdef __cplusplus
}
#endif

extern "C" char* someBuf;
extern "C" char* anotherBuf;

int main() {
const char* arg1 = "Hello";
const char* arg2 = "World";
const char* arg3 = "The 5th parameter";

char* res = testCall(arg1, arg2, &someBuf);

cout << res << endl;
cout << someBuf << endl;

res = testCall2(arg1, arg2, 5, 11, arg3);
cout << res << endl;

unsigned int placeholder = 0;
placeholder = testCall3(arg1, arg2, 5, 11, arg3, 108);
cout << "The 6th parameter = " << placeholder << endl;

placeholder = testCall4(arg1, arg2, 5, 11, &anotherBuf);
cout << "Length of the Buffer string returned: "<<placeholder << endl;
cout << anotherBuf << endl;

placeholder= writeToConsoleFromAsm(arg1);
cout << "ASM print Returns Success: " << placeholder << endl;

placeholder = testCaller(arg1, arg2, 5, 11, arg3, 108, 2);
cout << "Addition of 5th and 6th parameter = " << placeholder << endl;
cout<<"5th returns: "<< arg3<<endl;

system("pause");
return 0;
}

The ASM that gets parameters and returns parameters/values to CPP:

Code: [Select]



segment .data

someBuf dq 0
anotherBuf dq 0


message db "Changed 5th parameter", 0AH, 00H;
fxp db'From ASM by calling printf: %s', 0AH, 00H; ASCII format string

testbuf1 db "Hello World from 64Bit ASM!! By ML64.",0
testbuf2 db "Hello World AGAIN from 64Bit ASM!! By ML64.",0
len equ $-testbuf2


segment .text

;includelib msvcrt.lib
extern printf


extern calledproc

global testCall, someBuf, anotherBuf, testCall2, testCall3, testCall4, testCaller, writeToConsoleFromAsm

testCall:

mov rax, rdx
lea r8, [testbuf1]
mov [someBuf], r8

ret



testCall2:

xor rax,rax
mov rax, [RSP+28H] ;The 5th parameter is at RSP+28h location on the stack
;mov rax, [RSP+30H] ;The 6th parameter is at RSP+28h location on the stack [always add 8 from RSP+28H]

ret

testCall3:

xor rax,rax
;mov rax, [RSP+28H] ;The 5th parameter is at RSP+28h location on the stack
mov rax, [RSP+30H] ;The 6th parameter is at RSP+28h location on the stack [always add 8 from RSP+28H]

ret

testCall4:

xor rax,rax
;mov rax, [RSP+28H] ;The 5th parameter is at RSP+28h location on the stack
;mov rax, [RSP+30H] ;The 6th parameter is at RSP+28h location on the stack [always add 8 from RSP+28H]
mov rax, rdx
lea r8, [testbuf2]
mov [anotherBuf], r8

mov rax,len
ret

;This one calls another ASM routine in another file
testCaller:
xor eax, eax
xor r12d, r12d
xor r11d, r11d
xor r10d, r10d

mov r12, [rsp+38h] ;7th parameter
mov r11, [rsp+30h] ;6th parameter
mov r10, [rsp+28h] ;5th parameter



;In general, allocate max(32, n x 8) bytes where n is the maximum number of parameters (minimum 32 bytes of stack space for shadow space allocated)
sub rsp, 48 ; sub rsp, 32 ; plus 16 more for parameter 6. Parameter 5 goes to [rsp+32].

push r12
push r11
push r10

;calls another ASM routine in another file
call calledproc

pop r10
pop r11
pop r12

lea r13, [message] ; This line does not do anything now; may be to be used later with some shared memory like anotherbuf

add rax, r9

add rsp, 48
ret

writeToConsoleFromAsm:
sub rsp, 32
mov rdx, rcx; 2nd argument for PRINTF
lea rcx, [fxp] ; format string for PRINTF
call printf
mov rax, 0
add rsp, 32
ret


And another ASM that gets called from the above ASM and returns values:

Code: [Select]

section .data

section .text

global calledproc

calledproc:

mov rax, [RSP+18H] ;The 7th parameter
mov r10, [RSP+10H] ;The 6th parameter
mov r11, [RSP+8] ;The 5th parameter

add rax, r10 ; add 6th and 7th parameter
ret


This can be tested in Eclipse or Netbeans IDEs using 64 bit CPP compilers and NASM 64.

Best.

[fredericopissarra]

I have figured out both of these issues.

Thanks, and request to ignore it.

Nice, here's just the appropriate documentation from M$: x64 calling convention.