64-bit DLL assembles and links BUT is not recognized by external program

[paml27]

I have written a 64-bit DLL in NASM.  The DLL assembles with the NASM assembler, and links correctly with GoLink, and a DLL file is produced.

However, when I call it from another program (not an assembler program), the entry point "Calculate" is not recognized as a valid DLL entry point. 

My question is:  is this DLL code correct for 64-bit NASM, and if it is correct, is there something that would cause it not to be recognized by the calling program? 

      global main
      global Calculate
      export Calculate

main:
    xor eax, eax
ret

Calculate:

[CODE OMITTED FOR BREVITY]

ret

Thanks for any help.

[Frank Kotler]

I have no experience with either Windows or 64-bit code, but it might be worth trying it as "_main" and/or "_Calculate".

Best,
Frank

[paml27]

Thanks, Frank.  Here's a follow-up question: 

The problem may be the way parameters are passed from the calling function to the DLL. 

In 32-bit MASM, we use a standard C-type function signature, e.g.

FunctionName PROC A:PTR DWord, B:PTR DWord
where parameters are passed in the function header. 

However, in both 32-bit and 64-bit NASM, a DLL entry point is specified this way: 

FunctionName:
where in 32-bit NASM, parameters are passed on the stack and in 64-bit NASM, parameters are passed (for Windows) in rcx, rdx, r8, and r9, not by using a C-style function header. 

If calling from an external function, could this lack of a C-style function signature be the problem, that the external function cannot recognize a signature? 

[Frank Kotler]

I very much doubt if your external program is looking for a "C-style function signature", but I'm really not sure. I should perhaps have been more definate that you do need the leading underscore.

Best,
Frank

[Frank Kotler]

I am incorrect. GoLink does not want you to use the leading underscore. See the Friendly Manual here:
http://www.godevtool.com/GolinkHelp/GoLink.htm
Pay attention to the "/mix" switch. This looks like a mess to me! Good Luck!

Best,
Frank

[paml27]

Thanks, Frank, for taking the time to answer this.  I'm calling the DLL from a third-party package, so I will ask the package vendor what registers they pass parameters in.  Calling across process boundaries presents extra difficulties, like the problem I'm having here. 

I've used two different linkers -- the GNU Linker and GoLink (GoLink is much better and the documentation is great).  Neither one allows the leading underscore.  32-bit NASM requires the leading underscore, but 64-bit does not allow it. 

The 64-bit processors have been out for a long time now, so I'm puzzled by how few assembler programmers have migrated to 64-bit.  Many say they just don't want to, so there's very little information on 64-bit programming.  For me, it's not an option because I've just taken on a project that requires 64-bit DLLs; it's proving to be quite daunting because of the lack of a large knowledge base like 32-bit assembler has. 

Thanks again. 

[stressful]

Creating a 64-bit DLL in NASM is simple. I've compiled a short demo code for you

Here's the DLL

Code: [Select]

;----------------------
;The DLL
;nasm -f win64 libdll.asm
;golink /dll libdll.obj msvcrt.dll
;----------------------
global printz
export printz

section .text
printz:
        sub rsp,0x20
        call printf
        add rsp,0x20
        ret

extern printf
Now you can access it from C like so;

Code: [Select]

;-------------------------------
;Try to access from a C program
;gcc -m64 testme.c libdll.dll -s -o testme.exe
;-------------------------------
#include <stdio.h>

extern void printz(char *);

int main()
{
        char msg[] = "Hello World";

        printz(msg);

        return 0;
}
Or from another NASM program like this:

Code: [Select]

;--------------------------
;nasm -f win64 prog.asm
;golink /console prog.obj libdll.dll kernel32.dll
;--------------------------
global Start

extern printz      ;from libdll.dll
extern ExitProcess ;from kernel32.dll

section .data
msg db 'Hello World!!!',0ah,0

section .text
Start:
        mov rcx,msg
        call printz

        mov rcx,0
        call ExitProcess
All works as expected using golink. I hope this helps.

[paml27]

Thanks very much for this example.  It helps confirm that my DLL is constructed correctly, so the problem is the external call.  I'll try your example of calling it from C or another NASM program. 

I really appreciate you taking the time to post this working example.