NASM - The Netwide Assembler

NASM Forum => Programming with NASM => Topic started by: coldflame on April 19, 2017, 02:49:23 PM

Title: Functions with XMM works fine while functions with YMM fails x64
Post by: coldflame on April 19, 2017, 02:49:23 PM

Hello,

I have a codes for basic matrix/vector multiplication (overall 12 codes). They are made to use SSE, AVX and FMA. With AVX/FMA, I use YMM registers.

The whole code is linked into .dll and I use Matlab to call these functions.

While functions that uses XMM registers work fine, the ones that uses YMM registers crash SOMETIMES (not always, but mostly Matlab crashes and reports a bug in DLL).

This is kinda strange because I'm not sure what might be the problem here.

So I made a function with AVX instruction BUT with XMM registers and the funciton works well, no crashes.

It seems to be some kind of a problem with using YMM registers.

Have you ever seen something like that? Any help appreciated.

PS: ALL codes are OK - they work as expected (tested in Matlab and DBG).

Title: Re: Functions with XMM works fine while functions with YMM fails x64
Post by: coldflame on April 20, 2017, 12:13:39 PM

I guess it has something to do with alignment. Probably I need to align on 32-byte boundary instead of 16-byte one.

Title: Re: Functions with XMM works fine while functions with YMM fails x64
Post by: coldflame on April 22, 2017, 01:58:27 PM

For anyone curious about what was wrong:

- something was wrong with alignment - I've changed all aligned MOV unstructions to unaligned ones and it works fine now.