Author Topic: Error disassembling 16bit ModR/M in SIMD and other bugs  (Read 15093 times)

Offline vitsoft

  • Jr. Member
  • *
  • Posts: 17
  • Country: cz
    • About me
Error disassembling 16bit ModR/M in SIMD and other bugs
« on: September 24, 2018, 07:32:19 PM »
Hi folks, during developement of my assembler I was using NASM/NDISASM for mutual cross check and I discovered a couple of minor bugs.
Nasm-bugs mailist seems to to be frozen since March 2016 and Bugzilla doesn't allow formating, so I dare to post it here.
Code: [Select]
1                        ; Bugs found in NASM version 2.14rc15 compiled on Jul  6 2018
 2
 3                        ; NASM bug #1: Version info is not updated.
 4 00000000 02             db __NASM_MAJOR__
 5 00000001 0D             db __NASM_MINOR__   ; I would expect 0E here in NASM version 2.14.
 6 00000002 63             db __NASM_SUBMINOR__
 7 00000003 69             db __NASM_PATCHLEVEL__
 8
 9                        ; NASM bug #2:
10                        ; NASM assembles POP CS as 0x0F in all modes. This single-byte opcode existed only on the 8086/8088,
11                        ; see https://www.pcjs.org/pubs/pc/reference/intel/8086/
12                        ; I believe NASM should warn that MOV CS,AX or POP CS is an invalid instruction, at least in 64bit mode.
13                        bits 64
14 00000004 8EC8           MOV CS,AX ; Assembled incorrectly as 8EC8 without error.
15 00000006 0F             POP CS    ; Assembled incorrectly as 0F   without error.
16
17                        ; NASM bug #3:
18                        ; NASM incorrectly encodes INVPCID in 64bit mode with superabundant REX.W prefix.
19                        ; I believe that all VMX invalidating instructions are promoted to 64 bits.
20 00000007 660F388006     INVEPT  RAX,[RSI] ; Assembled correctly,   660F388006 expected.
21 0000000C 660F388106     INVVPID RAX,[RSI] ; Assembled correctly,   660F388106 expected.
22 00000011 66480F388206   INVPCID RAX,[RSI] ; Assembled incorrectly, 660F388206 expected.
23
24                        ; NASM bug #4
25                        ; SWAPGS should be marked X64,PRIV in https://github.com/ncoden/NASM/blob/master/insns.dat

; Bugs found in NDISASM version 2.14rc15 compiled on Jul  6 2018
; NDISASM bug #1:
; NDISASM -b16 incorrectly disassembles 16bit ModR/M in some SIMD instructions:
; F30F7E0F in 16bit mode incorrectly disassembles as movq xmm1,xmm12    instead of movq xmm1,[bx]
; F30F2D0F in 16bit mode incorrectly disassembles as cvtss2si ecx,xmm12 instead of cvtss2si ecx,[bx]

; NDISASM bug #2:
; NASM correctly assembles obsolete CMPXCHG486 rCX,rDX to 0FA7D1
; but NDISASM  in 16bit or 32bit mode cannot disassemble 0FA7D1 back.

; NDISASM bug #3:
; NASM correctly assembles MOVSS XMM1,XMM7 to F30F10CF
; and NDISASM correctly disassembles F30F10CF to movss xmm1,xmm7
; but it does not recognize its alternative encoding
; and incorrectly disassembles F30F11F9 as rep xmovups xmm1,xmm7  instead of movss xmm1,xmm7

; NDISASM bug #4:
; NDISASM -b64 incorrectly disassembles 8bit registers R8B..R15B from instruction VPBROADCASTB xmm1,reg
 VPBROADCASTQ XMM1,RDX
 VPBROADCASTD XMM1,EDX
 VPBROADCASTW XMM1,DX
 VPBROADCASTB XMM1,DL
 VPBROADCASTQ XMM1,R10
 VPBROADCASTD XMM1,R10D
 VPBROADCASTW XMM1,R10W
 VPBROADCASTB XMM1,R10B ; Incorrectly disassembled 62D27D087ACA as vpbroadcastb xmm1,r10w
                                                   ; instead of vpbroadcastb xmm1,r10b

; NDISASM bug #5: (seems to be already reported as 3392350)
; NDISASM incorrectly interprets {sae}, i.e. bit EVEX.b
; For instance VGETEXPPS ZMM1,ZMM2,{sae} is correctly assembled to 62F27D1842CA
; but this is disassembled back as vgetexpps xmm1,xmm2

; NDISASM bug #6:
; NDISASM ignores bit EVEX.P1.2 (difference between EVEX and MVEX prefix, see Phi reference.
; I respect the decission to not support Intel Xeon Phi processor,
; but NDISASM should detect that EVEX.P1 bit 2 is 0 (which signalizes MVEX encoding)
; and give up rather than misinterpret the prefix as if it were EVEX.
; For instance 62F178085A4D02 is incorrectly disassembled as vcvtps2pd xmm1,qword [rbp+0x10]
                                            but it should be vcvtps2pd zmm1,yword [rbp+0x40]

; Documentation error at

Chapter 11: Writing 64-bit Code (Unix, Win64)


instead of
All known 64-bit platforms except some embedded platforms
require that the stack is 16-byte aligned at the entry to a function.
In order to enforce that, the stack pointer (RSP) needs to be aligned
on an odd multiple of 8 bytes before the CALL instruction.

should read
All known 64-bit platforms except some embedded platforms
require that the stack is 16-byte aligned at the CALL OF a function.
In order to enforce that, the stack pointer (RSP) needs to be aligned
on an even multiple of 8 bytes before the CALL instruction.

I other words, if the first instruction of fastcalled function were
TEST SPL,1000b
it should return NONZERO (ZF=0).

See also https://blogs.msdn.microsoft.com/oldnewthing/20040114-00/?p=41053/
https://agner.org/optimize/calling_conventions.pdf