Author Topic: creating a flat binary bigger than 64kB  (Read 16213 times)

Offline Folkert van Heusden

  • Jr. Member
  • *
  • Posts: 2
creating a flat binary bigger than 64kB
« on: July 03, 2023, 06:57:34 AM »
Hello,

For fun I'm developing an IBM PC emulator (before XT even). At the point it currently is I'm working on test-code to verify that the processor emulation is correct.
Unfortunately I'm stuck on far/inter-segment jumps.
According to http://computer-programming-forum.com/46-asm/bf904883c2cb61c7.htm it is/was possible to create binaries > 64kB yet in practice this fails.
e.g.:

   org 0x800

   xor ax,ax
   mov si,ax

   mov ss,ax
   mov ax,0x800
   mov sp,ax

; JMP NEAR
test_001:
    mov si,0x1
    jmp test_001a_ok
    hlt
test_001a_ok:

; JMP FAR
test_003:
    mov si,0x3
    jmp far test_003_ok
    hlt
    hlt
    hlt
    resb 70000
    hlt
    hlt
    hlt
test_003_ok:
    nop
   mov ax,0xa5ee
   mov si,ax
   hlt


This results in:

$ nasm -f bin jmp_call_ret_0.asm
jmp_call_ret_0.asm:20: error: binary output format does not support segment base references
jmp_call_ret_0.asm:24: warning: uninitialized space declared in .text section: zeroing [-w+zeroing]


I'm using nasm 2.16.01 as found in ubuntu

Any hints?

regards

p.s. yes this has been discussed on stackoverflow as well only the solutions there were work-arounds and I'm hoping to find a solution here to make it work without any trickery

Offline Frank Kotler

  • NASM Developer
  • Hero Member
  • *****
  • Posts: 2667
  • Country: us
Re: creating a flat binary bigger than 64kB
« Reply #1 on: July 03, 2023, 08:39:34 AM »

Offline Folkert van Heusden

  • Jr. Member
  • *
  • Posts: 2

Offline fredericopissarra

  • Full Member
  • **
  • Posts: 373
  • Country: br
Re: creating a flat binary bigger than 64kB
« Reply #3 on: July 03, 2023, 01:23:29 PM »
In 16 bits mode (real mode) you cannot have more then 64 KiB in a single segment. Since you are not creating an EXE file, you cannot:

1 - Jump to another "segment" (there's only one);
2 - Declare uninitialized data (resb);
3 - Do a direct far jump or call...

This will compile, but will create the wrong code:
Code: [Select]
  org 0x800

  xor ax,ax
  mov si,ax

  mov ss,ax
  mov ax,0x800
  mov sp,ax

  mov [jmp_addr+2],cs

; JMP NEAR
test_001:
  mov si,0x1
  jmp test_001a_ok
  hlt

jmp_addr:
  dw  test_003_ok
  dw  0

test_001a_ok:

; JMP FAR
test_003:
  mov si,0x3
  jmp far [jmp_addr]
  hlt
  hlt
  hlt
  times 70000 nop
  hlt
  hlt
  hlt
test_003_ok:
  nop
  mov ax,0xa5ee
  mov si,ax
  hlt
Because that times 70000 nop will get you an image bigger than 64 KiB, so test_003_ok will be a reference inside your code segment (not outside)... test_003_ok will be offset 0x1_1196. but the offset reference in jmp_addr will be 0x1196, thus, wrong!

You can use 386 instructions in real mode and "normalize" the "physical address" like this:
Code: [Select]
  ...
  mov eax,test_003_ok  ; this will be a 32 bits offset.
  mov bx,cs
  movzx ebx,bx
  shl ebx,4
  add eax,ebx  ; EAX has the "physical address" of test_003_ok.
  mov ebx,eax
  and ax,0x0f
  shr ebx,4      ; BX:AX is now the logical "normzlized" address of test_003_ok

  mov [jmp_addr],ax
  mov [jmp_addr+2],bx
  jmp far [jmp_addr]

jmp_addr:
  dw 0, 0
...

Notice you'll have to calculate seg:offset to access anything beyond the first segment. (and I don't know if this will work with a .COM file no MS-DOS, for example).
« Last Edit: July 03, 2023, 01:36:29 PM by fredericopissarra »

Offline Deskman243

  • Jr. Member
  • *
  • Posts: 49
Re: creating a flat binary bigger than 64kB
« Reply #4 on: July 03, 2023, 03:11:53 PM »
This question is pretty intriguing to me as I had to make a good amount of posts here to try and answer this. Currently what I understand is that x86-16 is based on 8086 processors. Now the design itself is built on a certain amount of data lines as previously mentioned
Quote
In 16 bits mode (real mode) you cannot have more then 64 KiB in a single segment.
This is because the logical address is built from the A20 data lines. This clarifies the A1-16 controls for the standard segmentation for these programs. Now the meaning of the rest of the lines infers the area passed 64kb (0xFFFF + 1 =64kb and 0xFFFFF + 1 = 1 MB).  What I'm trying to get is whether these segments > 0x0FFFF are reserved in a way or more of a reserve for say one type of thing.

Quote
Notice you'll have to calculate seg:offset to access anything beyond the first segment.

My previous post was on how data is accessed by these segments because I could only ever get data reference passed 64kb. Other than that I was researching disk reading to perhaps get an answer on the correlated links below.

Adjusting segments for linux linker commands and build files
https://forum.nasm.us/index.php?topic=3881.0

Disk Routines under NASM
https://forum.nasm.us/index.php?topic=3884.0
« Last Edit: July 03, 2023, 03:37:31 PM by Deskman243 »

Offline fredericopissarra

  • Full Member
  • **
  • Posts: 373
  • Country: br
Re: creating a flat binary bigger than 64kB
« Reply #5 on: July 03, 2023, 03:34:32 PM »
Quote
In 16 bits mode (real mode) you cannot have more then 64 KiB in a single segment.
This is because the physical address is built from the A20 data lines...
No, it is because the logical address, SEGMENT:OFFSET. Since the binary format don't have "sections" you must change the segment part yourself to match a physical address...

The gate A20 was introduced to AT class PCs with 286 because the PHYSICAL address bus is 24 bits long and the final 64 KiB block don't exist for previous processors (8086) - because the carry is ignored (0xFFFF0 + 0xFFFF = 0x0FFEF + carry). To keep compability with the old 8086 (with 20 bits address bus) the A20 line is forced to zero by hardware. With gate A20 enabled  the physical addresses between 0x100000 and 0x10FFEF are avaliable.

Offline Deskman243

  • Jr. Member
  • *
  • Posts: 49
Re: creating a flat binary bigger than 64kB
« Reply #6 on: July 03, 2023, 03:49:56 PM »
Quote
No, it is because the logical address, SEGMENT:OFFSET.

Sorry didn't know that data lines were still only a conjecture.

Quote
- because the carry is ignored (0xFFFF0 + 0xFFFF = 0x0FFEF + carry)...

Like you had said earlier, these are based on the fact that there is an A20 gate. If you want to look at actual segment code you'd have to get a check by your NASM's assembler configuration or whatever linker program you use.

Offline fredericopissarra

  • Full Member
  • **
  • Posts: 373
  • Country: br
Re: creating a flat binary bigger than 64kB
« Reply #7 on: July 03, 2023, 05:48:04 PM »
Quote
- because the carry is ignored (0xFFFF0 + 0xFFFF = 0x0FFEF + carry)...

Like you had said earlier, these are based on the fact that there is an A20 gate. If you want to look at actual segment code you'd have to get a check by your NASM's assembler configuration or whatever linker program you use.
Not what I said at all... in real mode the logical address is always SEGMENT:OFFSET, calculated as PhysicalAddr = (SEGMENT << 4) + OFFSET, with 20 bits total AND offsets are always 16 bits. Let's say DS=0 and we have:
Code: [Select]
  mov bx,0xffff
  mov al,[bx]    ; get a byte from ds:0xffff
  inc bx
  mov al,[bx]   ; get a byte from ds:0, not from ds:0x10000!
Clearly this overflow has nothing to do with gate A20.

Offline Deskman243

  • Jr. Member
  • *
  • Posts: 49
Re: creating a flat binary bigger than 64kB
« Reply #8 on: July 04, 2023, 01:18:48 AM »
Would you care tell us what this response means?
Quote
Code: [Select]
  mov bx,0xffff
  mov al,[bx]    ; get a byte from ds:0xffff
  inc bx
  mov al,[bx]   ; get a byte from ds:0, not from ds:0x10000!
You should know that the only distinction between the A20 line and the Gate A20 is the AND gate they put on the line. That's what the gate for A20 actually means.

Offline Michael Petch

  • Jr. Member
  • *
  • Posts: 4
Re: creating a flat binary bigger than 64kB
« Reply #9 on: July 04, 2023, 06:22:51 AM »
He's trying to say that when you are using 16-bit addressing modes (whether it is on an 8086/8088) or on a 286+ the effective address (aka OFFSET) can't exceed 0xffff. Any calculations of the OFFSET portion are truncated to the lower 16-bits. A better example is this
Code: [Select]
    mov ax, 0xF000
    mov ds, ax
    mov bx, 0xFFFF
    mov si, bx
    mov al, [bx+si+0x7fff]
A logical question arises. What memory address is addressed with [bx+si+0x7fff]. Since the memory address isn't using register BP, the DS segment is used by default so it is the same as DS:[bx+si+0x7fff].

Some may blindly say that BX=0xffff, SI=0xffff, DISP=+0x7fff so the offset portion is 0xffff+0xffff+0x7fff=0x27FFD. Since DS is 0xF000, the physical address calculation would be (0xf000<<4)+0x27FFD =  0x117FFD. Now if this was correct then the A20 line would factor in.

What the fellow was pointing out is that the above calculation is wrong. The OFFSET portion is always the low order 16 bits of the calculated offset address. So 0xffff+0xffff+0x7fff is actually 0x7FFD and not 0x27FFD. Now the proper calculation is (0xf000<<4)+0x7FFD = 0xF7FFD. In this case the A20 line doesn't factor in.

Had the fellow given you this:
Code: [Select]
    mov ax, 0xF000
    mov ds, ax
    mov bx, 0xffff
    mov al, [bx+1]
The memory address read from would be (0xF000<<4)+0x0000=0xF0000 and not (0xF000<<4)+0x10000=0x100000.

This holds true when using 16-bit addressing modes on an 8086/8088/80286/80386+ whether in real mode or protected mode.

Over the decades there have been some simulators that have done this calculation wrong, potentially leading to the wrong memory addresses being accessed.
« Last Edit: July 06, 2023, 12:36:00 AM by Michael Petch »