Ok I finally got this working.

[ben321]

I wrote a program that from real mode sets up and loads a GDT, and then calls a protection mode switcher function that is responsible for doing everything to put it into 32bit protected mode, including the far call. Then from in 32bit protected mode it calls a function to switch video modes. Of course this requires the instruction INT 10h, but you can't do that in protected mode without an IDT and setting up your own interrupt functions to replace those of the BIOS. Suffice to say I didn't do such an extensive thing as that. Instead I wrote an ordinary function that in addition to calling INT 10h, actually first calls a function I made to temporarily switch back to 16bit protected mode. After the INT call, my video mode switching function calls a protected mode resuming function, and after that it returns to the main protected mode function. After doing that, it then returns from the protected mode function back to the 16bit real mode function that was initially responsible for far calling the main 32bit protected mode function. From there it returns to the program's main code and finally returns to DOS, leaving the graphics mode set to mode 13h (320x200 256 color mode).

Yes that is a LOT of work to accomplish a simple task, but it demonstrates a way to call 16bit real mode interrupts, without ever having to load an IDT and have interrupt gates. It also demonstrates how to get into and then out of 32bit protected mode in such a manner that you can get back to DOS without crashing the system by messing up the stack (or any other number of things that could go wrong). And that doesn't even touch on switching to ring3 user mode (which I haven't quite figured out yet how to do). All of the protected mode stuff I did here was in ring0 kernel mode.

Here's the code.

Code: [Select]

ORG 0x100
USE16



Main:
push bp
mov bp,sp
xor eax,eax
mov ax,cs
shl eax,4
mov edx,eax
shr edx,16
mov [GDT_Entry_1.Base0],ax
mov [GDT_Entry_1.Base1],dl
mov [GDT_Entry_2.Base0],ax
mov [GDT_Entry_2.Base1],dl
add eax,StartOfGDT
mov [PointerToGDT+2],eax
lgdt [PointerToGDT]
push ProtMode32EntryPoint
call SwitchToProtMode
leave
ret

SwitchToProtMode:
push bp
mov bp,sp
mov [OldCS],cs
mov [OldDS],ds
mov [OldES],es
mov [OldFS],fs
mov [OldGS],gs
mov [OldSS],ss
mov dx,[bp+4]
push 8
push dx
cli
mov eax,cr0
or al,1
mov cr0,eax
push bx
push si
push di
call far [bp-4]
mov ds,[OldDS]
mov es,[OldES]
mov fs,[OldFS]
mov gs,[OldGS]
mov ss,[OldSS]
nop
pop di
pop si
pop bx
sti
leave
ret 2

PointerToGDT:
dw EndOfGDT-StartOfGDT-1
dd 0


StartOfGDT:
dq 0
GDT_Entry_1: ;32bit code
.Limit0 dw 0xFFFF
.Base0 dw 0x0000
.Base1 db 0x00
.Access db 0x9A
.FlagsAndLimit1 db 0xCF
.Base2 db 0x00
GDT_Entry_2: ;32bit data
.Limit0 dw 0xFFFF
.Base0 dw 0x0000
.Base1 db 0x00
.Access db 0x92
.FlagsAndLimit1 db 0xCF
.Base2 db 0x00
EndOfGDT:


ProtMode32EntryPoint:
USE32
mov ax,8*2
mov ds,ax
mov es,ax
mov fs,ax
mov gs,ax
mov ss,ax
nop
push ebp
mov ebp,esp
call SetVideoMode
leave
mov eax,cr0
and al,0xFE
mov cr0,eax
retfw

SetVideoMode:
push ebp
mov ebp,esp
call TempSwitchToRealMode
USE16
mov ax,0x0013
int 0x10
call ReturnToProtMode
USE32
leave
ret

TempSwitchToRealMode:
mov eax,cr0
and al,0xFE
mov cr0,eax
mov ax,[OldCS]
push ax
push TempSwitchToRealModePart2
mov eax,esp
jmp far [eax]
TempSwitchToRealModePart2:
USE16
mov ds,[OldDS]
mov es,[OldES]
mov fs,[OldFS]
mov gs,[OldGS]
mov ss,[OldSS]
nop
add sp,6
ret


ReturnToProtMode:
mov eax,cr0
or al,1
mov cr0,eax
push 8
push ReturnToProtModePart2
mov bx,sp
jmp far [bx]
ReturnToProtModePart2:
USE32
mov ax,8*2
mov ds,ax
mov es,ax
mov fs,ax
mov gs,ax
mov ss,ax
add sp,4
ret



OldCS dw 0
OldDS dw 0
OldES dw 0
OldFS dw 0
OldGS dw 0
OldSS dw 0
Once you copy this code and save it in a text file (asm or txt file extension doesn't matter to NASM), you should be able to be compile it with the command line:

Code: [Select]

nasm -f bin -o filename.com filename.asmIt runs fine in DosBox. I haven't tested it on real hardware yet.

[Frank Kotler]

I'm impressed, Ben. Thanks for sharing!

Best,
Frank

[fredericopissarra]

You are in the right track, but here's a few pointers:

1 - PUSH EBP/MOV EBP,ESP and LEAVE aren't necessary (in 16 bits they are because you access the stack only though BP register).
2 - To end the program a simple RET is wrong. You need to call service 0x4c from int 0x21;
3 - To switch back to real mode you need to reload the GDTR, not simply reset PE bit from CR0 and do a JMP to serialize the processor.

If this code works, you got lucky... I'm sorry, but it shouldn't.

[ben321]

1 - PUSH EBP/MOV EBP,ESP and LEAVE aren't necessary (in 16 bits they are because you access the stack only though BP register).

It may not be required for 32bit code, but it's still useful. I do it all the time when writing assembly code for Windows. It provides a fixed reference point within a function to find both local variables and parameters for the function pushed on the stack. And yes, I know that in 32bit code you can reference stack memory with any of the registers, even the ESP register (though that's less useful as it changes with every push or pop, so it's not a fixed reference point). In fact, most 32bit Windows software I've seen will push EBP on the stack and then copy ESP to EBP at the start of a function and then use leave (or a combination of mov and pop instructions) to go back to the previous stack frame.

2 - To end the program a simple RET is wrong. You need to call service 0x4c from int 0x21;

I didn't know you needed to do that. I assumed that when I typed the name of an executable EXE or COM file in DOS, and then pressed Enter to start the program, what it would do is cause DOS to use a CALL instruction to go to the executable file's code. After that, since a CALL instruction was used in the OS to start the executable program, you use the opposite function (the RET instruction) to leave your program and go back to the OS. The only requirement I thought for this was to make sure to return the state of certain required registers back to the exact state they were in when your program first started running. Those registers that are mandatory to preserve before calling RET to quit your program are BP, SP, DI, SI, BX, and all of the segment registers (CS, DS, ES, FS, GS, and SS). My program made sure to preserve the state of all those registers before calling the RET instruction. And as far as I know, the CALL instruction run by the OS to launch your executable file is a call instruction without any parameters pushed onto the stack, so a RET 2 or RET 4 isn't require, just a simple RET.

Not sure about int 0x21 being used to close a program. I thought it was int 0x20 to close a program via interrupt.

3 - To switch back to real mode you need to reload the GDTR, not simply reset PE bit from CR0 and do a JMP to serialize the processor.

Interesting.  Reload the GDTR with what though? You mean pass a null pointer to the LGDT instruction, to tell the CPU that a GDT is not in use?  By the way, I didn't just to a JMP. I did a far JMP. This sets the CS register to the specified segment value, and if the PE bit in CR0 is clear, then I figured the CPU looks at the CS as part of the memory address rather than an index into the GDT, so once I did a far jump after clearing the PE bit I figured it would force the CPU back to real mode. And at least in DosBox, it seems to have worked. If indeed that is the wrong way, it may not work on real hardware. Maybe this is just a bug in DosBox that makes it less accurate to real hardware.

[fredericopissarra]

1 - PUSH EBP/MOV EBP,ESP and LEAVE aren't necessary (in 16 bits they are because you access the stack only though BP register).

It may not be required for 32bit code, but it's still useful.

But totally unecessary. As well as coding your routines (in assembly) using cdecl like calling convention if you don't use any libc functions.

I do it all the time when writing assembly code for Windows. It provides a fixed reference point within a function to find both local variables and parameters for the function pushed on the stack.

You can do this using structures if you really want to use stack frames:

Code: [Select]

; A simple funtion returning 2*x:
struc fstk
    resd  3   ; saved registers (by called function)
    resd  1   ; return address (pushed by call)
x:  resd  1   ; argment
endstruc

f:
  push  ebx   ; let's assume you want to save
  push  esi   ; these registers.
  push  edi

  mov   eax,[esp + fstk.x]
  add   eax,eax

  pop   edi
  pop   esi
  pop   ebp
  ret
No need to use EBP...

... In fact, most 32bit Windows software I've seen will push EBP on the stack and then copy ESP to EBP at the start of a function and then use leave (or a combination of mov and pop instructions) to go back to the previous stack frame.

Most unoptimized code you mean...

2 - To end the program a simple RET is wrong. You need to call service 0x4c from int 0x21;

I didn't know you needed to do that. I assumed that when I typed the name of an executable EXE or COM file in DOS, and then pressed Enter to start the program, what it would do is cause DOS to use a CALL instruction to go to the executable file's code.

It may be, but DOS prepares a lot of things before making this call and must unprepare them when it leaves the process. That's why there is a service for process termination. There is two ways to terminate a process in MS-DOS: int 0x21 / ah=0x4c and int 0x20 (for COM files ONLY). The first allows you to return an ERRORLEVEL.

3 - To switch back to real mode you need to reload the GDTR, not simply reset PE bit from CR0 and do a JMP to serialize the processor.

Interesting.  Reload the GDTR with what though? You mean pass a null pointer to the LGDT instruction, to tell the CPU that a GDT is not in use?  By the way, I didn't just to a JMP. I did a far JMP. This sets the CS register to the specified segment value, and if the PE bit in CR0 is clear, then I figured the CPU looks at the CS as part of the memory address rather than an index into the GDT, so once I did a far jump after clearing the PE bit I figured it would force the CPU back to real mode. And at least in DosBox, it seems to have worked. If indeed that is the wrong way, it may not work on real hardware. Maybe this is just a bug in DosBox that makes it less accurate to real hardware.

See table 9.1 at Intel SDM to have an idea of how to do it... Notice you load the segment descriptors with Granularity flags turned on and never turned it off to go back to real mode (just an example). As for the far jump, to go to protected mode you should do a far 32 bits jump after loading GDTR, not a simple far jump. You are probably NOT in protected mode there because the processor will jump to protected mode just after this specific kind of jump.

[fredericopissarra]

Here's an example on how to go to protected mode (not back to real mode) using a lagacy MBR...

Notice, as an example, you forgot to enable Gate A20 when jumping to protected mode (and disable it when back to real mode).

[ben321]

Notice, as an example, you forgot to enable Gate A20 when jumping to protected mode (and disable it when back to real mode).

I wasn't using the extra memory space, just testing how to transition in and out of protected mode.

[ben321]

See table 9.1 at Intel SDM to have an idea of how to do it... Notice you load the segment descriptors with Granularity flags turned on and never turned it off to go back to real mode (just an example). As for the far jump, to go to protected mode you should do a far 32 bits jump after loading GDTR, not a simple far jump. You are probably NOT in protected mode there because the processor will jump to protected mode just after this specific kind of jump.

I absolutely was in protected mode. I made sure to use a far jump (or in this case a far call). If you read the code I posted at the top of this thread (in fact you initially were replying directly to the code I'd written, when you criticized it), you would see I did make this far call. In fact, DosBox Debugger is what I used to test my code, and like any good debugger it reports the internal state of the CPU. Its mode indicator changed from Real to Pr32 when I made that far call. And after turning off the CR0 bit and then making a far return, the debugger's mode indicator went back to showing Real. And I didn't even have to reload the GDT. Then when I exited to leave back to DOS, I had no problems. Had I attempted to go back to DOS while still in protected mode, it would have crashed.

However you say you need to reload the GDTR when leaving protected mode (even though it worked for me in DosBox without doing this step). So, please inform me more about that. What am I supposed to reload the GDTR with, in order to properly leave protected mode? Am I supposed to load it with a null pointer? Also, am I supposed to do this reloading of the GDTR just before or after I clear the PE bit in CR0?