Author Topic: how to switch video modes (switching to VESA video modes too) in protected mode  (Read 26469 times)

Offline lukassevc

  • Jr. Member
  • *
  • Posts: 5
Hi, I am Lukas,
and I am writing my operating system but I have problem with switching video modes ( switching to VESA modes too ) because I don't know how to do it and I don't found any tutorial on it so I need help. I know that in real mode it can by done by using ah = 0x00 and with interruption 0x10, but I am in protected mode so I don't know how to do it. Can someone help me please?
Thanks for any help!

Offline fredericopissarra

  • Full Member
  • **
  • Posts: 388
  • Country: br
http://www.petesqbsite.com/sections/tutorials/tuts/vbe3.pdf

See "Obtaining the Protected Mode Entry Point", page 21 -- this is NOT available in all implementations and I was never been able to make it work properly...

Offline lukassevc

  • Jr. Member
  • *
  • Posts: 5
http://www.petesqbsite.com/sections/tutorials/tuts/vbe3.pdf

See "Obtaining the Protected Mode Entry Point", page 21 -- this is NOT available in all implementations and I was never been able to make it work properly...

I read it, and it seems to that it can work properly, but can you please tell me how to do it in C language?
Thanks!

Offline fredericopissarra

  • Full Member
  • **
  • Posts: 388
  • Country: br
I read it, and it seems to that it can work properly, but can you please tell me how to do it in C language?
I could, but I thought this forum was about assembly language, isn't it?

Offline ace012k8

  • New Member
  • Posts: 1
I ran into the same headache back when I was experimenting with writing a small OS kernel for fun. Switching to VESA modes in protected mode easily becomes a rabbit hole. I had to do a bunch of hacks just to get the BIOS to cooperate. The legacy BIOS interrupt (INT 0x10) just won't fly once you're out of real mode, so I remember relying on setting up a trampoline to call into real mode temporarily or using something like a VBE 2.0+ Linear Framebuffer approach after querying video modes beforehand.

For videos and demo recording, I always used a lightweight editor to polish up footage or build quick walkthroughs. If you’re working on dev-related content and want to make clean quick edits, especially on Windows, I picked up a solid one here: https://www.movavi.com/imovie-for-windows/. Easy to use and doesn’t overcomplicate things.
« Last Edit: April 11, 2025, 08:41:07 AM by ace012k8 »

Offline fredericopissarra

  • Full Member
  • **
  • Posts: 388
  • Country: br
I ran into the same headache back when I was experimenting with writing a small OS kernel for fun. Switching to VESA modes in protected mode easily becomes a rabbit hole. I had to do a bunch of hacks just to get the BIOS to cooperate. The legacy BIOS interrupt (INT 0x10) just won't fly once you're out of real mode, so I remember relying on setting up a trampoline to call into real mode temporarily or using something like a VBE 2.0+ Linear Framebuffer approach after querying video modes beforehand.
Yep... OR you could setup and use Virtual 8086 mode (NOT available in x86-64 mode!) to do it... Anyway, it is a nightmare.

Another problems are: Getting the Linear Framebuffer Physical address from ModeBlockInfo strucuture... Often this field is zeroed in VMs like QEMU, for example... And page switching, for double buffering... another nightmare.

This and the Protect Mode Interface, described in VBE 3.0 documentation usually are not present (not even in Option-ROM that a nVidia card puts there)...

Offline marknoble

  • Jr. Member
  • *
  • Posts: 3
For VESA in protected mode, you gotta use VBE functions, not BIOS interrupts. Check the VBE specs on OSDev wiki for mode info and framebuffer setup. It’s tricky but doable. What bootloader you using? Might help narrow it down.

Offline ben321

  • Full Member
  • **
  • Posts: 187
Hi, I am Lukas,
and I am writing my operating system but I have problem with switching video modes ( switching to VESA modes too ) because I don't know how to do it and I don't found any tutorial on it so I need help. I know that in real mode it can by done by using ah = 0x00 and with interruption 0x10, but I am in protected mode so I don't know how to do it. Can someone help me please?
Thanks for any help!

That's because the interrupts work differently in protected mode. In protected mode the CPU ignores the IVT (interrupt vector table) that was used in 16bit real mode, and instead requires you to set up an IDT (interrupt descriptor table). In 16bit protected mode, you might be able to (if you're lucky) get away with pointing the IDT to the same interrupt functions that the IVT originally pointed to (but if not, then you'll need to take the same steps as if you were working with 32bit protected mode). But if you mean 32bit protected mode, then that's not a possible shortcut at all, because the functions pointed to by the IVT are all written in 16bit code, not 32bit code. So you will need to write your own 32bit functions (or 16bit functions if you are talking about 16bit protected mode) for EVERY SINGLE FUNCTION that exists in the IVT. That's 256 functions you will need to write! Not worth the effort, if you are doing anything other than writing an entire operating system. Since I assume you are writing software, not an OS, I would advise you to not bother with this effort, unless you really just want to go through with it for the sake of learning how to do it (maybe as a project in a programming class at a university or something).

Offline fredericopissarra

  • Full Member
  • **
  • Posts: 388
  • Country: br
For VESA in protected mode, you gotta use VBE functions, not BIOS interrupts. Check the VBE specs on OSDev wiki for mode info and framebuffer setup. It’s tricky but doable. What bootloader you using? Might help narrow it down.
You can do this only with PMI is available to you. It isn't the case of QEMU and even at ROM BIOS extentions of major modern graphics cards like nVidia's (at least I can't find it using the described method from VBE3 documentation).