Writing an x86 bootloader in Rust that can launch vmlinux | Hacker News Reader

Writing an x86 bootloader in Rust that can launch vmlinux

vmm.dev

148 pointslukastyrychtr52 comments

I can't help but suspect the first item on the list was the whole reason for this project. :)

     Hardware initialization:

  ==>  Setting the keyboard repeat rate.
       Disable interrupts and mask all interrupt levels.
       Setting Interrupt descriptor (IDT) and segment descriptor (GDT). As a result,
       all selectors (CS, DS, ES, FS, GS) refer to the 4 Gbyte flat linear address space.
       Change the address bus to 32 bits (Enable A20 line).
       Transition to protected mode.
       If the target is ELF64, set the 4G boot pagetable and transition to long mode.

Looks nice.

I personally don't use Bootloaders anymore , using UEFI directly to boot my kernel.

Edit: It doesn't require any manual tinkering/maintainance even when updating. Also it boots fast.

https://wiki.archlinux.org/index.php/EFISTUB

How do you do that ? I didn't know you could compile a kernel directly to an UEFI payload.

Edit: Indeed it's possible with CONFIG_EFI_STUB=y : https://wiki.archlinux.org/index.php/EFISTUB - thanks for the hint !

Note that just because you can, it doesn't mean you should. How much maintenance this needs depends on your distro, your machine and firmware's own quirks (if any), and your own usage/update/maintenance patterns/quirks.

Also: directly booting the kernel as a UEFI module makes it much more difficult to set kernel parameters temporarily on boot.

If you need to boot into runlevel 3 (or use init=/bin/bash) for maintenance, it's going to be much more complicated to make this happen than it is when booting through Grub.

Is it really?

You can just boot into the UEFI Interaractive Shell (think of it as a BIOS terminal/cmd.exe), which most BIOSes have, and launch your kernel with those parameters, as regular arguments to a regular command, right?

What's hard about that?

Not really, the EFISTUB has the kernel arguments baked into it, and to my knowledge, you can't just execute it with arguments and expect it to use them.

According to the Arch Wiki [0] that can be done by using an `.efi` file but it is not necessarily the case. The UEFI can directly load the linux kernel and add parameters.

However, it's not particularly practical.

Of course, as with all things UEFI, YMMV. This worked for me on an HP EliteDesk 800 G2 before I got around to setting up the keys for secure boot.

[0] https://wiki.archlinux.org/index.php/EFISTUB#Using_UEFI_dire...

From the very same wiki you linked, and just like I claimed, you can do this directly:

> fs0:

> \vmlinuz-linux root=PARTUUID=3518bb68-d01e-45c9-b973-0b5d918aae96 rw initrd=\initramfs-linux.img

Again: how is this hard?

I never said it was hard, just not as practical as modifying a command line, for example in Grub, that already has the UUID setup.

Also, not sure how many people have the UUID lying around ready to be copied. If you have encryption setup, there's even more parameters to add. Again, it's not hard, but a bootloader with an editable command line is just much simpler.

Why go to the trouble of typing a UUID (from memory?) when Grub just works?

> \vmlinuz-linux root=PARTUUID=3518bb68-d01e-45c9-b973-0b5d918aae96 rw initrd=\initramfs-linux.img

> Again: how is this hard?

Is this a joke? Is this something you do yourself with any frequency? Do you realize how utterly painful just typing the correct UUID is? I've had to rescue non-booting systems like this many dozens of times and it makes me absolutely hate my life that on top of having to fix the actual boot problem, I also have to scramble to find a paper and pencil or grab a phone or whatever (or flip back and forth between two screens approximately 51,295,183 times) just to copy 32 random hexadecimal digits from another screen just to test other parameters from a screen I really shouldn't have to bring up in the first place. Especially given the inevitable transcription mistakes that will force you through another reboot cycle. It's such an utter frustration and waste of time to have to go through this nonsense to test a fix to a boot issue. And that's based on the ridiculous assumption that the process is somehow intuitive for people—what fraction of Linux users do you think even know there's both a PARTUUID and a volume UUID? What fraction do you think even know what to type when they face a GRUB screen, and also know what kernel parameters they need to pass? What fraction of those do you think know (and remember!) how to do all this in the UEFI shell?

> You can just boot into the UEFI Interaractive Shell (think of it as a BIOS terminal/cmd.exe)

> What's hard about that?

That you had to explain what the UEFI interactive shell even is yourself is I think quite a clear answer to your question.

> What's hard about that?

Try booting from an encrypted drive identified by an UUID in a machine with a lot of disks

What does encryption have to do with you not having a label?

In some cases, the fstab needs specific volume names, which you also need to get right.

Just have two UEFI boot entries: one booting the EFI stuff directly, and one booting GRUB, which then boots the same kernel. Then you can use GRUB to change kernel command line.

What tangible benefit do you get from booting directly if you're going to have to maintain the GRUB entries anyway? Some ε > 0 improvement in boot time?

You can boot into the EFI shell or use your firmware setup (likely DEL upon boot) to reconfigure your boot records and entries.

How common are those features? None of my systems have the UEFI shell in firmware or provide that level of control over boot records from the firmware setup interface.

Just in case you absolutely cannot boot from the rescue USB stick?

Does that make kernel updates a more manual process? Are you building your own? Are you signing and using secure boot?

I'd be interested in giving this a try.

That is what I am doing as well. I'm building my own kernel but not using secure boot.

Kernel upgrades are just copying .config from the old kernel to the new kernel.

sudo make olddefconfig && sudo make && sudo make modules_install && install_kernel

The contents of install_kernel

KERNEL=`readlink /usr/src/linux`

NUMBER=${KERNEL##linux-}

NUMBER=${NUMBER%%-gentoo}

sudo cp arch/x86_64/boot/bzImage /boot/EFI/Gentoo/vmlinuz-${NUMBER}-gentoo.efi

sudo efi bootmgr -c -L "Gentoo ${NUMBER}" -l "\\EFI\\Gentoo\\vmlinuz-${NUMBER}-gentoo.efi" -d /dev/nvme0n1

Amazing thanks for the explanation, I'll have to give this a go!

Don't compile as root, there's no need for that. You should only elevate your privileges when needed. The only thing you need root for are modules_install and your install_kernel script.

People always say this, but if a project is malicious, it can take your SSH keys and get persistence even without root. It can alias sudo in your bashrc to get root if it wants too.

Yet if you don't use sudo, some make/compiles fail with random permissions errors.

I mean I see the reasoning, but in 2020 I think the Linux desktop security model is sufficiently broken that it doesn't matter.

> Does that make kernel updates a more manual process? Are you building your own?

In Debian/Ubuntu it should be on by default, and has been for almost a decade:

https://askubuntu.com/questions/510009/does-the-ubuntu-kerne...

That means you can run your kernel directly as a UEFI-target and it will support secure-boot.

Thanks, by "it" I now understand you mean that CONFIG_EFI_STUB=y is on by default. This is the feature required to boot the kernel directly as an UEFI-target, now I need to look into how I make it an UEFI-target, I believe I need to use bootmgr as GP has, but I need to use my kernel image as the EFI image?

EDIT: I managed to create the target, with efibootmgr and boot it partly but it panics. I assume that's due to FDE so I'll have to look into how to configure it to support that (if it's possible without a bootloader).

it's possible, but you must include the path to the initramfs as a kernel parameter (using DOS-style paths, not sure if unix-style works too).

A linux-aware bootloader will usually load the initramfs itself and pass linux the memory address where it's loaded, in UEFI the kernel gets "executed" like a normal executable, so it has a commandline but not a pre-loaded initramfs.

My efibootmgr entry looks like this (root is btrfs on LUKS):

  HD(disk-id)/File(\vmlinux-5.6.0-2-rt-amd64) initrd=\initrd.img-5.6.0-2-rt-amd64 ro root=/dev/mapper/cryptpv0

The latter part is what you must supply manually if you want to start the kernel from the UEFI command line.

Great thanks, I'll give this another go.

For those interested, a colleague of mine wrote a tutorial series on implementing an OS using Rust that targets RISC-V: https://osblog.stephenmarz.com/

> "Although the slice and the target buf should be the same, the target is filled with 255 from some point. "

Doesn't Rust prevent things like overwriting memory?

Not necessarily, there are a bunch of unsafe methods you kinda have to use going this low level and if you're in unsafe mode, then you can overwrite memory or do things to corrupt it.

So all all of rust's issues, and none of it's safety?

You still get safety, if you write normal rust code then rust will largely behave the same.

But it allows you to for example to dereference raw pointers (and by proxy, allows pointer math).

A lot of unsafe functions become available, like unguarded array access (ie, you can go out of bounds of an array) but you have to explicitly do that.

No, you only face this in the unsafe code; the rest of your code is still safe.

No, unsafe code may affect the rest of your code (safe code).

You only have to audit the unsafe code for unsoundness though.

Honestly, many of rust’s safety promises don’t help the type of low level code I write, so I’m skeptical of the “embedded rust” push. But embedded is a huge field so some people will definitely benefit from what rust makes available.

But yeah, I think for some projects rust brings you a great type system, a gross syntax, and very little additional safety. But OTOH, on those projects writing rust will be not that different than writing C. Either language will still let you forget to sanitize a length value from an external struct that is put into a DMA engine. And either language will let you create a dangerous_u32 type that needs to be sanitized before your routines write it into a peripheral. But rust’s probably would be a little more ergonomic for the programmers, a little harder to use improperly, and involve pressing every punctuation mark on your keyboard.

> Honestly, many of rust’s safety promises don’t help the type of low level code I write, so I’m skeptical of the “embedded rust” push. But embedded is a huge field so some people will definitely benefit from what rust makes available.

Why exactly? Once you write an abstraction on top of the raw addresses for configuring the hardware (usually already provided by a library) you don't need to use unsafe at all. One of the first things you do in C is write macros for all the addresses you use, writing safe wrappers isn't that much more complex than that.

Because nothing the C code is doing would be "unsafe" in rust terms. What is the advantage rust has for following an SDMMC controller init sequence?

Once you define a safe interface, you only have to audit the implementation of that interface for errors that aren't guarded for in unsafe code.

The safe interface you can then use without fearing memory corruption or having to audit that code for unsafety. Any safe code is pretty much only logic errors.

Using unsafe doesn't disable the safety features that Rust has like ownership rules or type safety, it's an acknowledgement that undefined behaviour can exist and that it's the programmers responsibility to ensure that undefined behaviour isn't introduced into safe Rust. There's no real way around this, if you want to write to arbitrary memory addresses Rust can't magically prove that you're doing so safely but if you can prove you're doing so safely then Rust will take care of the rest and make sure you're not introducing data races, use after free, buffer overflows, etc.

None of it is safety?

I would think some of it safety, like strong type checking. But, you're right in that this isn't the primary use case of Rust.

This is lower level than code you normally expect. The change seems to happen around switching addressing modes, so it could be something like pages descriptors not being set up correctly. In short - rust safe memory guarantees don't hold if you mess with memory mapping at runtime.

Is the site down right now? I can't seem to connect.

Must be swamped with traffic.

Google cache works:

https://webcache.googleusercontent.com/search?q=cache:QBPnre...

That URL (to Google’s cached version) took about three minutes to load for me. Does anyone know why Google’s cache is so slow sometimes...?

Click “text only version”. Full version is just online fetch with extra header and often naturally fails to load

It was still attempting to load images, which probably caused the slowness - there is an option in the cached page header to request a text-only version which will load immediately.

Github repo has the same text in the README: https://github.com/o8vm/krabs

Yes

Reply on news.ycombinator.com