Hacker News Reader

Comment by 5- | Hacker News Reader

note that you can switch to long mode directly, without going into protected mode first, with way less code:

https://wiki.osdev.org/Entering_Long_Mode_Directly

i've had a bootloader for a small 64-bit kernel based on this that fit comfortably into the bootsector, including loading the kernel from disk and setting up vesa modes, no stage2 required.

View on HN

> i've had a bootloader for a small 64-bit kernel based on this that fit comfortably into the bootsector, including loading the kernel from disk and setting up vesa modes, no stage2 required.

How in the world do you fit all that in 512 bytes? I'm guessing you don't have a real-world filesystem (that allows the kernel to be anywhere on the disk just as a normal file)? Because just dealing with file fragmentation should bump you way over 512 bytes I would imagine.

yes, the kernel was in a known location on disk (directly after the bootsector).

the whole boot disk image was generated during build, which is common for small systems.

I've been thinking about how to structure a filesystem such that code complexity - in the boot loader and elsewhere - can be minimized. You'd want something similar to NTFS, except that it should be possible to refer to a file (MFT entry) directly by starting sector number. So they would either need to always remain at a fixed position, or have a link pointing back to any reference so it can be updated.

Somewhere in the first sector (aligned to a 64 bit boundary), there would be a small structure that just contains a unique signature (not ASCII but some random value; 64 bits seems like more than enough as opposed to a GUID), and a pointer to the "FS info block". Leaving all remaining space in the sector available for boot code or possibly another "overlayed" filesystem.

That info block in turn would point to the MFT entry for the stage 2 boot code. An MFT entry would contain at a minimum an easy to locate list of (start sector,count) extents. Maybe there could be a flag that tells the OS that a certain file should never be fragmented?

File names would be entirely optional and for human consumption; for direct links within a filesystem, sector numbers would be used, and some kind of unique numeric identifier for anything "higher-level".

I'm genuinely wondering if some expert here sees any problem with this scheme, other than it not conforming to how mainstream OSes do things?

> File names would be entirely optional and for human consumption; for direct links within a filesystem, sector numbers would be used, and some kind of unique numeric identifier for anything "higher-level".

Each file has index number ("Inode"?) in the MFT. The first 24 (0-23) are reserved, and 11 of them contain metadata about the volume . https://flatcap.github.io/linux-ntfs/ntfs/files/ - somewhere in the Windows API is something that allows opening a file by "Inode" number. This link and info may be really old so it could be more of the reserved inodes are used now.

So, if 23 isn't being used yet, you could use that to put your 2BL - create a file there and name it $2BL or something. Would be funny to see what future Windows update that does use it does to it, if that ever happens (and of course maybe it is used).

> Maybe there could be a flag that tells the OS that a certain file should never be fragmented?

Haven't looked but I recall from an old book I read that small files are stored right in the MFT and I think existing data + the cluster size is the limit there.

the first somewhere in the first sector - this makes me think of the superblock concept. u scan the start of the disk for it (ext2 or 4? or both?). Not the first sector because that's for MBR purposes, but after that...

not fragmenting files.... well. U don't fragment files until you do.. u know. If u write an FS it will only fragment what u tell it to. So if u want all contigious files then u can simply do it that way. store them as (file_id,sector_count,start). or even more simple (fname_len,fname,sector_count,start_sector) so u dont need to keep nameblock around for filenames (fat?)

If u look at Ext2/4, FAT16/32 and others u will see that it all started quite basically, keepin a list of files and offsets. But due to things like disk reliability, user errors, system stability issues etc., you need a lot of extra stuff.

Also, questions like : how long is a maximum for a filename length? This kind of stuff can really impact what kind of features u need to implement.

How long is a file allowed to be? How many files are maximum for the FS?

This might sound silly, but there's already datacenters out there (a lot actually) who cannot use most filesystems because either files are too huge, partitions are too huge, too many files are present for index structures etc.

If you want dead simple for a starting OS: (sector_count,start_sector,fname_len,fname,0)

If you want more, try looking at ext2 or FAT32 or if your system specifications require it, look even beyond. (ext4, ZFS, NTFS, etc.) - A lot of these are subtely different, trying to solve different problems, or similar problems in different ways.

> Because just dealing with file fragmentation should bump you way over 512 bytes I would imagine.

Historically, many filesystems have had a special file type or attribute for contiguous files - a file guaranteed to not be fragmented on disk. Most commonly used for boot loaders, OS kernels, or other essential system files - although historically some people used them for database files for a performance boost (which is likely rather marginal with contemporary systems, but decades ago could be much more significant).

Some systems required certain system files to be contiguous without having any special file metadata to mark them as such - for example, MS-DOS required IO.SYS and MSDOS.SYS to be contiguous on disk, but didn’t have any file attribute to mark them as contiguous. Unlike an operating system with proper support for contiguous files, DOS won’t do anything to stop you fragmenting IO.SYS or MSDOS.SYS, it is just the system will fail to start if you do. (Some might interpret the System attribute as implying Contiguous, but officially speaking it doesn’t.)

you can just stick the kernel at sector 2 and read it from there using extended bios disk read. specify your DAP to read the kernels amount of sectors starting from sector 2, load it to something like 0x7e00 or some reachable place. It will have still limits on how much it can read, per read and in total.

If you do this all within 1 sector, equally you do not do any error checking. just ram stuff into memory and yolojump into it.

the basic would be: load kernel from disk using bios interrupt get memory map using bios interrupt parse kernel ELF header / program headers and relocate it - elf header to find ph_off and ph_num and entry_point - program headers to find all pt_load and rep movb them into tgt phys addr.

Also with 510 bytes generally u will not make nice 'page tables' though this is actually possible with only few bytes. - i did not manage it yet in 510 :D but i am sure there's someone who can do it... it can be done really efficiently.

the disk would be formed by assemblding the mbr. then doing something like

cat mbr.bin kernel.bin > disk.bin (maybe here use truncate to padd the disk.bin to a certain size like 10+MB or so - helps some controllers recognize it better)

all that said it's not useful to do this. you will find it an interesting excersize at best. like trying to make 'tiny ELF' file or so. fun to learn, useless in practice.

Or just use https://limine-bootloader.org/, which greatly simplifies everything. No messing around in real mode (even when doing SMP), automatically loads your kernel using a higher-half mapping, and also works on aarch64 and riscv64.

To be fair, writing a bootloader is an interesting and educational project in its own right. But yes, for most people interested in osdev they should just use an existing bootloader. It gets you to the part that interests most people (writing the kernel) faster, and without having to worry if you are going to run into gnarly bugs because the bootloader is buggy and you never realized.

you are right. Though with the partition table in there so u can support a 'modern' AHCI controller and SATA it will shrink your bootloader further and require some optimizations.... - you don't have 510 bytes for the loader in this case but a bunch less. if you want to populate the table with valid entries then it becomes even more tricky (can't use any bytes inside of the table...)

If you want to use an actual modern harddisk, you might want to look at GPT rather than MBR, as it won't overflow partition table stuff and allow for very large disks (2TB+?) (uefi gets rid of all of that and allows u to use a proper disk layout without any additional difficulty!)

there is no need for protected mode if you want to launch into 64-bit mode. I would say though, DO NOT USE BIOS. It's a piece of rubbish which will just make things more tedious.

Using UEFI via EDK2 or GnuEFI is the way to go, and both methods are really easy and a blessin to implement. It's a bit hard to get around the initial idea of UEFI, but if you view some other people's example projects on github you can find easily how it works. EDK is a bit shitty with .dec and .inf files etc, and GnuEFI is just reading headerfiles to discover what is there, but it's inifnitely better than the unspecified bios interface. You can litterally not even assume the int 0x10, int 0x15 etc. are there properly if you start running on real hardware. On UEFI systems, you can assume a stable minimal basis, and easily enumerate other things (hardware/platform capabilities etc.) in a sane way.

Also UEFI sets up the platform a long way already, so you don't need to do any initialization for your os-loader component (stage2 or 3). You can simply start loading your os right away. or drivers or whatever kind of design your kernel is. get memory map, get some access to efi file system, start pulling in and loading stuff.

ACHI (which is how SATA is exposed) doesn't change any of this. The only thing that is affected is how the loaded OS has to talk to the disk controller. The real thing that loses space is the BPB (a 60 bytes or so, iirc) because some BIOSes are broken and require it and the MBR (but that's only a couple bytes). At least [bootelf] manages to fit in (without a BPB or an MBR) with 128 bytes to spare, enough for a dummy BPB that makes all BIOSes happy.

Additionally, UEFI's reliability is.. sketchy, as far as I know (using the classic logic of "if Windows doesn't use it does it really matter?"). And GNU-EFI suffers from build portability troubles, AFAIK.

[bootelf]: https://github.com/n00byEdge/bootelf

If you chose AHCI on QEMU it will require a partition table to be present on the disk, so it recognizes it as a bootable disk. in addition to the magic value. If you do not add the partition table.

Thanks for this comment. It makes me realize this is likely not an AHCI thing ,but how the seaBIOS(? qemu's flavor?) handles enumerating disks via AHCI rather than IDE.

If it uses the IDE controller then it will recognize the boot disk. If i pick AHCI, i need to add the partition table.

UEFI reliability is sketch, but really BIOS is incredibly crap, so much more than UEFI.

Windows DOES use EFI/UEFI, how else will it boot on a system that has EFI/UEFI firmware inside of it? It can let you do secureboot, edit efi variables... - where do you get this classic logic from? (maybe i am totally misisng something, but they interface with it, and should thus use the spec? even tho they might not use gnuefi or EDK2 ofc :P (edk2 is still likley...))

Oh, cool. I never knew that was possible. Showing my ignorance here, but assuming we're just trying to get to long mode, why would we tour through protected mode at all?

i may be misremembering, but it's likely that intel/amd never documented this one-hop procedure, and it was discovered by the community through a combination of rules-lawyering and experimentation.

Yes, you can do that too

Reply on news.ycombinator.com