Hacker News Reader

For me my reversing started and ending with discovering and documenting the List of Lists:

https://fd.lod.bz/rbil/interrup/dos_kernel/2152.html

That allowed TSRs to do clever things.

I didn't use CP/M before DOS, but these days I know all about the PSP via writing code for CP/M, in Z80. Kinda impressive how long all these backwards compatibility things last.

One of the first things I looked up, as a budding Turbo C developer in the DOS era, as it held all manner of secrets.

I seem to recall 'one guy' had this hand-curated master list of interrupts. Might've been this list; perhaps another.

I was obsessed with TSR programs back then. The notion that you can run a background process (in exchange for gobbling up all-valuable conventional memory) thanks to clever hacking really captured my eye back then.

Chap you're referring to is Ralf Brown, he published a list of all known interrupts. It is on the web https://www.cs.cmu.edu/~ralf/files.html

There was a pair of books that were instrumental to me: Undocumented PC and Undocumented DOS. Those books taught me so many tricks!

I still remember the DWIM (Do what I mean) interrupt they used as the demonstration of how each interrupt was documented.

The Undocumented PC, A Programmer's Guide to I/O, CPUs, and Fixed Memory Areas by Frank van Gilluwe is great. Frank van Gilluwe is the author of Sourcer disassembler https://corexor.wordpress.com/2015/12/09/sourcer-and-windows...

> the List of Lists:

> https://fd.lod.bz/rbil/interrup/dos_kernel/2152.html

> That allowed TSRs to do clever things.

I think they were also used (=manipulated manually) by viruses, in order to prevent DOS from reporting the reduction in memory, and the user from noticing the virus.

Yes, stealth viruses did report memory without their space, by monitoring the read, and returning the "clean" value instead.

Boot sector viruses couldn't do anything like that because they load and run before DOS enters the picture. So that was one of the things my boot sector looked for. File infectors had a number of tricks at their disposal, one of which you just mentioned. Other methods are still similar, just different means to that end.

Awesome! I encourage you to post your code and blog about it. I would find it fascinating, and suspect others would too.

I didn't write a blog about it, but the code for the boot sector and installer (dropper) is here: https://github.com/ksaj/antivirus-mbr I separated it out because it was useful for all versions of DOS at the time, and boot sectors, being outside the DOS realm, were always a sitting duck.

I documented it to death, so it is super easy to read even though it is ASM code.

> One thing that I always thought was interesting was all the stuff that gets packed in the first 100h bytes before a .COM file's entry point ...

AIUI that design was in CP/M already, and retained for compatibility.

The two File Control Blocks are also vestigial remains of CP/M, and they can also be found in the same section of memory.

You and I would have good party conversations

Well tonight we're going to party like it's 1999.

This article was originally published right before the issuance of MS-DOS 2.0. That was not even near the '90's yet.

By the time this was captured in the year 2000, the MSDOS.SYS file had already become almost a dummy file of few bytes in Win9x.

The DOS OS itself with only the native commands is basically contained within the IO.SYS, MSDOS.SYS, and COMMAND.COM files, originally written in that order starting at the first working sector of a formatted floppy disk so it would boot.

On a hard drive the 3 files would have to be written starting at the first working formatted sector of an actively flagged primary partition, and a proper master boot record needs to be on sector 0 (along with a valid partition table) if any boot files on that HDD are to be recognized by BIOS to begin with. Otherwise it would not boot.

IOW once the volume boots you can do things like copy files and list directories, but if you want to edit files or format other volumes you needed more than just the 3 OS files. You needed the additional OS apps like EDIT.COM and FORMAT.COM on your floppy so they would be there when you needed them after you booted.

Once booted, the SYS command could be used to properly write the OS to a target volume, so you needed SYS.COM for that.

And by W9x, the three DOS OS files would no longer need to be right there at the start of the volume (as if they were the first things copied to the volume), and DOS FDISK would handle HDDs of hundreds of GB even though each FAT32 partition was only supported up to 32GB.

The undocumented FDISK /MBR command executed from an A: or B: (floppy) prompt would simply write a fresh bootable DOS master boot record to sector 0 of C: (what should usually be considered) your only HDD on the mainboard. Without disturbing the partition table there.

The SYS command also replaces the Volume Boot Record (bootsector) of the target volume with a fresh DOS bootsector as it freshly (over)writes IO.SYS, MSDOS.SYS, and COMMAND.COM into the file system.

So you want to make a zeroed HDD boot, first boot to a DOS floppy (USB stick will do):

Run FDISK and create a primary partition (bigger than 512MB) smaller than 32GB on the fixed (HDD) disk. Make the partition Active.

(Alternatively you could use some other way to make a suitable partition, like Linux or a modern Windows version but the resulting "geometry" of the disk layout may not be the same or fully recognizable by DOS, which can affect the bootability. Sometimes with Linux or Windows, the older the version the better. Outcomes can also be surprisingly interesting when certain sectors of the HDD are not exactly zeroed beforehand. AMD also not quite the same as Intel, nor old Seagate identical to WD.)

Now you've got a partition table on sector 0 with one primary partition marked "active" so the BIOS will boot to it.

Run FDISK /MBR and it silently writes an MBR to sector 0 so the BIOS will REALLY boot to it.

(You may need to manually copy a working MBR to the first 440 bytes of sector 0 if the FDISK /MBR command does not do the trick. The MBR is supposed to be generic for DOS and a Linux version like MBR.bin from the Syslinux project can be copied bytewise with binary tools. Newer Windows will also upgrade the MBR but it will still boot DOS (or Linux) afterward from a submenu.)

All other sectors are still unchanged.

Run FORMAT on the partition and I would always suggest labeling it as to HDD/Partition# when done. 11 characters max.

Now you've got something other than zeros in places other than sector 0 of your HDD, but still no data actually located on the formatted volume yet.

Run SYS on the formatted volume and it properly writes the 3 critical DOS OS files to the volume.

Power down, remove the floppy or USB device.

The HDD should then boot DOS according to mainboard BIOS priority.

Add W9x to the same FAT32 volume and you still get a DOS boot entry on your Microsoft boot menu for when you want plain DOS.

Install WXP to a different formatted partition (even FAT32) and your old W9x/DOS volume still can boot when desired as a submenu of the NT5 NTloader that replaces the DOS bootloader on that active partition.

Then with Win10 (non-UEFI setup) on a third (NTFS) partition you can have an NTLDR submenu within the replacement NT6 Bootmgr and boot any of the above.

Note the volume you installed the new Windows version to does not become active itself nor contain any boot files. By default additional boot entries are made to the established active partition. Even logical partitions containing OS's can be booted this way, according to OS version, from one of the few primary partitions you are allowed to flag as active.

BIOS only. Too bad with foolish-type UEFI or GPT, no DOS for you.

Also remember with bare-metal DOS/W9x and audio/MIDI (no USB), no latency for you either.

Even on dog-slow PC's.

Things like that.

This is a wonderful explanation. Thank you very much.

If you were mucking around with dual-booting in the 90s DOS-era, you had to know about `fdisk /mbr' as you'd inevitably trash your master boot record. It became a bit of a ritual for people to tell others about it, and I do remember it being undocumented at the time.

One of the deficiencies in GRUB was the tendency for Linux experimenters to start with a Windows PC, install Linux to a completely different unused primary partition, and end up with GRUB installed to the drive the Linux partition was on, rather than the Linux partition itself.

This overwrote the generic (DOS/Windows) MBR on sector 0 with a Grub-specific MBR which hard-linked to more Grub-specific code stored on otherwise unused sectors located in between sector 0 and the first sector of the (not necessarily formatted) first partition, expected to be at least 31 sectors available for Grub to install to. And for Grub to have those sectors to itself (not true if other apps treated it as scratch space outside the file system).

You could link Grub to other sectors if you wanted to but this was where the default location was.

You could also always have a dedicated boot partition but this did not become mainstream until Windows 7 and it was with BIOS booting. That approach has now been continued into UEFI with the dedicated EFI folder on the hidden ESP partition.

So Grub took over your PC and that's how it was going to boot. Most of the time in Grub1 you had to carefully manually edit the grub.conf file to get your old untouched Windows partition on the boot menu so your dual-boot layout was ideally functional.

This didn't always happen during a user's first Linux experience so the HDD was basically stuck in "Linux only" until they figured it out. If the urgent need to boot the old Windows came up, reboot to the emergency boot floppy containing FDISK.EXE, then run FDISK /MBR and everything would be back to "Windows only" after that.

With Grub2 it kind of got worse since grub.cfg came along in place of grub.conf, and grub.cfg is intended to be autogenerated and not manually edited. The recommended recommendation is to never manually edit the grub.cfg file. I recommend not always doing what's recommended if you want to really get what you want. But don't fail to back up your work in case it autodisappears. In the long run you could really get into Grub2 and make it autogenerate your dreams.

Either way it was much better when installing Linux to a type 83 partition to leave the generic DOS/Windows MBR in position at sector 0, and instead direct Grub to be installed to the same partition (that partition's bootsector) that Linux was destined for. This made the (EXTx-formatted) type 83 Linux partition completely bootable on its own. But only if that was a primary partition to begin with and was flagged active with a binary 80 in its partition table entry in sector 0. And at the time the Windows partition would instead have a 00 flag, therefore not being active and was not the primary to boot that time.

You would have to manually assign which primary partition to flag active so you could multiboot, but it is accomplished that way without even a menu to configure. This was best done manually with a binary disk editor although some menu-interface shells arose such as Bootmagic which handled self-contained bootable partitions especially well. Boot magic also usually installed to "unused" sectors on the HDD outside any file system.

Further, the functional grub bootsector for the partition it is installed to can be copied to a binary file (named something like bootsect.lin) on an active FAT or NTFS partition where an NT install had often autosaved a copy of a FAT DOS bootsector as bootsect.dos. (Before it was replaced with an NT bootsector).

BOOT.INI on the active Windows bootup partition could then be edited to include a boot entry invoking bootsect.lin and the NT bootloader would jump to the target type 83 Linux partition and carry on from there.

So you could install and boot Linux ideally to an extra partition on a Windows PC without disturbing the Windows in any way. Just adding the Linux option to the default Windows boot menu.

When a small separate FAT32 boot partition began to appear before the main NTFS volume on OEM Win7 PC's, that's when Syslinux began to really shine. Because Syslinux is made to boot Linux from an active FAT partition similar to how Isolinux boots Linux from CDROM. Syslinux has its own kind of bootsector and relies on a syslinux folder and/or syslinux.cfg file so for a while there it was a better choice than Grub. You just had to use the same version of syslinux that the distro was isolinux'ed with, and rename the isolinux folder & isolinux.cfg file to syslinux & syslinux.cfg, with judicious manual editing.

Syslinux is stll superior for BIOS booting but for UEFI Grub2 is what seems to work most predictably now.

Under UEFI Grub2 acts about the same way always booting from the dedicated EFI folder on the hidden FAT32 ESP boot partition. It simply overwrites \EFI\boot\bootx64.efi with a distro-specific version from the distro that you are booted to when you update grub. For instance if you install Ubuntu to a Linux-formatted GPT volume and properly direct Grub to install in the EFI folder it makes a new bootfolder \EFI\ubuntu in the process and the new \EFI\boot\bootx64.efi no longer seeks \EFI\Microsoft, it seeks \EFI\ubuntu instead (where it finds Ubuntu's grub.cfg). But it leaves \EFI\Microsoft untouched, and each additional distro adds its own like \EFI\fedora and \EFI\debian, and they all coexist wonderfully.

Don't let different mainboard UEFI implementations fool you, the firmware itself is supposed to recognize different boot options as they arise and you can get a firmware boot entry (available from a separate usually disabled mainboard UEFI bootmenu) for each installed OS, which all may not be functional choices after a while. But \EFI\Microsoft\bootmgfw.efi can be like a hardcoded firmware boot entry as a failsafe alternative (Windows boot manager) to an expected \EFI\boot\bootx64.efi. Either of which \EFI\boot\bootx64.efi or \EFI\Microsoft\bootmgfw.efi are further alternatively executable from a UEFI shell. Whether the UEFI shell is built into the mainboard firmware, or if not you have to provide shellx64.efi on a pre-boot (usually FAT32) file device.

Whichever distro you are booted to will be the one whose grub.cfg is reautogenerated when you update grub. You are expecting the grub update process to autodetect all installed OS's including Windows and place entries for them on a boot menu in grub.cfg which appears routinely every time you restart the PC. Some distros do this better than others.

So it's a good idea to back up the Windows version of bootx64.efi for possible restoration to \EFI\boot before allowing Grub to overwrite it with a Linux version. Then again you might want to back up your Linux versions of bootx64.efi as they come along too.

Pretty much the modern substitute for an FDISK /MBR, although the BCDBOOT command (sometimes requiring the /uefi option) overwrites \EFI\boot\bootx64.efi with a fresh Windows version, while it examines your \EFI\Microsoft\boot\BCD for any corrections it wants to make.

Just to bring things more up-to-date, it's not 1999 any more.

How much of DOS did you learn by reading MS's raw source code vs reversing the OS yourself?

I glanced through it. I didn't learn anything new, except the definition of bloat. The raw source wasn't available when I was doing this, so I learned it entirely by reverse engineering, and documentation.

Ironically I learned how the boot sector worked and basic ASM purely by using DEBUG.EXE. My computer got infected by Stoned virus, and I was pretty excited about it! This was back when DOS came with a green binder with actual documentation. It was a year or so of that before I discovered TASM and then MASM assemblers, which of course are a LOT more human-friendly.

The earlier versions of DOS were entirely written in ASM, and they were super-tight. But later versions were all rewritten in high level languages. The stuff in the leak is all the bloated high-level language versions.

"and 97% of the population probably would not be nearly as enthusiastic"

Thats likely a strong understatement.

Did you ever write a book? I'd love to read up on this kind of stuff.

Other people did e.g Peter Norton wrote a whole library around this stuff.