Hacker News Reader

Comment by chihuahua | Hacker News Reader

It must have been difficult and frustrating to work as part of the Windows team back in those days.

You see all the wacky software that doesn't follow the rules properly, does whatever it wants, breaks things. And you have to figure out how Windows can accommodate all that software, keep it from breaking, and also prevent it from messing up a computer or undo the damage.

They did not have the option of saying "this app developer wrote shitty software, sucks to be them, not my problem."

I wonder how much of this problem was caused by lack of adequate documentation describing how an installer should behave, and how much was developers not reading that documentation and being content when it works on their machine.

View on HN

> I wonder how much of this problem was caused by lack of adequate documentation describing how an installer should behave, and how much was developers not reading that documentation and being content when it works on their machine.

There is a third option: the developers knew the rules and chose to ignore them for some reason. A modern example of this is the Zig language’s decision to reverse engineer and use undocumented APIs in Windows in preference of using documented APIs.

https://codeberg.org/ziglang/zig/issues/31131

Kinda wild

>ignore them for some reason

The reasons are clearly stated in the issue you have linked.

"As Zig has evolved, it has become a target to avoid calling Win32 APIs from kernel32.dll etc., instead using lower-level ones in ntdll.dll."

If we needed an example of why we should avoid using passive voice, this is it.

This sentence doesn't include examples of the passive voice.

Ha, you're absolutely right. The "has become a target" got me there. So glad, Zig wasn't targeted there.

The sentence that you quoted is more generic. The Zig issue is only one example of "some reason".

I understand the stated reasons and disagree with their conclusions. It seems like a lot of extra work on the part of the Zig developers to have to reverse engineer undocumented interfaces. There is potentially extra work for the Windows developers if they want to change their private implementation details that Zig programs are now relying on. And in the case where the lower level APIs are missing features implemented higher level APIs, Zig has to either reimplment the functionality themselves or not have the same level of support for Windows features that normal Win32 apps have. A concrete example is looks like Zig programs can't open serial devices:

https://codeberg.org/ziglang/zig/src/commit/6193470ceea89a98...

All this to save a little memory CPU and memory usage? The juice does not seem worth the squeeze.

Wow! What a mind-bogglingly stupid idea. I will cancel my plans to learn Zig.

My excitement for Zig dropped the longer they stayed at 0.x (and they really have meant 0.x with the breaking changes they were making). This decision from them completely killed it for me.

I understood not using the C Runtime and instead creating direct wrappers over the Win32 API, but going a level lower to APIs that are not guaranteed to be stable is nutty.

ntdll.dll APIs are guaranteed to be stable though

The best statement of whether or not calling functions in ntdll is here: https://learn.microsoft.com/en-us/windows-hardware/drivers/k...

> documentation for the WDK and Windows SDK recommends that application developers avoid calling undocumented Nt entry points

So it's safe to call documented ntdll functions. But calling undocumented functions is more risky.

Part of this was the allowed or special treatment given to Zig at certain sites. It then allowed the false impression to be created of it being production ready and/or far more stable than it actually is. Often intentionally overlooking or glossing over that it's still in beta, making breaking changes, and has thousands of issues (over 3,000 on GitHub).

They were allowed to have it both ways, pre-1.0 and yet somehow (near, like, just about) production-ready. Almost there, for years. Strangely given a free pass to get away with this, for what looked to be undisclosed financial and other fuzzy reasons.

This comment is pretty wild:

> In addition to what @The-King-of-Toasters said, the worst case scenario is really mild: A new version of windows comes out, breaking ntdll compatibility. Zig project adds a fix to the std lib. Application developer recompiles their zig project from source, and ships an update to their users.

Uh so what if the application developer isn't around any more?

The fact that they consider the worst case to be one where the application is still actively supported and the developer is willing to put up with this nonsense is pretty surprising. Not sure how anyone could believe that.

One of the craziest Raymond Chen stories is one where a Windows API call would return a pointer to a data structure the OS had allocated for the operation. The programmers at Microsoft made the data structure bigger than they needed, for future expansion. But some third party devs noticed the extra space, and started to use it to store data for their program. Then when Windows tried to start using the extra space, those applications would crash.

Reasonable people can disagree on a lot of things in programming. But I still do not understand how one can consider writing to memory the OS owns to be ok. It's sheer professional malpractice to do that kind of thing. With stuff like that, I don't think that any amount of documentation would have helped. The issue was that those programmers simply did not care about anything except getting their own program working, and did whatever the most expedient method was to get there.

> But I still do not understand how one can consider writing to memory the OS owns to be ok.

Your manager tells you to reduce memory usage of the program "or else".

Or you desperately need to tag some system object and the system provides no legitimate means to do so. That can be invaluable when troubleshooting things, or even just understanding how things work when the system fails to document behavior or unreasonably conceals things.

I've been there and done it, and I offer no apologies. The platform preferred and the requirements demanded by The Powers That Be were not my fault.

TBH i think a more likely explanation is that they needed to somehow identify separate instances of that data structure and they thought to store some ID or something in it so that when they encountered it next they'd be able to do that without keeping copies of all the data in it and then comparing their data with the system's.

^^ The voice of experience, here.

>I still do not understand how one can consider writing to memory the OS owns to be ok.

Things were different back then. People did a lot of hacky stuff to fit their programs into memory, because you were genuinely constrained by hardware limitations.

Not to mention, the idea of the OS owning the machine was not as well developed as it is today. Windows 3.11 was just another program, it didn't have special permissions like modern OSes, and you would routinely bypass it to talk to the hardware directly.

> Things were different back then. People did a lot of hacky stuff to fit their programs into memory, because you were genuinely constrained by hardware limitations.

Are you going to tell them what "32-bit Clean" meant for Mac developers, or will we let them find out that particular horror movie for themselves?

"Not to mention, the idea of the OS owning the machine "

I agree--back then when computers had <=4MB or RAM I would've called hogging unused memory for some selfish speculative future use "professional malpractice".

Some of that attitude survives to this day:

When an OS uses any memory that's otherwise unused as a file cache, which is instantly available if an application wants more memory, but isn't shown as "unused": "This OS is terrible, I have 16GB of RAM but all of it is being used!"

When an OS doesn't do this: "This OS is terrible, I bought all this RAM and the OS doesn't use it!"

> But I still do not understand how one can consider writing to memory the OS owns to be ok.

Go to Vogons and look at all of the memory tricks people will use to get various games running on MS-DOS. This kind of juggling exactly which drivers to load, etc. is why Microsoft added the boot menu in MS-DOS 6.0 to CONFIG.SYS.

I'm not necessarily saying that this was the case here, but it smells like that to me.

Ask the Zig people, who just started relying on undocumented unstable Windows behaviour intentionally due to some kind of religious belief: https://codeberg.org/ziglang/zig/issues/31131

Back then, many programmers originally learned their ropes in an 8-bit home computer era (or earlier), where it used to be completely normal and even necessary that you used whatever memory region you got away with.

For example, on the C64, you would get away with using the memory locations $02, $2A, $52, $FB to $FE, $02A7 to $02FF, and $0313 as scratch space for your own programs. Memory was incredibly scarce. I can’t blame programmers for sticking with their habits and for taking several years to unlearn and adjust their misconceptions about who owns what if they came from a home computer era where that pattern used to be the only way to get stuff done.

Before Windows 95/3.x, there was DOS.

There were no rules in DOS, or r_x permissions like Unix.

The DOS kernel itself didn't really impose any structure on the filesystem. All that mattered was:

- The two files that comprised DOS itself (MSDOS.SYS, IO.SYS) had to be "inode" 0 and 1 on the disk in early versions,

- the kernel parsed \CONFIG.SYS on boot, and I think looked for \COMMAND.COM if you didn't specify a different shell with COMSPEC= in CONFIG.SYS. There were defaults if \CONFIG.SYS didn't exist, but of course all your DEVICE= stuff won't load and you'll probably not have a working mouse, CD-ROM, etc.

\AUTOEXEC.BAT was optional. That's it. Any other files could be anywhere else. I think the MS-DOS installer disk put files in C:\DOS by convention but that was just a convention. As long as COMMAND.COM was findable DOS would boot and be useable-and if you mucked something up you just grab your DOS boot floppy with A:\COMMAND.COM on it and fix it.

From what I recall most installers-if provided-made a directory in \ and put all their files there, mixing executables with read-write data. There was no central registry of programs or anything unless you were using a third party front-end.

Windows 3.x and 95 inherited the DOS legacy there.

> I think the MS-DOS installer disk put files in C:\DOS by convention but that was just a convention.

That assume that you where going to install the OS, which assumes that you had an hard drive :-). The original IBM PC didn't, and anyway MS-DOS didn't support folders until version 2.0.

On those old PCs you would boot your computer on a floppy drive with all the files on the root of a floppy, and execute your command there. There was not much to work with anyway, check the content of the boot floppy of MSDOS 1.0 [1].

And also, especially if you had a single floppy, you wouldn't even use it: to run your software you would boot a disk with a IO.SYS, MSDOS.SYS, COMMAND.COM and an AUTOEXEC.BAT that would start your favorite word processor (WordStar of course :-D ).

[1] https://www.youtube.com/watch?v=0-X7Thsn0pI

Yep I don't think the Microsoft installer was there until version 4 or 5. Around the time MS was making DOS more "user-friendly" with things like /LONGDESCRIPTIVESWITCHES, DOSKEY, MIRROR, UNDELETE and UNFORMAT. It looked like the blue text-mode Windows XP installer.

> I think the MS-DOS installer disk put files in C:\DOS by convention but that was just a convention.

Yes. For whatever reason my father used C:\SYS and I inherited it, along with C:\WIN for Windows.

It was mostly the latter. And when Windows broke, people would blame it on Microsoft, not on the software they installed. The same if the software broke. And you didn’t have online updates at the time that could retroactively add fixes. So Microsoft had to do everything they could to ensure broken software would still work, while also keeping Windows working, the best they could.

> So Microsoft had to do everything they could to ensure broken software would still work

I think they chose to do everything they could to keep it limping along. An alternative would've been a name-and-shame approach, like "This program crashed because the author made this mistake: [short description or code or whatever]", and leave them out to try until the devs stopped doing those dumb things. After a few years of pain, people would've gotten with the program, so to speak. Instead, they chose the path that put zero pressure on devs to write correctly-behaving software.

> After a few years of pain, people would've gotten with the program, so to speak.

Not necessarily. This was still very much the time in which choosing to stick with an old version which worked (e.g. Windows 3.1) wasn't uncommon.

Just look at how many people jumped from XP to 7 due to the network effect of "Vista sucks" and then multiply that by the fact that, at the time of 3.1->95, people had far fewer computer security concerns, if any.

Yes, but that doesn't solve the customer's problem

And what does the customer do if the vendor has discontinued it? Or charges for an upgrade? Or has gone out of business?

https://devblogs.microsoft.com/oldnewthing/20031224-00/?p=41...

I'm pretty sure another one was "what if you're wrong/have a false positive detection, and slander another company, one with lawyers?"

> And what does the customer do if the vendor has discontinued it? Or charges for an upgrade? Or has gone out of business?

Those can all be filed under Not My Problem (as in, Microsoft's problem,) and safely ignored. On the other hand, when Highly Influential So-And-So upgrades from 3.1 to 95 or whatever, and Very Population Application v4.9.6 starts falling over, Microsoft gets the black eye whether they deserve it or not. The whole equation changes.

> Those can all be filed under Not My Problem (as in, Microsoft's problem,) and safely ignored.

It was absolutely Microsoft’s problem. Had it ignored compatibility issues, what incentives would users have had to buy Windows 95?

The thing is, Microsoft got its position of dominance exactly because they did that - and that was because by doing this, the users' programs kept working. Remember that users outnumber developers by far and the last thing Microsoft wanted was for people to not upgrade Windows because they broke their previously working programs.

This was even more important at a time when Microsoft had actual competition in the OS space and people weren't able to just go online and download updates.

> The thing is, Microsoft got its position of dominance exactly because they did that

Yeah, right. No bribes, no preinstalled software...

They dominated by ... accident.

Nobody said it was by accident. They had deals with PC manufacturers for decades where they sold licenses based on the number of PCs sold by the manufacturer regardless of if the PC used MSDOS or another OS (like DRDOS), making all other options more expensive unless the client asked for them.

But the important thing is that the clients pretty much never asked for other OSes because all of their software worked on MSDOS - and later Windows. People bought computers to run their software, so if the software they wanted to run needed MSDOS or Windows, they'd buy the machines that ran that OS.

And by extension, if the software they wanted to run wouldn't run on the next version of MSDOS or Windows, they wouldn't have a reason to upgrade MSDOS or Windows. But from a user's perspective MSDOS/Windows was the best choice because everything supported it.

Microsoft didn't rely on just backwards compatibility (especially since the idea of "backwards compatibility" relies on something to be compatible with in the first place) but it was an incredibly important aspect of their strategy.

Why would I buy a new version of Windows, if none of my existing software will work on it, so I have to buy new versions of everything? Sounds expensive.

But your computer will be secure and then pedophiles and terrorirst wouldn't stand a chance.

Raymond Chen already discussed this. Microsoft wants to sell Windows. Windows exists to run software. If Windows doesn't run software, Microsoft doesn't make that sale.

If your business runs on some obscure piece of software for which updates are neither cheap or easy, you're not going to buy Windows if it doesn't run that software.

Name and shame doesn't work because the developer isn't part of the transaction.

One workaround Microsoft has done for use-after-free is detecting when an application is prone to this and using an allocator that doesn't actually free RAM immediately. It believe that lovely bit of fun is a function of "Heap Quarantine".

Yes, the real, can't say no world of system software is not what one might wish.

IIRC Sim City 2000 is one such piece of software.

It was SimCity Classic.

Not too different to using MS's mimalloc to run zenlisp under OpenBSD because the core malloc will just tell good try, but GTFO to the interpreter.

The biggest cause for this problem isn't lack of docs, but poor OS design. Like, why would you let apps change anything without restrictions to begin with? Of course, then you have to have some dumb hidden folder wasting space to restore the changes, and this "waste space for no good reason because we can't architect properly" is still a common issue

You're not wrong, but largely as a result of dubious architectural decisions made in the name of backwards compatibility and minimal hardware requirements, Microsoft sold 40 million copies of Windows 95 in its first year, compared to 300,000 copies of Windows NT 3.1.

Consider:

Windows 95 ran the vast majority of MS-DOS and Windows 3.1 applications with minimal performance loss, supported MS-DOS and Windows 3.x drivers for hardware that lacked 32-bit driver support, and ran acceptably on a 386 with as little as 4 MB RAM.

The properly architected Windows NT 3.1, released two years before Windows 95, had limited MS-DOS and Windows 3.1 application support, required NT-specific drivers for all hardware, and required 12 MB RAM to boot, 16 MB to do anything useful, and you really wanted a 486 for decent performance.

Now try a 3rd comment that actually connects to the design deficiency described in the article instead of a generic grievance about rearchitectrue that included a gazillion of changes

I agree. Why should the person who bought a computer be allowed to own it? Phone ecosystems got this the right way - the company that made the device owns it, and the person who bought it does not!

Don't speak in empty slogans, connect it to the article/comment!

Some app does some thing, then the OS reverts it! Where is "you" and "own" in this process? Do you own the "C:\Windows\SYSBCKUP" folder? Do you own the undo process?

Would your "ownership" rights increase if instead the OS didn't waste any space, but simply blocked downgrades of system components without user warning/intervention? Or had an even better process?

They could have had file permissions. They could have had a package manager instead of third-party installers.

Also note that Microsoft Office has a long history of not following Windows rules. Microsoft didn't even set a good example.

In the era of Windows 95, having a network connection was still a rarity. Expecting modern systems to have package management and sandboxing mechanisms would have been 20 years ahead of their time.

They had. Sort of. MS-DOS was a single user system. See attrib for details.

>back in those days.

> You see all the wacky software that doesn't follow the rules properly, does whatever it wants, breaks things.

Just like today. Software is hard, software engineering even harder.

Reply on news.ycombinator.com