Hacker News Reader

Comment by glhaynes | Hacker News Reader

They may have even had a test case that entered "root" and no password and hit OK once, but that wouldn't have caught it. What if you'd had to hit "OK" forty-one times to trigger the bug? It's deeply unsatisfying to just say "the problem space is so vast that it's hard to even know what to try beyond the basics that every QA person knows" but I don't personally have a better answer than that. Maybe those that study the space do, though?

View on HN

Where are all the unit/integration tests for the APIs that this damned button is calling? Hindsight being 20/20, but I cannot imagine not asserting that a newly created/re-enabled root user has a non-empty password.

I don't think empty password on root is the issue here; enabling root when the user did not intend to and did not have permission to is. Isn't empty password for root a default state for systems where direct root login is not enabled? And isn't enabling root with no password the intended behavior when booting into Single User Mode?

Exactly, testing the UI may be hard, but whatever API that UI is calling is amazingly horribly broken.

And fascinatingly wasn’t broken until very recently...

Following and furthering your logic, what the hell could they have been doing in the codebase to revert a control mechanism that was effective up to and including 10.12.6, but unsafe as of 10.13.0 onwards???

A graceful upgrade mechanism towards a new password hashing algorithm.

Change is so, so risky.

There's a lot that can go wrong, which is why writing testable code and then /actually/ testing it, matters so much.

The code was not supposed to enable the root user in the first place, so I'd be surprised if someone added an assertion that anything about the root user's account was set. It was only supposed to check the root user's password.

I suppose you could write an assertion that the code didn't enable the root user, but I'm pretty sure that no password-validation routine anywhere in the history of the world has ever had a test case to make sure it didn't modify the account while validating it.

These are unknown unknowns. If you knew enough to write the right test, you wouldn't have written the bug in the first place.

But if the 2nd attempt does something different, there must be some sort of logic happening, and whoever built this magic (maybe it's an if-block like: (if loginAttempts++ > 0) ... ) should've tested it.

We cam see that the function f(a, b) { return a+b; } should return the same value for the same a and b, so testing that once is enough. But if there's something in there that reads a global variable, maybe check who changes that global variable, and what happens with different values of that global variable?

The "logic" in this case is the first attempt failed because the root user was disabled, but as a side-effect it enabled the root user. The second attempt worked because the root user was now enabled. There's no explicit logic in here that says the second attempt should have behaved differently.

Oh it's never that easy. It happens 20 layers deep, with configuration set a certain way... only every Friday the 13th.

I.e. MacOS is based on UNIX/BSD, root is all powerfull. So root is disabled form login with <star> password, but it's still there. Then there is the Apple autentication framework on top of that, here root is disabled. But if you try to log in with root, it gets enabled but can't login because password is still <star> an disabled. Then somebody decides new password hashes are safer and we should update old hashes whenever possible. So now when a disabled root account gets enabled, the update routine mishandles <star> password and updates the root password hash with <empty string>.

Fuzzing inputs

Reply on news.ycombinator.com