I wonder what other fallout we might experience from this.
I wonder what other fallout we might experience from this.
/* Given a path to a DER-encoded CRL file and a path to a PEM-encoded
* CA issuers file, use OpenSSL to validate the CRL. This is a hack,
* necessitated by performance issues with inserting extremely large
* numbers of CRL entries into a CSSM DB (see <rdar://8934440>).
http://opensource.apple.com/source/security_ocspd/security_o...ocspd was introduced with 10.4. A decade ago. And that's really the problem with macOS: There's no refactoring of old hacks, but rather just bolting on of ever more new stuff.
http://blogs.marketwatch.com/thetell/2016/10/25/apple-earnin...
We may someday get an XNU rewrite, but probably not until software engineering produces a new approach to building complex systems reliably that works at the scale (here: number of developers and shipping schedule) Apple needs.
It's conceivable to perform a gradual transition away, though. They could demote Mach to a fast IPC system that just augments BSD, similar to the way the kdbus/bus1 proposal for Linux does. That would be difficult and a long-term project, but it would fix the underlying issue in a way that mostly retains userspace compatibility. Driver compatibility would be more difficult, of course…
My prediction is that we don't ever see an XNU refactor/ redesign/ rewrite so long as C/C++ is the kernel implementation language.
I don't think SeL4 means much for macOS/iOS.
Also, seL4 (being a microkernel) leaves out a huge swath of kernel-level facilities that need to be implemented with the same standard of verification (resource management, network stack, drivers, etc.). Running on a verified microkernel provides a great foundation, but these still add a ton of code that needs to be verified. Plus the concurrency problem will strike again at this level.
If you have just a few extremely simple applications you'd like to run in an enclave, L4 is a good way to minimize the surface area between the applications themselves and the hardware.
If you'd like to host a complicated operating system on the simplest possible hosting layer: again, L4 is your huckleberry.
Otherwise: not so useful.
Note that if you just host XNU on top of L4, you might rule out a very small class of bugs, but the overwhelming majority of XNU bugs are contained entirely in the XNU layer itself; having XNU running on an adaptor layer doesn't do much to secure it.
More often than not, systems that get boldly rewritten from scratch end up playing catch-up for years. Frankly, I can't remember a single case when a full rewrite with an ambitious timetable wasn't a full-scale disaster. The few success stories, like (what eventually became) Firefox have taken a vastly different approach and took a lot more than users would have wanted.
A lot of idealistic (I was about to write naive) engineers think it's all a matter of throwing everything away. That's the easy part. Coming up with something better is the really hard part, and it's not achieved by just throwing the cruft away. If you innocent souls don't believe me, come on over to the Linux side, we have Gnome 3 cookies. You'll swear you're never going to touch anything that isn't xterm or macOS again.
Of those, the last one is very impressive: it's a decade-long from-scratch project that has succeeded in competing with a very entrenched project (GCC) in a mature market.
Regarding GNOME 3, the delta between GNOME 2 and GNOME 3 is far less than the delta between NeXTSTEP+FreeBSD and the first version of Mac OS X.
This whole premise is a false dichotomy. Apple does not have to throw away Mac OS X, and it does not have to keep piling crap on without fixing things. If you stop the excuses and rationalizations and commit to code quality you can ship an operating system with quality code and minimal bugs. The OpenBSD project has been doing this for two decades with minimal resources. There is no valid excuse other than "we are too lazy and incompetent."
Oh too much code, bad code, we inherited it, throwing out away won't work etc are baloney excuses without meat. All it takes is the will to hire and commit the right resources with an objective of increasing code quality. I mean take this bug itself - Apple did fix it but only after GPZ was on their arse. No reason they couldn't have reviewed it themselves and fixed it.
> Apple needs to take a bit of those tens of billions of dollars they have sitting around and spend it on starting from scratch with something that's not horrifically crufty.
They certainly don't have to throw everything away. Not having thrown everything away is one of the reasons why OpenBSD is a good example here. Remember all that quality code that was in place before Cranor's UVM? (Edit: actually, the fact that UVM is an improvement over it should say something, too...)
And, at the risk of sounding bitter, in my experience, very few companies have the capability to "commit to code quality", and I don't think Apple is one of them.
Edit: BTW, I really like your blog. You should write more often :-).
So much before my time I was not even aware of it. For the uninitiated: https://www.usenix.org/legacy/events/usenix99/full_papers/cr...
> Edit: BTW, I really like your blog. You should write more often :-).
Thank you. :) Just this week I started thinking of getting back into it.
Also currently running on a nice pure wayland system, no need for that X11 cruft
1. Apple inherited OSX from NeXT, and with it the Mach subsystem. Mach overcomplicates XNU.
2. XNU has become incredibly popular by dint of being shipped in the iPhone. Avie Tevanian probably did not see that coming when they designed the original BSD/Cocoa/XNU/whatever architecture. Regardless: it is now difficult to make sweeping architectural changes in XNU, because of the enormous installed base.
3. Ian Beer is simultaneously very clever and also willing to wade into the XNU Mach fire swamp.
I think it's fair to criticize Apple for designing the XNU frankenkernel. I think it's less legit to say that the presence of this bug class "looks bad" --- it's 2016 and this is just getting published. This is one of those bug classes that is sort of obvious in retrospect, and you wonder why people didn't catch it earlier.
That's not very justifiable no matter how many storms and twisters you throw in the mix ;) I am not going to argue but for OS development this looks bad that it wasn't even looked at much rather fixed for so long given many of Apple's kexts had the same issue.
[Also this isn't an isolated problem by the way - look up the thread and you'll find two other egregious errors including one involving execve that was pointed to Apple in vain. Not long ago I could just remove and insert a kmod and that would lock up the system - go ahead and argue that's not a hot path or nobody does that but it does speak to general lack of code and testing quality. To their credit Apple fixed that particular one though when I reported them.]
I disagree that this is a simple bug that anyone should have been able to find. If you'd like to put money on whether or not this is going to win a 2017 Pwnie Award, I would be happy to take your money.
Read the TLDR - exploitation of dangling pointers aside you can't write code and/or design APIs that does or makes it easy to do what the TLDR says - hold or use a task struct pointer and expect the euid of that task to stay the same. Many many places in the kernel do this and there are a great many very exploitable bugs as a result.
Ian Beer certainly is talented but that doesn't excuse Apple being so sloppy!
It's not a, pardon me for the expression, fucking XNU specific programming rule - it is a general rule that was invented long before Ian got to it! You don't hold a reference counted pointer and operate on it without taking a :gasp: reference first - having that shit in your sample code is just, well extra shitty!
Also, separately from the dangling pointer issue, the first sentence of the post is literally - This post discusses a design issue at the core of the XNU kernel!
The larger problem was an inherent TOCTTOU bug in the interface semantics between the BSD subsystem and Mach. AFAIU that wasn't a dangling-anything problem; the reference was still valid. It was a logic and design problem that could happen in any language, even in Rust, and even without resorting to unsafe code.
I've never really thought of process spawning as a hot-path (in the hundreds+ of calls per second sense). What software so heavily relies on spawning so many processes so quickly that the overhead of malloc would be noticeable?
Also my bigger point though was not that it was in hot code path (although I would prefer to not be in a position where I need to change execve to add two allocs and copies if I can avoid it) - it was that they had to change execve() this late in the OS's development cycle to fix this long standing bug - typically you get to a point where you don't really need to touch core OS code and when you do you risk adding new issues to a central piece of code.
Oops!