I think I found a Mac kernel bug (2018)
jvns.ca
jvns.ca
I distilled the problem in a repo so it can be reproduced with a single command: https://github.com/alin23/m1-panic
I found it while on Monterey and reported it 2 times through Feedback Assistant, but it still happens on Ventura.
NOTE: Don't try it without saving all your work, it has a very high chance of restarting your computer forcefully.
Maybe tracing the CPU instructions using LLDB might be possible, but the bug is most likely in the kernel code so this would not help much.
If that still fails, virtualization tools provide debugging interfaces you can use to step the execution of the virtualized CPU; e.g. VMware’s “debugStub” feature.
> I found it while on Monterey and reported it 2 times through Feedback Assistant, but it still happens on Ventura.
I have no idea of they work on the arm Macs, but I will have the ability to check in a couple of days. Probably nothing exploitable, but still a hard crash.
https://github.com/apple-oss-distributions/xnu/blob/5c2921b0...
https://github.com/apple-oss-distributions/xnu/blob/5c2921b0...
This uses the macro SLIST_REMOVE (https://github.com/apple-oss-distributions/xnu/blob/5c2921b0...) to remove an item from the linked list. If you look at the code to do this it's a pretty simple linked list traversal: https://github.com/apple-oss-distributions/xnu/blob/5c2921b0.... However, it doesn't have a check for the end of the list, so the item must be in the list, or it's going to walk right off the end and dereference a null pointer. In this case that's exactly what it ended up doing.
I know you're well versed in reverse engineering Darwin, and I'm reading your posts and trying to improve my skills in this daily, but this seems way over my skillset.
Did you debug this using KDK or m1n1? Do you have a setup always ready for debugging a Darwin kernel?
In this case I do not have a KDK for that build. In fact Apple has been unable to produce one for a couple of months, a inadequacy which I have repeatedly emphasized to them because of how critical they are for stuff like this. Supposedly they are working on it. Whatever; in lieu of that I got to figure out how good the tooling for analyzing kernels is these days, which was my real goal anyways.
For this crash log I downloaded the IPSW file for your build, 22A400. All of them get linked on The iPhone Wiki, e.g. https://www.theiphonewiki.com/wiki/Firmware/Mac/13.x. Once you unpack the IPSW (it's a zip file) there are compressed kernelcache files inside. Apple changed the format of these this year so most of the tooling breaks on it, but https://github.com/blacktop/ipsw was able to decompress them. Then I loaded it in to Binary Ninja, which apparently doesn't support them either but compiling this person's plugin (+166 submodules, and a LLVM & Boost build) gets it to work: https://github.com/skr0x1c0/binja_kc.
From there you can load up the faulting address from the crash log and see what the function looks like. In this case, a bunch of junk has been inlined into it but there's a really obvious and fairly unique string reference for "invalid knote %p detach, filter: %d". From there, you can compare it against the actual source code to see which one matches the "shape" of the function you're looking at. I happened to also pull up an older kernel which did have a KDK available and then compared its assembly to the new one to match it up to ptsd_kqops_detach. The disassembly of the crashing code is obviously doing a linked list walk so you can figure out exactly which line it is from that.
If I wasn't lazy I might also fire up a debugger to see why the function had walked off the end of the list but without KDKs things get pretty bad, not that they're very good to begin with. I don't have a m1n1 setup (I should probably do this at some point) and the things I do have, like remote debugging or the VM GDB stub, are not really worth suffering through for a Hacker News comment.
I was in the process of trying to get I²C working through the built-in HDMI port of the Apple Silicon Macs (the one containing the MCDP29xx HDMI-to-DP converter chip) and been hitting a lot of dead ends while looking at kexts and opaque firmware blobs. This is going to help a lot as the KDK seems to contain logging messages related to DDC that I've never seen before.
I also found SIP disabled + Frida very useful for debugging without going through the KDK/m1n1 route. Not sure if it also helps with kext code though, I mostly used it for SkyLight and other private frameworks, but it's very nice to be able to also alter the code while it is running in realtime, or sometimes simply log specific function calls with argument value to get an idea what action causes which code to run.
I remember a decade or two ago I ran into a linux bug where the kernel would panic if a process was killed with an open descriptor on its /proc entries. That is:
open /proc/$pid/something; kill -9 $pid #kernel crash
We unfortunately discovered this when using fuser in a runscript to kill stale versions of a process, eg:
sudo fuser --kill --namespace tcp 80 # kill whatever is listening on port 80
This would reliably cause kernel panics every so often, with one straightforward shell command. This ended up causing a big problem because it was part of a runscript which ran on bootup. But, it normally would do nothing so it went unnoticed until the app in question had a startup problem, leaving a copy of itself dangling listening on the port -- and instead of killing the old instance, it began crashing the entire system in a loop. Oops.
That was a wild time. Nobody talked about security back then. The idea that everything in our lives would eventually run over the internet just wasn't on people's minds.
And it's very easy to cause ps to hang. Many different kernel syscalls hang / are blocking. Mostly you see this with kernel features dependent on a resource that doesn't resolve itself, like a stuck disk, network filesystem, etc. But other various quirks of the system can cause blocking.
The kernel should never let any user process consume so many resources as to cause a system freeze.
The kernel must not only be able to preempt any user process at any time, stopping it to consume all CPU time, but it must also prevent any user process to completely fill a SSD or HDD, because that can prevent many programs from starting.
Pretty much the only kernels that are totally preemptible are RTOS and they still don't stop you from shooting yourself in the foot.
A phone is different since it always has to be able to make phone calls.
The interesting thing here is that the described bug isn't just overloading the kernel with work or starving it of resources, it's something which seems completely innocent.
I think I found a Mac kernel bug? - https://news.ycombinator.com/item?id=16250677 - Jan 2018 (117 comments)
This was one call to kqueue with incorrect (but not particularly malicious, just normal C silliness) arguments, and boom!
Over the years, I have found numerous four different bugs in Apple's Calculator app. Here is today's wtf.
Switch calculator to Scientific mode (⌘-2). Type: 1, 0, ^, 2, 0, enter, command-c (to copy), command-v (to paste)
Expected result: an amount in $$$ I wouldn't mind having. Actual result: smallest number to appear 6 times in Pascal's triangle
I reported all four and they never acknowledged any of them. They didn't fix any of them either. Doesn't motivate anyone to report more bugs to them.
Some of the bugs got auto-closed after a while. Eventually, the bugs did get fixed, except the clipboard thing ¯\_(ツ)_/¯.
You mean exacty 6 times or at least 6 times?
The behavior is maybe a bit surprising, but I could also see it being defensible. You can't type "1e20" into the calculator, so why would you be able to paste it in?
Outside of plain text, I don't think clipboard operations are necessarily expected to be reversible.
That said, I tested again, and once again copying "1e20" and pasting it into the Calculator works just fine. It's definitely not treating it the same as pressing each key separately. I'm testing on macOS 13.0.1.