Asahi Linux goes from Apple Silicon port project to macOS bug hunters
theregister.com
theregister.com
You can often use technical solutions (e.g. JTAG) to fix "bricked" devices.
https://social.treehouse.systems/@marcan/111337509620995637
There are often arguments about what is "bricking" when these things happen, so here's my take (having dealt with embedded device ecosystems for a decade+): "Bricking" is when a device is put into a state that can only be recovered from by using specialized repair/recovery tools, opening up the case (for non user-serviceable devices), or software not legitimately available to the public. Apple Silicon devices are mostly "unbrickable" because you can always recover using DFU mode. In fact they are probably the most unbrickable consumer computing devices in existence, due to how thoroughly a DFU wipe restores everything (not just all software, but even device calibration and settings get downloaded from a server). DFU wipe is documented, relatively user-friendly, and requires only publicly available software and another Mac, which makes these unbootable states not a "brick". In contrast, most PCs are brickable: just wipe or corrupt the BIOS Flash. Most of the time this isn't super easy to do, but it's rarely fully protected and there have been many instances of something as simple as setting UEFI variables wrong bricking x86 machines. The exception here is x86 motherboards with a "BIOS FlashBack" type low-level recovery feature, which is as close as you get to DFU mode in the x86 world. Most Android devices are brickable too, and very easily at that. Just deleting/corrupting the wrong partition on disk will make your device unrecoverable. While in principle they have DFU-like recovery modes, the tools to use them are almost never made available to the public (you need vendor-specific tools, fastboot won't work) nor are they intended for use by end-users, which makes this qualify as a "brick". There is also no mechanism to recover calibration data like Apple has. There is, however, a tangential aspect: data loss. Any mention of bootability issues should qualify whether the fix makes you lose all your data or not. For example, on Apple Silicon, deleting the first partition on the disk is a very quick way to end up with an unbootable device where the fix requires a full wipe and losing all your data, even though it's not a "brick". For this reason, I would say Apple Silicon is much better than x86 at system recoverability, but is worse than x86 at data recoverability.
Soft bricked == here's how you can fix it.
Hard bricked == here's how we can fix it.
Also, soft brick is as oxymoron as dry water.
I've had a camera firmware update go wrong and not even Fuji was able (or willing) to fix it. That's a brick.
I'm just glad and grateful there's super smart people out there volunteering their time on worthwhile causes.
I mean, they're still super clever, but people learn that stuff by reading about it and practicing. You could go that way too, it's not black magic.
Then why do I have "black magic" on my LinkedIn profile???
I learned web dev first by using Geocities and Frontpage, then Dreamweaver. These days I actually get paid to write Javascript (something young me would've found unbelievable enough already), but to me that's still a huge difference from "real" programming, especially in C or assembly.
Web dev (at least the kind I do) is mostly still just declaring UI in a XML like syntax, then wrapping it up in some events and state management. It's not that different from, say, Visual Basic. At its core it's a bunch of event driven reactions and API calls based on UI clicks. Once upon time that was Perl or PHP or jQuery, now it's usually React, but the fundamental process of "declare UI, add event handlers, send to APIs" hasn't changed much IMO.
But an OS? Man, how do you work with memory access? Overflows? Garbage collection? Device drivers? Caching? Graphics pipelines? USB? Lightning? I don't even know where to start with any of that, and an OS is ALL of that and then some.
Mad props, is all I'm sayin'.
If you want to get a taste of what that means that is definitely 100% within your abilities, try to write a Brainfuck interpreter in JS, and a Brainfuck to JavaScript compiler (that yields a string that you can eval() to run the program).
Then try to write some simple Brainfuck programs, then try to write a Brainfuck interpreter in Brainfuck. At this point you won't yet know how to write an operating system, but you'll definitely know how it feels to write one.
Another time, I read a Brainfuck wikipedia entry for about 30 seconds.
I still haven't recovered.
I think I have pretty underwhelming dreams relative to OS devs, lol. I feel a ping of reward when a web page loads without crashing, or a npm audit passes without red, or I finish a level in Mini Motorways.
Generally, my secret to a happy life is to aim low, and then be satisfied once I hit half that :)
You start much simpler. Grab some ESP32 or ARM development board and start with trivial things on them. Simple devices also have simple drivers - sure, trying to connect to an NVIDIA card is hard, but an 128x128 LCD display is trivial. Then you build more complex things on top of that. Same with PHP - you start with 'echo "hello world"' rather than full reactive page with many server routing endpoints.
Web only started in the 90s, and was intentionally designed to be simple (html and stuff). Even so, it's morphed into a very complex thing already. It doesn't feel like so because you (and to some extent, I too) have been witnessing its evolution as it happens, so it's all incremental, but I can imagine a total newbie coming into the field and feeling overwhelmed.
Like, try to explain things like CSRF tokens, Web Assembly, HTTP/3...
It is black magic to people who read and don’t understand a thing. :|
Similar reason as to why companies have bug bounties; they want to incentivize hackers to report bugs through official channels early enough in development that they can get patched before release and before tech journalists write articles about them. They don't want to find out that their products have bugs via social media. Even if that process happens out in the open via a Github issue, getting giant problems like this caught before release and quickly escalated internally through official channels would go a long way towards mitigating article titles like this.
Having said that, does Apple care about the Register or HN? Probably not? And assuming that Apple did care about bad PR among extreme power-users, would Asahi Linux want to be paid by Apple to do testing on their releases? That's also not necessarily a given, the team would have to decide if they wanted to have a more official relationship with the company or not.
The issue has never been finding the bugs. It's having the capacity to fix the bugs.
And when you are this early in the OS release cycle (we are only at 14.1) there are inevitably a lot of P1 issues that limited resources are competing to fix.
Only 14/16-inch models where users have explicitly disable ProMotion and are desperate to boot into the RecoveryOS.
To ship this severity of bug in an X.6 release too is definitely very eyebrow raising.
I hope this was a joke :)
The worst thing that Apple did to themselves is force everything and everyone into a yearly release cycle
This is actually the best of both world, they stay independents, they get paid, Apple fixe the reported exploit.
Here's the clip where Asahi Lina says she received $150,000 for CVE-2022-32947:
- https://youtu.be/hDek2cp0dmI?t=11451
And Apple credited them under GPU Drivers for macOS Ventura 13:
- https://support.apple.com/en-us/HT213488#:~:text=CVE%2D2022%...
Related coverage:
- https://secry.me/explore/news/cve-2022-32947-first-critical-...
The results differ immensely, and neither approach are perfect.
A big part of the reason people are paid multiples of six figures is to put up with an unhealthy amount of stressful bullshit.
Which it seems they did, since Apple's reaction comes after the Asahi publication of the bug
There are better ways to communicate about a black screen bug on boot than leaving people with the fait accompli, one would naïvely think.
Companies exist to make money. That means they naturally deprioritize a lot.
Asahi is doing a pretty common pattern of the loud, young new comer. They tackle something that’s low hanging fruit. It’s typically a pretty small issue. Then make a lot of noise.
Everyone focuses on the small problem they’ve solved while ignoring everything else that still doesn’t work.
This sounds like a very weird edge case and it's very easy to say this sort of statement in retrospect, but I wonder what percentage of engineers would think of testing this specific combo beforehand.
As someone who works on embedded devices with a lot of experience with Hardware-in-the-loop testing, this is 100% a QA oversight.
The way this feature is implemented screams "test it from boot to reboot". I wouldn't pick too much on a small company doing some basic BSP/application work for some HMI device or something like that, but this is the richest company in the world with major funds/infrastructure to be better than this.
Do we have info about the implementation somewhere?
It's a single setting prior to a standard update. Now I'm not saying they should test every combination of settings, or even every individual setting. I'm more curious how there is any interplay between this setting and the update, and why that has any effect. On the surface it smells like a system design issue to me - like something is going in there on that shouldn't be.
There is (probably) a logic behind it.
https://mstdn.social/@marcan@treehouse.systems/1113296141101...
> Why [does disabling ProMotion cause this problem]? I can tell you why: because Apple hates display modeset flicker, and switching modes between ProMotion on/off causes a modeset flicker, so of course they made it so that is stored in nvram somewhere and applied when the screen is turned on during early boot, so when macOS boots it doesn't have to flicker again.
I personally appreciate how Apple prioritizes these little details—and I'd argue it's a huge part of what has made the company successful. This is the company that put a speaker in the iPod just to have a clicking sound for the scroll wheel.
For context, I hate it when the built-in display is not consistent with my external displays.
Few displays actually support ProMotion rates. So chances are that if you use any external display, you’ll have this jarring asymmetry. So you’ll want to set your rates manually.
Using an external display with your MBP doesn’t seem like a weird edge case to me.
On the other hand, Apple has problems with the M series chips and high refresh rate mode timings on external displays, so you're technically correct about the compatibility.
Even if you don't have FileVault (macOS disk encryption) enabled, data on SSD (or NAND chips with Apple silicon) is encrypted, so it's quite unlikely that you would be able to recover anything.
Not to mention that SSD/chips are soldered on the motherboard.
In the event of an OS crash/issue, or just minor corruption, that might be an option. Related to the original article -- I'm not sure what is meant by DFU mode on a MacBook (only have heard of that on iDevices) so it's unclear to me if you can EFI boot into another OS. Given that it also affects Asahi Linux, sounds like you can't EFI boot into anything, even OS Recovery for Mac Sharing mode.
https://support.apple.com/guide/apple-configurator-mac/reviv...
There's also not exactly a boot picker in firmware-- that lives in the recovery OS. Which will also be unbootable if you're impacted by this issue because it's a minimal macOS environment; if your only OS is 13.6/13.6.1, that recovery partition will also be 13.6/13.6.1, and, if you have upgraded to 14.0/14.1, that installer can fail to update the recovery partition and leave it at 13.x (whatever you were on when you upgraded).
That suggests its only goal is to frustrate right-to-repair and data recovery. Also as a way to sell their cloud backup solution.
"Sorry, we can't recover anything, but you should've used iCloud!"
Aren't there laws against these sorts of business practices?
The article seems to have been deleted[3] at some point since the link from marcan goes to a 404
[1] https://social.treehouse.systems/@marcan/111334488235016591
[2] https://mstdn.social/@osnews/111334720022394898
[3] https://www.osnews.com/story/137678/apples-macos-sonoma-make...
Paraphrasing here: “The current macOS (macos 11 at the time it was written) looks and feels like it was designed by someone who never used macOS and has only used an iPhone.”
I went to restart my machine (was plugged into a lightning hub with 3 monitors, I don't ever recall messing with refresh rates) and it went to update. At some point in the loading bar, the screen went black and then flashed an apple logo. When I pressed power, I could sometimes get the apple logo to display on boot, then it went dark. To me it looked like it wasn't booting at all but the machine is pretty quiet even when it's on so it was hard to tell what was happening.
We went to do the firmware revival process using another mac and it would barely recognize the device depending on what we tried. Eventually we were offered the option to do a full system restore, and after many hours battling this, we decided to just lose the data on the machine.
Luckily I keep most major stuff in cloud or in github, but I lost a significant amount of work from the previous 24 hours that ended up delaying a release and causing significant pain. For how much these machines cost, this is completely unacceptable. I know windows machines have their fair share of issues, but in 15 years of working on various windows and mac workstations this is the only time I've lost an entire drive.
The DFU key combination is finicky for portable machines: connect a charger (preferably Magsafe so you can watch the power LED) and your configuring system, press and hold power to be sure the system is off (if doing this turned the system on, repeat this step), press left ctrl+option and right shift at the same time as the power button, count ten seconds, let go of everything but power until the device shows up as "DFU" in Configurator (you may be prompted to allow more accessories to connect to your configuring system before it does).
If asked to perform a software update before/during reviving, choose "Quit and update" and start the process again. If you upgraded to 13.6 or 12.5 before facing these issues, you may have to enter recoveryOS instead of booting normally and perform a system upgrade to Sonoma.
If done correctly (without choosing Restore), you will not lose data. If you can't do these steps yourself or think you will have trouble walking a family member through them, the Apple Store can do a revive for you (be explicit that they are only to revive the machine, not restore or replace).
Full details at https://support.apple.com/guide/apple-configurator-mac/reviv...
And Sonoma appears to be riddled with odd little bugs, manifesting as UI failures. Windows don't come up, or they come up behind others when previously they didn't. For example, disk images, when mounted, open a window as usual but that window appears behind others on the desktop so you're left wondering what happened.
I just had keyboard input rejected with "boink" noises in a dialog raised by Safari, over and over. In the end I couldn't log into the site.
The "About this Mac" item in the Apple menu did nothing, over and over...
and sure enough, I just tried this, and it happened again. But when I started Screenshot to capture it and file a bug, it started working. This is the second day in a row this has happened.
It's looking like a shitty Mac OS release.
And yes, I have been using Macs for 16 years and have never had a catastrophic issue and neither one of my friends of family. The worst problem was kinda self-inflicted, pre-SIP I was once typing something along the lines onf sudo rm -rf /Library and then accidentally pressed enter. Yay for backups.
No this happens every Mac OS release for the last 20+ years.
Sonoma has literally just been released and we only recently had the first major patch. So there will be bugs both known and unknown. Almost all by definition not fixed.
As always if you are doing mission critical work then for any piece of software you should always hold off upgrading until at least a few major patches have been released.
If it could be restored by wiping the SSD, it’s not bricking. It’s terrible, for sure, but bricking means your device turns into one and you might as well throw it away. If it’s recoverable via software, it’s not bricked.
There is no scientific definition of "bricking", and I agree this is a bit of a gray area, but I think it's fair to use the term.
I think that's my last bug report, as none of my feedback/bug reports, which are very detailed, and some cause data loss/corruption.
0 feedback, 0 fixes. Total waste of time.
I wonder where their software quality and consistency team went to. If you'd release similar buggy software when the AppStore on iPhone became available, your app would simply not pass the quality criteria.
Debian looks awfully nice these days, especially combined with proxmox / passthrough devices
m1 16" MBP here, this happened to me on the last OS version too. Still happens on Sonoma.
For about 1 year (a few years back) I had an issue where on my three external Samsung monitors (all in portrait orientation) there was a strip about 4cm all across the bottom of all of them, and up the right side of the right monitor where the mouse wouldn't go. Apple couldn't work it out, spent hours talking to them about it. In the end an OS update fixed it.
This is why I hate the loss of HDD and other activity lights. It's unfortunately not specific to Apple, but they're a leading offender of this "opaqueness" trend.
I admittedly have little Mac experience but it sounds like you weren't able to boot from a USB and copy the files you needed off the drive? That's the usual approach when something like this happens with a PC. I've even done a bit of Hackintoshing and you can certainly boot the macOS installer and use the terminal there. There's currently a gray comment at the bottom of the discussion here lamenting the "smartphonization" of computing, and this is a great example of that. You're expected --- or rather, forced to be --- helpless and treat the machine (or "device") as disposable when something goes wrong, and the same goes for your data; unless you give up some ownership by letting some thirdparty store it for you, that is. (Then again, I own an Android where I can backup and restore the whole eMMC, so it's not a complete characterisation.)
Relevant Douglas Adams quote: "The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong it usually turns out to be impossible to get at and repair."
There is a good reason UEFI is stored on a QSPI flash.
On a PC, the situation is different. At worst, with bad GPU drivers, you can still fallback to good old 640x480 VGA, or a slightly better generic unaccelerated VESA framebuffer, and be able to troubleshoot from there.
I'm undermining my own counterargument... On a PC, a bad BIOS flash is recoverable. Some have two BIOSes - like on video cards. Some have a recovery mode via a USB stick. And attaching a programmer to the SPI flash is also a viable recovery method...
Maybe he doesn't think any of the mac firmware environment is really that different from a general purpose computer?
Ok, I don't know what the counterarguments are. ¯\_(ツ)_/¯
The x86 side is fine: just don't use MS operating systems.