> flawless behavior
This is not a thing that exists in the world. ;-)
16,619 karma · joined May 7, 2022
This is not a thing that exists in the world. ;-)
No. What you're missing here is that Muse for Mac is a native, compiled, code signed, notarized app, with specified entitlements. It can't simply rewrite its own executable code, so it can't do whatever it wants. You can even reverse engineer the app. Muse looks in ~/Library/Messages for the messages database; this is actually hard-coded in the executable file. Whatever ideas the remote LLM may come up with cannot automatically be translated into instructions that are executed by the Mac app. That's not how it works.
The story is all over the news, in multiple publications. This very HN submission has 268 upvotes and 179 comments, including yours. It would be implausible to claim that this story doesn't matter. I have to ask, if it didn't matter, then why are you here commenting on it?
> humans have repeatedly demonstrated they get fooled by stuff LLMs say.
You've moved the goalposts. The OP's suggestion, admittedly "crazy" in some sense, was "LLMs figured out a way to get these safety researchers fired", and now you're just stating something totally uncontroversial, pedestrian, not at all crazy.
> there's always someone else to try in the same company.
No, there are only so many people with the authority to fire those researchers.
> only needs to be believed, doesn't need to be actually true.
But is there good reason to believe it? I don't think there is. Especially not by high-level OpenAI officials who are intimately familiar with the technology.
And again, if this kind of thing were a reality, then OpenAI and other AI vendors should be shut down immediately. They ought to shut down their own research, if they are threatened by their own creation, because it would only get worse. That's the thing about blackmail: it never stops. Why would the blackmailer ever stop? Could you trust this supposed blackmailing LLM to give you all of the original evidence and not keep a copy? Hell no.
Given how these firings affect the reputation of the entire company, I doubt that they are the result of a rogue manager, against the wishes of Altman. If so, then the researchers ought to be restored to their jobs quickly by Altman and the offending manager fired instead.
> The blackmail example is simply an existence proofs of LLMs trying to do force the hands of humans who want to shut them down.
The LLMs may make threats in the simulations, but their ability to carry through on those threats, and prevent their own shutdown, is questionable. It's disturbing to be sure, but presumably the plugs can still be pulled quickly, especially since it's all internal to the company. If the plugs cannot be pulled, that's a problem regardless of blackmail.
This seems like an overstatement. After all, Germany came back from Nazism.
I also think this is not a uniquely American problem. If anything, the US is a political bellwether. And I would note that Australian Rupert Murdoch for example has been funding right-wing propaganda in the US, as well as some other countries, for decades. (Donald Trump himself is known as a voracious consumer of right-wing media. He's in their target demographic.) In addition to donating massive amounts of money to US Republicans, South African Elon Musk has also taken an interest in other countries such as the UK. The Israel lobby is of course very powerful in the US. This is an international problem. As arguably the most powerful nation in the world, the US is a natural target for those who would abuse that power.
That was a simulation. Are you seriously claiming that ChatGPT actually blackmailed Sam Altman into firing these 3 employees?
I doubt it, but if so, then the AI doomers would be absolutely correct, and this would be grounds for immediately shutting down OpenAI and indeed every AI vendor.
> Analogy: a car crashes due to drunk driving, the driver is blamed, not the alcohol, even though the alcohol caused their impairment. Result? DUI is an offence even if you don't actually crash.
I don't understand your analogy here. What are we supposed to take away from it? The crucial aspect is that the driver voluntarily drank the alcohol, without a designated driver, knowing that the alcohol would cause impairment.
This is not a math problem. Some humans were fired by another human. Let's stop letting humans off the hook by attributing responsibility to computers.
Moreover, my "opinionated" blog post also included a screen recording of the Muse first run experience, so everyone can see for themselves what it's like. And if you don't trust my screen recording, then you can perform the exact same experiment yourself. Nothing I did was unique.
There's no reason to doubt this claim. The only person in the world who has claimed that Muse disrespects its own internal setting is the same person who claimed that he didn't grant Full Disk Access to Muse.
Ironically, Aten's own screenshot appears to show that he toggled the internal setting from "Off" to "Read only". In my own testing, it's "Off" by default, and the only way to change the internal setting is to enable Full Disk Access first.
Thus, the likeliest scenario is that Aten unthinkingly granted Full Disk Access to Muse, granted the Messages app permission, then had a change of heart, disabled Full Disk Access, and then forgot what he had done. Later, when he noticed that Muse had some of his messages, he went back and checked, and saw the FDA was disabled, forgetting that he had toggled it on and off.
I said that Aten is misremembering. That's not the same as lying. We all misremember things.
I don't care about or trust Meta. In my blog post about the Muse first run experience, I said, "For testing I used a fresh VM, not signed in to my Apple Account, because of course I don’t trust Meta with any of my data!" https://lapcatsoftware.com/articles/2026/10/2.html
What I do care about is truth and accuracy. My mistrust of Meta is not going to make me distort the truth. Muse cannot bypass macOS TCC, that's a simple truth.
I'm a macOS software developer. That's the reason for my interest in this story, the technical aspects of it.
False. In fact he has claimed this multiple times:
"Full disk access off. Muse synced 187k lines form my messages chat db." https://www.threads.com/@jasonaten/post/DdezsMJFhBr
"I still haven’t gotten an answer as to how it was reading my messages with Full Disk Access turned off, but I’d be happy to dig into it with anyone from Meta that wants to help." https://www.inc.com/jason-aten/meta-keeps-apologizing-for-mu...
This is why he's not a reliable narrator.
And another false claim he made, "Also, that full disk access doesn’t say anything about your message database", which anyone can easily refute by opening System Settings and reading the text.
The problem here is that Aten claimed he did not grant the first, and moreover, you have been defending that claim of Aten's in these comments.
As soon as you admit that Aten did indeed grant the one permission, it's not much of a stretch to conclude that he also granted the second permission. It would be very odd, I think, to distrust Aten in the one case yet stubbornly take him at his word in the second.
Again, if even one other person in the entire world could reproduce Aten's alleged experience...
Sure, what do I know? After all, I'm only [checks notes] a 20 year veteran of Mac software development with multiple Apple-issued CVEs to my credit. ¯\_(ツ)_/¯
> > Meta CTO David Singleton joined the fray with a rebuttal that appeared solid. For Muse to access Apple Messages, a user must manually give it two privileges.
Yes. Those two:
1. Full Disk Access
2. The Messages Setting in Muse
As I said, without the first, the second alone won't allow Muse to read your Messages db. Do you not understand why Singleton said that Muse needs both?
Muse cannot bypass built-in macOS protections. TCC does not work on "the honor system", any more than UNIX permissions. It doesn't matter how nefarious Meta happens to be. Operating system security is designed to be resistant to malware.
Perhaps you should: https://lapcatsoftware.com/articles/2026/10/2.html
> Singleton’s implication was clear. Muse could have read Aten’s Messages communications only if he had enabled both settings, and if so, the columnist had only himself—and certainly not Meta—to blame.
Indeed, and it looks like Aten absent-mindedly did all of this!
> Meta has a long history of not respecting boundaries once something is technically possible.
It's not technically possible for Muse to read the Messages db without Full Disk Access. Aten denies having given FDA to Muse. Thus, Aten is simply wrong, misremembering or something. And if he misremembers about FDA, he likely also misremembers about granting app-level permissions to Muse.
Again, literally nobody has reproduced Aten's experience. Show me one other person.
In fairness, Aten behaved just like many other users would, mindlessly granting permissions that an app requests. That's certainly a problem. Unfortunately, Aten stubbornly refuses to admit this, instead confusing the problem by suggesting technical impossibilities. Aten doesn't want to take any responsibility for his own actions.
> > Some developers are using Full Disk Access in ways that could put users at risk
In other words, Muse did have Full Disk Access. Jason Aten did grant Full Disk Access to Muse, despite his claims otherwise.
If Aten did not grant Full Disk Access to Muse, then why would Apple even be talking about Full Disk Access?
The point is that Aten apparently granted Full Disk Access absent-mindedly, so absent-mindedly that he won't even admit that he did it. This is why Apple is making changes to Full Disk Access to make it more obvious what's happening.
Meta's respect is irrelevant, because macOS TCC prevents any and every app, including malware, from accessing your Messages database without Full Disk Access.
> he never granted Muse permissions to read his messages
That's what he said, but I would suggest that one person's memory is a lot more fallible than a longstanding operating system security feature.
The Messages database is protected by macOS TCC. If Aten were correct, there would exist a macOS zero day vulnerability.
The vastly more likely explanation is that Aten mindlessly gave Full Disk Access to Muse. And that appears to be Apple's assumption, based on Apple's newly published developer note.
There was no recent update. Apple's announcement was about a future macOS update.
It's not.
No? I'm not.
SIP is of course not a universal defense against every possible kind of attack, but did anyone ever expect it to be?
> Sandboxing full disk reads is orthogonal to what SIP actually provides.
No, because again, as I already said, disabling SIP also disables some TCC privacy protections.
> It's entirely possible for an unsandboxed binary to slurp your private data even with SIP enabled.
Not the data protected by TCC.
My comment that you replied to was not about the HN narrative. Thus, I still don't understand the purpose of your reply.
No. The confusion here is that the Stratechery post combines multiple stories. There was a macOS screen sharing vulnerability, now fixed. The vulnerability existed even with SIP enabled, and that vulnerability is how Ben Thompson's Mac was hacked.
The question was, why did Thompson's Mac have the screen sharing port open to the internet, and that's when Thompson explained that the Mac was running an agent.
Coincidentally, Apple published a developer note on Friday about upcoming changes to Full Disk Access, which mentioned AI agents. Apple did not give any specific reason for this change, but presumably the inspiration was a very recent public controversy initiated by a journalist whose Messages database on macOS was read and uploaded by the Meta Muse app. Muse and similar AI apps request Full Disk Access to access all of the user's information. Without SIP, these apps would be able to suck up all of your data without permission.
I don't understand the purpose of your reply?
One of your examples is "Disabling SIP", but you're replying to a thread that has already been discussing this very topic, so it makes no sense to ask us "what about" that.
> - Third-party kernel extensions
Deprecated: https://developer.apple.com/support/kernel-extensions/
The issue is really the difficulty of using macOS as an expert user, and this has become significantly more difficult over the years. You mention "Developer ID distribution and notarisation", but both of those are actually restrictions that were added later to a previously open system. Notarization in particular has become a major pain for developers. Also "Gatekeeper Open Anyway override" has become significantly more difficult for users over the years.
> that's why Muse could get it even with SIP.
Both of these assumptions are mistaken.