Apple ML Research: Muse Spark Open with MLX on Mac Studio | Hacker News Reader

Apple ML Research: Muse Spark Open with MLX on Mac Studio

machinelearning.apple.com

365 pointsantipaul213 comments

So, Meta will spend millions/billions on building open-weight models and then people will send Apple $20,000 to run them. Seems like a good deal for Apple.

Exactly why does Apple need to do any model training/development? They are out here selling the shovels.

Apple’s Photos ecosystem is privacy oriented and as a result some (now) baseline features require the use of local models.

The Foundation models keep a surprisingly decent set of data available for fast filtering by a local LLM.

I’ve found a slower, second pass, is needed by Qwen to clean it up. (I am running on the new Mac Mini m5 pro 64gb.)

I’ve run my own tests using a date range, a short description of a red cedar planter box build I did last year, and a set of known correct photos.

There is some cool stuff you can do without sending your family’s pics up to any of the frontier model providers.

I've found some strange guardrails with the Foundation models. Like blocking describing of any photo with a single woman in it. It does this relatively consistently but with a different subset every time. Like out of 10 photos in the set, it's skips 6 then the next run, it skips 3. Have you seen guardrail issues like this?

I switched to Gemma 4 instead.

same i am using new embeddinggemma. seems far more capable than foundation models.

I am going to assume that means "a women by herself" and not that it can magically detect her marital status before refusing to continue (I suppose it could be laser-focused on wedding rings...)

Haha, that would be an interesting LLM fine tuning experiment.

I don't want any of these features and yet they've been forced upon me and are now getting in the way of me actually using my computer how I want. Games and heavy applications slow down and stutter because of "AI" background processes, even with App Tamer, my machine runs too hot to even touch when just idling. Compared to when I got my M2 in '23 and thought, "This is the fastest and snappiest machine I've ever used", my experience with an M5 and the latest macOS version is nothing short of an anti-consumer hellscape.

Interesting, this has not been my experience at all. I have an M5 machine and I'm not even aware of what AI features have been added, and it's streets ahead of my old M1 Pro in speed.

peep the comment I just made to the adjacent commenter, that has been my experience. I basically have to run my fans at full speed all day while working now in order to type on my computer (when I'm not using my magic keyboard). It is like a huge problem, not even just a small one but like something that is affecting me negatively every single day, increasingly making me want to break this machine in half and disappear into the woods.

The M1 used to be adequate in speed. Then Apple kept adding always-on cloud daemons that run even if you don’t use iCloud.

My Macs run cool and silently. I've never observed a background "AI" process consuming resources.

It's not just AI processes. But things like mediaanalysisd, duetexpertd, etc. even the freaking menu bar agent, all regularly pound my computer both when idling and when in use.

I recently installed App Tamer which is helping a bit, but it doesn't allow me to tame everything. I just checked now and duetexpertd was trying to use nearly two full cores. I have all iCloud integrations turned off, I do not use iCloud at all, and I do not have an iPhone, so I am flabbergasted that this iCloud desktop-phone syncing process runs at such high core usage in the background with extreme regularity. Proof: https://imgur.com/VkBh9bi

Unfortunately, this does nothing for services such as WindowServer which now often takes up 50-100% of a core just on its own.

I have reinstalled and upgraded to the latest version. This MacBook is only a few months old.

While typing this message and literally just having open Codex with one running task that is not doing anything substantial, and Firefox, with no high CPU usage from any tab, my laptop got so hot that I had to use another installed program, Stats, to manually bump up the fan since macOS also has absolutely garbage native system fan controls.

I have fibromyalgia and seem to be sensitive to hot keys; my Thinkpad regularly getting to hot to touch with even light usage is EXACTLY why I went and bought a MacBook one morning in '23. One morning I touched my keyboard and realized I was tired of singeing the nerves on my fingertips and having them hurt for hours. So now in '26, finding myself in the same exact position but with a much more expensive computer, I feel like I am living in a corporate hellscape nightmare.

One thing I have noticed is all of my cores can sit idle except a single one and that will still heat up the machine a lot. If the hardware is designed as such, the software MUST be designed to work around this and not blast a core for 10 minutes straight every time one of a dozen or two misbehaving system processes wants to complete a background task. If the software is not designed for this, the hardware is incomplete and faulty. This would be true if it was shipped as firmware; software doesn't get a free pass here.

But now Pages and Numbers recommend document titles for me, so.... worth it?

this is not my experience at all. I wonder if one of your tweaks is causing the software to get into a bad state. Likely worth reporting to Apple even if it feels like it's going into a void.

Which tweaks? This is stock macOS 27 on a stock machine, with of course my development stack on top. I have already tried reinstalling the OS. Besides that, everything else has been configured appropriately in system settings and profiles using Ansible, so that I know exactly what configuration state my machine is in. There's nothing weird I am doing to the system processes other than now using App Tamer to prevent the processes from continuing to use multiple cores while I'm trying to use my machine.

I do feel like my complaints go to the void, and honestly I have seen the kind of responses people get on the Apple support forums and it gives me the same ick that Microsoft's does. And lately I'm just going to get someone's canned, patronizing LLM-generated resume-building response. My only hope out of this mess is Gravity Linux catching up and adding full SEP and thunderbolt support. Once that happens, I'm gone.

I do wonder about the prevalence of my experience vs yours. I can quickly find hundreds of threads across various platforms which confirm my experiences. From what I have been able to ascertain, my experience is increasingly common if not more common than yours. Hard to tell.

You said you were running App Tamer, and I presumed it was doing something unusual, however looking into it, it appears to be using public APIs, so it shouldn't be causing issues.

There's a lot of Macs out there. I would be surprised if that's the common experience for users, however, I fully admit that my sample size is small.

And I totally hear you on the frustration for reports these days. I set my expectations to be that it's a tally mark in some database somewhere with the hope that enough tally marks means a human looks at it, but it's so disheartening when you've spent such a large amount of money on something.

Well this is a software issue, all of the Macs out there run the same operating system, the same software. This can't an issue of a defective Mac. I also have an M2 which is giving me the same issues.

I am right now using my M5, on low power mode on a flat surface and it's almost too hot to type on. I have Codex open, just idling, Firefox and Logseq.

All of the offending processes at the top of activity manager are currently system processes: duetexpertd, mediaanalysisd, MenuBarAgent, WindowServer, Spotlight Engine, Time Machine, etc. A lot of these deal with caches, files, indexes. I suspect the root problem is a shared internal philosophy which regards the owner of the machine as a second-class citizen to the system.

It is bewildering given that I do not want the majority of "features" this tangled mess is providing. Issues such as bad caches, misbehaving files or directories, etc. are likely not the culprit since I experience this on fresh OS installs across two MacBooks. In the sense that if it is those issues, it's a system issue which needs addressing, and not PEBCAK. I have Siri disabled, Time Machine off, iCloud off, all iPhone integrations I can adjust are off, etc. Yet these processes increasingly dominate my computer.

I do appreciate you engaging with me on this and I hope I haven't come across as confrontational.

Have you tried not fucking about with all the processes? All my Mac’s run fine and I haven’t fucked with their settings. You’ve fucked with your settings and they all run fucked. “it’s a fresh install”. No you just acknowledged it’s a fresh install that you have immediately put in a very non default state.

I already mentioned I've wiped the OS and I said I recently installed App Tamer to deal with the ongoing problem. Recently. So yes, I have in fact tried not fucking about with all the processes.

> You’ve fucked with your settings and they all run fucked

No, I think what happened is you lack adequate reading comprehension and would rather not interpret comments in good faith and decided to jump the gun and chastise a fictional version of myself.

> No you just acknowledged it’s a fresh install that you have immediately put in a very non default state

No. I shouldn't need to explicitly spell this out. And if you're not sure, you can ask instead of coming in hot with off-base assumptions and attitude. You can rest assured that I know what I'm fucking doing, man. I know how to isolate behavior, eliminate possibilities and debug systems, it is literally my profession. Assume a lot less, please.

When I say I wiped my system, it obviously means to anyone competent that I observed my system after wiping it and before modifying anything substantial. You literally have no idea what my configuration is. What is "fucked with your settings" supposed to even mean? They are settings, my man. They are meant to be set. I know what each one does, and how it affects my system. My system is largely stock other than the user-level applications and environments I manage for work. If changing the theme or disabling iCloud/Siri is causing these problems then I'm going to throw my machine out the window and find a different profession. Come on, man.

I just want to give you props for how reasonably and reservedly you replied to this person. When I read their comment I just about put my fist through my monitor in sympathetic rage.

Ha, I still maybe got a bit too snarky. But it doesn't feel undeserved, their comment was totally out of left field.

> I know what each one does, and how it affects my system.

That is a bold claim for someone complaining that their system doesn’t work. Look, I’m not apologizing for macOS but it does seem that they test the happy path these days and you either use it as they give it to you or it runs like ass. Like if you want a computer without Siri and without iCloud etc then get a Linux box.

There are people who are qualified to fuck about with their computer and there are people who aren’t and how you tell the difference is after they fucked about with it they don’t fucking complain about it not working. (Because either it’s working or they understand that they fucked it up and they are choosing the particular form of the fucked-upness because they got some benefit).

Holy hell, shut up. It was clearly stated that the “fucking with” of the computer happened after and because of the “fucked up” state of the computer. Your hypothesis is ruled out by the laws of causality and the arrow of time, and moreover if we violate all logic to assume your hypothesis is true, you are still wrong because if it was not fucked, he wouldn’t have fucked with it.

But somehow you are possessed of such inexhaustible and unwarranted confidence that you have not even noticed your claim is both nonsense and impossible, and furthermore you feel justified in serving this impossible nonsense with a side of condescension!

This is such an abject failure of reason and decorum that the only course of action to positively contribute that I can suggest to you is to not contribute at all, i.e., shut up.

I assume you’ve never done technical support. The customers who think they know what they’re doing fuck things up so much worse than those that don’t. They don’t listen and they do so much more damage. They know they’re right and they are so entitled to what they think should work. They’ll tell you what they did and the logs or outcome or reproduction will prove otherwise. The loudest ones are always the ones who fucked it up the most - it’s just defensiveness.

You are projecting so much, friend. And assuming even more. You literally sound unqualified. I work on computers for a living. The amount of contempt and arrogance you're displaying is nothing short of sad.

You assume incorrectly. Furthermore, your description of tech support is wrong, so much so that I half-suspect it is you who has never done technical support. I mean, come on, one of the mantras of every tech support division I’ve worked in has been some variant of “believe the user”. What you are describing is pretty much the very first attitude you learn is simply unhelpful and ineffective, apart from also being aggravating for the customer.

You are the worst kind of corporate apologist. You don't sound qualified to have this conversation at all. You're highly emotional and talk like a fucking idiot. Buzz off.

> Well this is a software issue,

> You don't know that for sure.

> all of the Macs out there run the same operating system,

Wrong. Lots of Mac users skipped the Liquid Glass OSes. I was one until I bought an M5 that came with it. In fact, I was two versions behind on my M1 because the one before Tahoe also offered nothing that I wanted.

> the same software.

This referring to third-party software (my interpretation) is also wrong. People have wildly different setups, at least among us nerds.

> This can't an issue of a defective Mac.

Again, you don't know that. But your comment later on about two models having said issue is concerning. That said, there's a high likelihood that you run the same software on each.

> I am right now using my M5, on low power mode on a flat surface and it's almost too hot to type on.

If that's the case, I begin to suspect that the physical sensitivity that you mentioned in a different comment is the source. If that's the case, I'm empathetic cause it'd really suck to have that reaction. I can only make my M1 hot enough to noticeably warm up (or get hot) when slamming the CPU / GPU. I would hate for it to feel that way to me during normal use and you have my genuine sympathy if that's your experience on a machine running at normal levels. And if that's it, then get an external kbd and mouse. Not an ideal solution, but it sounds like the best option. Not portable. I know. But I'm looking for solutions here and there may be some compromise needed.

> Wrong. Lots of Mac users skipped the Liquid Glass OSes.

I said the same operating system, not the same version of that operating system. Please make sure you're correctly reading my comment before criticizing it.

> This referring to third-party software (my interpretation) is also wrong. People have wildly different setups, at least among us nerds.

Your interpretation is wrong, I am talking about system software, this entire time I've been explicit that this is an Apple daemon and system software issue.

> Again, you don't know that. But your comment later on about two models having said issue is concerning.

Correct, I don't know anything at all 100%. What I do know is I have two laptops worth $10k between them which both have a ton of issues that others online have corroborated and which seem to be tied to 26 and 27 especially. And that these issues all trace back to several Apple daemons. I also am well aware of what 26 and 27 introduced, and can connect the dots. My theory is a solid theory until proven otherwise, but sure, if you want to be smart, literally anything is possible.

> That said, there's a high likelihood that you run the same software on each.

I have already stated in this thread that I have accounted for this extremely obvious possibility while debugging, and assuming I'm an idiot who has not considered this already is patronizing and unhelpful.

> If that's the case, I begin to suspect that the physical sensitivity that you mentioned in a different comment is the source

I hope you think I am capable of reading temperature gauges and differentiating between heat and any additional painful sensation on top before opening my mouth about this issue on a public forum. I also have a partner who I have asked to touch my machine plenty of times to gauge her reaction. I'm not saying these machines get mildly warm. I am saying they get hot.

> I can only make my M1 hot enough to noticeably warm up (or get hot) when slamming the CPU / GPU

Yes, and if you read my comments, you would know that my complaint is that this is exactly what multiple system processes are doing. You would also have seen that I already mentioned I use external peripherals but that I am also often on the go. It is hilarious to suggest that the compromise needs to come from my end when my extremely expensive workstation cannot even idle without sometimes getting warm or hot.

Read the first two quotes and responses. You weren’t as unambiguous as you think you were.

Context clues go a long way. I said the same operating system, not the same version of the same operating system. I can't help it if you want to read something that isn't there.

Daemons like duetexpertd should be running at a low QoS if they're doing background work and shouldn't be capable of interrupting an app. So you've found a bug. You can report it with Feedback Assistant if you want.

I already have encountered dozens upon dozens of threads, dating back nearly a decade, complaining about duetexpertd. Even if it's meant to do background work, when it starts taking multiple entire cores to do this work, it is categorically not a background service.

Personally I have seen the situation get massively worse since I entered the macOS ecosystem in '23. My machine is running hot right now on flat surface with a nearly 0 user-side load. It's a soup of system processes each taking 15-200% of my cores. I used to not have to mess with the fans at all. Now, I have to keep them on literally full blast via third-party software just to comfortably use my laptop even while simply browsing the web (and I have JS turned off by default).

So I feel silly even considering engaging the Feedback Assistant. My demands are for the ability to categorically disable these services. I do not use Time Machine at all, why is the backup service slamming my machine? I do not need any AI-enabled media features. I don't need document title suggestions. I don't need LLM-powered autocorrect at the cost of heat and performance degradation. I don't need Spotlight indexing. At all. I want it to be a calculator and app launch bar. Literally nothing else. I despise user filesystem indexers as a rule and do not require them to use my machine effectively. I do not use iCloud in ANY capacity, nor Siri. So why are duetexpertd and Siri constantly obliterating my cores and causing sudden performance degradation even while in Gaming mode? I have plenty of free space, so it's not a thrashing issue, either.

My feedback would be that literally half of these services are completely unnecessary to me and yet I cannot fully disable or even configure them. I know this feedback will be completely ignored by Apple, because they are in the business of dictating to me what to expect from my computer and not the other way around.

Now appstoreagent is taking up 95% CPU! What! The! Fuck! Why is this a daemon at all? Apps, user or system, should not be updating silently in the background, and if the App Store is not explicitly open, there is nothing I need from it. The problem isn't just that this daemon is blasting one of my cores; the problem is Apple's perverse design philosophy which allows such a daemon to exist in the first place.

Well, first step I'd recommend not looking. There is someone in the world whose job it is to look at this. You're doing their job for free! [0]

As for turning things off, it's a bad idea to sell a product you haven't tested, and it's also a bad idea to add settings that randomly break stuff because users will forget they changed those settings. So it's difficult.

https://limi.net/checkboxes

[0] you can also become that person: https://careers.apple.com/en-us/details/200687840-0836/softw...

> I recently installed App Tamer

Plus one for this tool. I've been using it for about eight months or so. Useful for keeping Backblaze and other daemons from taking over when they're supposed to be invisible.

> Unfortunately, this does nothing for services such as WindowServer which now often takes up 50-100% of a core just on its own.

WindowServer can be a real dog (Finder, too, if opening lots of windows with videos that can be previewed in QuickLook). Closing all of the open windows is inconvenient, but it solves the issue until they get built up again and stay open for an extended period of time. I hope they address this in the future; we'll see how a new CEO prioritizes software quality. I'm the most hopeful that I've been in five or more years given the change in leadership. Doesn't mean it'll happen.

> I have fibromyalgia and seem to be sensitive to hot keys; my Thinkpad regularly getting to hot to touch with even light usage is EXACTLY why I went and bought a MacBook one morning in '23.

That shouldn't happen on a machine built in 2023. Take it to an Apple Store (or third-party) and see if a repair tech agrees. I repaired laptops that overheated when I had that role. There was a debacle about thermal paste a decade and a half ago (maybe longer). If it's under AppleCare, you might be able to get it fixed at no extra charge. If not, I'm sorry that it'll cost money, but Apple offers a flat-rate repair for laptops. Maybe they'll say it's fine and it'll be frustrating, but it's worth exploring.

We will see, Ternus was previously senior VP of hardware engineering so my money has been on it getting worse. I'm excited for Gravity Linux though.

I appreciate your suggestion. I'm curious if the machines you repaired then had aluminum unibodies? My understanding is that if my chassis heats up quickly, then the thermal coupling is good, since the unibodies act as a heat sink. Certainly, blasting the fans at 100% typically makes it cold enough to use after just a couple minutes unless I'm actively pelting my cores. Very impressive cooling system. Unfortunately the default curves are terrible and I have to use third-party software to control the fans, but they do work well. Anytime a Rosetta application is running I usually have to blast the fans as well; Thunderbird alone makes the computer too warm to comfortably use. Thankfully, the benevolent overlords at Apple have decided to cut Rosetta support for 28.

The problem is these background services each taking up 50-200% CPU, and it being enough of them that at least one seems to usually be misbehaving. That means the machine is just running hot a lot more often during casual use. My M2 is under Apple Care but I plan to transition it to Asahi or Gravity soon and just live with the lack of proper SEP support on it for a year or two, and see if that clears up the thermal issues, since it was fine until I upgraded to 26.

The M5 is under Apple Care and I do plan to get around to taking it in. It's a daily driver for work so it's been difficult making time and I had some misplaced hope that Apple would correct this as 27 matured.

Always use an external keyboard, much better for ergonomics also.

Agreed, great advice! I have RSI, lifelong arthritis/gout and other issues so I have to take ergonomics seriously. Laptop stand, external monitor, mechanical and split keyboards and a vertical mouse. But I also work from home and try to move around throughout the day instead of sitting in one spot, and am often on the go, so I find myself having to use the laptop keyboard often.

tl;dr — free up disk space and delete ~/Library spotlight index and medianalaysisd caches

There's a really nasty bug in Golden Gate 27.0.1 which causes ANECompileService, mediaanalysisd, duetexpertd, and other apple intelligence-related features to thrash and permanently spike your CPU. I tested this yesterday on a machine with ~12GB of space left and the result was launchd writing 163 GB and ANECompilerService writing 70 GB in less than 24 hours. The noticeable effect is that the AI text-recognition for photos in QuickLook will not work and repeatedly download and compile ~/Library/Containers/com.apple.mediaanalysisd/Data/Library/Caches/com.apple.mediaanalysisd/com.apple.e5rt.e5bundlecache items. One in particular repeatedly fails and downloads a 69.9 MB file with model.mil and weights1.bin inside of the package folder. This will happen every 6 minutes, which further compounds the severity of the issue since the bug itself is due to low disk space. This folder has risen to 2–5 GB for me before, which I had to just delete manually. The filenames are <HASH>.tmp.<number which increases every time>.<random>.bundle.

I figured this out by watching `$ log stream` and seeing what was happening, which was lack of disk space while ANECompilerService was compiling ai features after opening a file in Preview or in Quick Look in Finder.

I fixed this by freeing up ~10GB (bringing me up to 20GB) and deleting the ~/Library/Metadata/CoreSpotlight and ~/Library/Containers/com.apple.mediaanalysisd/Data/Library/Caches/com.apple.mediaanalysisd/com.apple.e5rt.e5bundlecache/. After rebooting, I left it on overnight to reindex everything.

I’m on an M1 workout these issues. How are you isolating them to be AI related?

medianalysisd for example, which frequently pummels 1-2 cores of my machine for long stretches of time, runs local on-device models to analyze the media on your machine.

You know, that brings something to mind that I haven't seen mentioned (so far). If you wiped the machine, it has to rebuild all that "knowledge". Apple announced that they'd rebuild the Spotlight index with an update to 26 so that people upgrading to 27 could use the new AI features immediately. Maybe the timing rules this out, but keep in mind that the device could be running processes that will eventually expire / run only on new content. Could that be a factor?

I know a new phone or Mac bangs hard on indexing even pre-foundation models. I haven't set up a new device with 27, so I don't know how much that might have increased.

I wiped the machine weeks ago, not recently, and I wiped it while debugging the issue, which existed prior to wiping it. I was having these issues on Tahoe on my M5 (came with it, unfortunately), but not Sequoia on my M2 (occasionally things would spike but it always ran cool when idling or doing my normal work). The issues got even worse on Golden Gate. Upgraded my M2 to test and it started exhibiting the same problems.

I also have as much indexing turned off on Spotlight as I can, and that is another issue. Spotlight seems to disrespect my settings and indexes/displays more than it should. Seems like an open, unaddressed bug. I've assumed this is related to the Spotlight-specific CPU usage issues I have but it doesn't explain the others.

Huh, I hadn't seen App Tamer before, that's quite a neat bit of kit. St. Clair does serious work...

I'm glad I found it. Jettison looks like a nice tool as well. It's working great so far, just does what I need it to do. Fits in nicely with Stats, which I use for fan control and monitoring power consumption. I also found https://github.com/Digital-Shane/OpenTamer but figured App Tamer was still the more mature option.

No. Apple’s Photos ecosystem uploads all Photos effectively unencrypted to Apple in a way that is readable by Apple, allowing Apple to do server-side analysis of all content. This content is also provided to US or Chinese federal authorities without due process or warrants.

Nothing about iCloud Photos is private.

With Advanced Data Protection enabled (it's not available globally but it is in the US), Photos is E2EE.

Even with ADP on, it uploads the unique hashes of the PLAINTEXT of every photo to Apple, so they know who had what unique files, when, and who had them next, and when, etc. You can’t receive or transmit an original Winnie The Pooh meme between two phones with ADP on without Apple and the FBI knowing who had it first and when the second person received it.

Also insofar as ADP is off by default the whole system can NOT be said to be “privacy oriented”, which was the claim I was refuting.

Apple, for the most part, does not care about the privacy of their users. ADP would not be built today, and it will probably eventually be quietly removed (like it was in the UK), because ~nobody uses it.

The UK removal of E2E was anything but quietly removed. Unless your definition of "quiet" included the cover of the YT, the WSJ, and assorted other world wide papers.

> The UK removal of E2E was anything but quietly removed.

I assume customers were notified?

Yes

Exactly my thought as I read it.

They make and sell the hardware that will actually make AI something everyone can used without needing to rely on OpenAI/Anthropic et al.

Apple has never had a moat, anyone else could have built this sort of hardware but they haven't.

So they don't care which model wins, because it will always move out to the edge of the network, and they own a large chunk of that edge.

Well, the moat is that cpu/npu development is very hard, requires talented engineers, and requires access to cutting edge fab process, all of which in turn costs a lot of $$$. So, sure, “anyone” who has all that can definitely breach Apple’s moat.

Ok.. and why specifically do people work at Apple?

Hint: money is not the only reason.

Jeez.

Most of the smart people I know have left Apple because over the last ten or fifteen years, all of the reasons other than money have gone away, and they don’t pay well.

Don’t kid yourself, Apple paid’s very well and that includes stock options. Apple in comparison to their competition is very careful on who they hire, and they’re also careful on who they layoff in comparison to their competition, who seem to make big mistakes and layoff thousands when it suits them ala Meta or Microsoft recent times.

If you fortunate to get a job at any of the big tech companies today, quibbling about the pay package is the least of your worries.

I’m at a loss as to why you think I said that people would only work at Apple for money.

Do Microsoft, Google and others not have access to all of those items?

Ermmm what?

Apples largest competitive advantage has always been and always will be - trust combined with epic marketing to make the vast majority of people aware of it.

Trust of quality, privacy w.r.t data etc.

I’m sorry but I’ve seen so many bad takes re. Apple over the years. Many of you think you are so smart on topics outside of your domain - but you’re not.

> Apples largest competitive advantage has always been and always will be - trust.

Lmao

> Many of you think you are so smart on topics outside of your domain

Turn the mirror around

What exactly about the parent comment is wrong? Apple has always been good at making hardware that people are willing to pay a premium to use. I can easily imagine them playing a similar role in local LLMs to the role they played in personal computing in the 1980s or smartphones in the 2010s.

Don’t underestimate how much of their success is due to unusual internal advantages over competitors.

They spent decades learning to eliminate a 1000 headaches for themselves, and many for their customers, by accumulating vertical integration:

The co-design of their own comms, processors, devices and OS variants. Their own integrated supply chain and sales channels.

The better the results, the more invisible or clock-like their delivery.

Very easy to overlook. But nobody can match them on their terms.

(Nintendo also applied a unique set of internal advantages to generations of successful devices: vertically integrating humble components, highly scoped software, a strong theory of fun, and deep branding with novel polished interfaces.)

This post could be taught in business school. Kudos.

Apple is the only company that actually listened to Alan Kay when he said if you care about software, build hardware.

??? Apple’s advantage over their competition is the fact that they do hardware, Apple Silicon and they do operating systems Apple OS very very well, in comparison to the present competition in the marketplace.

They are the last company that’s vertical from the 1980s. Unfortunately, all the other companies (SGI, Sun, Digital etc..) died in the face of the Wintel colossus.

Marketing is irrelevant. If you don’t have something to sell that is actually any good see the Microsoft Surface fiasco and its lack of an impact in the marketplace.

> Apple has never had a moat, anyone else could have built this sort of hardware but they haven't.

I think both points are very incorrect. Apple has a massive moat. And Apple is making hardware (specifically, CPUs) that are two years ahead.

That "moat" is being reduced (again, compared to previous Apple history) as other companies catch up. Apple has continually cannibalized their own hardware to keep ahead of others.

Unlike the "pure" AI players, who rely on NVIDIA and others, Apple is relying on its own capabilities, vertically organized, to maintain its lead in consumer space.

They are using the "cloud" to again, vertically integrated, to preserve their "privacy first" consumer marketing, their hardware to maintain their "premium product" lead, and they are usually second to a market after a first-mover advantage has been reduced by ongoing development.

apple is pretty well set up for this because they do all their hardware in house. if they wanted to, they could pivot in this direction. Ternus is also a hardware guy.

Nvidia is also selling the shovle but they train their own LLM and by doing so they can fine tune their CUDA/Chip to have a better shovel than others.

Nvidia's own model does not have to beat OpenAI or any SOTA model. Nvidia just need to build up the in house expert and knowledge to make a better chip since hyperscalers and none hyperscaler like Meta is building their own AI chip now.

Apple can chose not to train their own model to learn how to sell a better shovel.

That's just BS cope. They tried and miserably failed - and still are failing badly. Remember, they were the ones to push for AI with "Siri". Many firings later and 2 CEOs later we are still exactly in square one. The fucking thing can't even get who I want to call, let alone set an alarm without touching my phone FFS.

What is there to cope about? Apple is the last vertical computer company left from the 1980s. Apple Silicon just gets better every year. They have also managed to replace several companies that well gave them problems over the years or said no to doing anything that involved moving into the future. i.e, Motorola of Schaumburg, Illinois, IBM and Intel one company’s out of business. and the other two are trying to hang on to past glories.

By the way, the new Apple folding phone appears to be a hit. a real hit the last test left is whether or not it is as good as it appeared to have been in the demonstrations. only time will tell when it gets into the hands of the general public.

Lets not overlook the fact that Apple is only around due to sheer luck with NeXT reverse acquisition, and having Steve Jobs back on the steering wheel.

Nowadays they are basically the iDevices company, if macOS wasn't required for XCode, the worldwide sales would be much less.

None of what you said is relevant to my comment about Apple's failed AI strategy nor Siri.

I have no interest in Siri or always-on AI assistants. I turn as much of that off as I can.

I do have an interest in local, private inference though and I am willing to pay for that.

Apple segmenting the market is actually a good strategy. Horses for courses, as they say.

> local, private inference

Yeah, sure, inside a closed-source OS communicating constantly to private servers on the internet. Sounds very private to me.

As long as you can disconnect it from the network then it's more private than something running on a cloud server.

> They are out here selling the shovels

just wish their containerization / linux compat wasn't so nerfed. Just moved my homelab's inference engine over to a GB10 from a Mac MTP setup to have first class k8s support, as well as open up access to the broader CUDA ecosystem (cuOpt etc.).

They’re not really. They make the things useful for two classes of users: Mac Studio for people running models locally and Siri backed by Gemini for voice interaction.

They’ll likely embed text models more deeply in the future, but the history of “Apple Intelligence” doesn’t give a lot of hope that they’ll nail it.

So they’re not selling shovels (a precursor to making the thing) but products and services to use it.

They have the best strategy of bigtechco: let others figure out the squishier layers while they focus on selling the best hardware. Privacy has incredible value to a lot of individuals and industries. $20-80k in hardware that *appreciates* the moment you buy it is a reasonable investment for a lot of SMBs, and still less than a single Nvidia rig. Privacy pays.

If we're being frank here, Apple's AI strategy lags behind Nvidia, AMD and even Google. Nvidia has CUDA, AMD ships CDNA, Google has the TPU family, and Apple... well, Apple discontinued the rackmount Mac Pro, forfeit the HPC ARM datacenter to Nvidia's Grace CPUs, refuses to sign CUDA drivers, and spent half a decade designing raster GPUs instead of GPGPUs.

All of their competition has real inference and training stacks. Apple ships an iGPU and a smartphone NPU, they're reliant on Nvidia and Google hardware to deliver Apple Intelligence.

Saying they are lagging assumes they are in the same race

They aren’t

Apple is a high net worth consumer technology integrator that focuses on highest coherence user experience. They pick the most efficient stable technology, integrate it and sell the package

Apple only needs to lead in precisely that they don’t need to lead in literally anything else

in fact if they tried to lead in those other things they would give up their integration capacity

But they are. Nvidia ships GPGPU ARM SOCs to consumers; why can't Apple? They're both trillion-dollar companies, surely Apple isn't lacking in engineering talent.

The vertical integration argument relies on Apple having a good architecture for AI. First it was the ANE, which everyone (mistakenly) thought would differentiate Apple Silicon from other NPU platforms. Then everyone parroted the "unified memory" talking point without checking to see if professionals actually used it. Now people are pivoting to say the next-gen Apple GPUs will finally blow the last-gen GPGPUs out of the water. Which is good, because there are 4-year-old GPUs that beat M5 Ultra in compute efficiency.

I know that "Apple is behind" is a tough pill to swallow, but they literally have no analog to CUDA, CDNA or even TPU technology. Nvidia SOCs like Tegra prove that Apple could absolutely bring GPGPU tech to low-power hardware like the iPad and iOS without giving up any real features. Apple just refuses to invest in it.

I feel like you didn’t read my original post cause you’re just repeating the original claim that they need to compete in components market instead of consumer product market

Theres a faulty mindset on here - where people view things as zero sum.

When S Jobs arrived back at Apple the mentality was not to compete with MSFT on everything and to co-operate where it made sense. Positive sum.

Indeed one of the great moments in computing history was the microsoft investment in apple

The greatest moment was Apple being the last vertical computer company left over from the 1980s, OSX second the iPhone third and Apple Silicon fourth. What saved Apple was their frugality as a company sound familiar? Not Microsoft’s measly $150 million settlement. No OSX and no iMac and Apple would’ve been doomed.

Microsoft currently is scrambling because they missed mobile and they’re currently losing in the gaming market. Microsoft is not even close to winning, because Sony and Nintendo have basically kicked their butt in gaming and now we have a new competitor in Steam, Microsoft is all in on Copilot and if it doesn’t work, Nadella and his game manager are done.

Yeah I’d agree

I can’t think of a single Microsoft product I would choose at this point

Ask Valve about it.

Yes, because they were almost bankrupt, hardly one year business left.

Nope - if anything anti-trust is closer to the truth. Bill needed Apple to survive.

I always find it interesting when poster's here post so confidently and are wrong.

I disagree.

I think the iOS App Store is going to be in steep decline in a few years. People will generally have a few apps: ChatGPT, Instagram, messaging, and something like a Photos/Camera app.

Everything else will be done in your AI app.

I've been hooking up more and more data and plugins into ChatGPT/Claude. I only want to use it. I don't want to use other apps if I don't have to. For example, research, buy, plan. I prefer not to use other apps.

I think all it takes is for OpenAI to make a phone from the ground up for this and Apple is going to have serious competition.

How many 5080 16 Gig cards would you have to buy to run the same model, and what would that cost?

idk but naive me sees the article say the model needed 158GB of RAM so I guess 10x 16GB GPUs? Apparently retail price is $1k per 5080 16GB but they're actually selling at $1.5k so I guess $15k for just the GPUs.

158GB number is for Mistral Medium 3.5 128B model, Muse Spark is way larger.

  Muse Spark Open would require more than 1.6TB of RAM in its native precision (BF16) for the model parameters alone. In a 4-bit quantized (NVFP4) format, the model memory footprint is lowered to about 493GB.

So about 24gb/expert at NVFP4. If you compressed the KV cache, you might be able to get away with 2x 24GB cards and stream the experts.

If you're happy with generation speed in 5 t/s range, sure -- PCIe 5.0 x16 is only 64GB/s.

I never said it would be fast!

In any case, I'd much rather run GLM 5.3 Flash on that sort of hardware. Better fit for the performance profile, and will probably beat Muse Spark Open in most uses.

Both companies are profitable and AI is not their main line of business. This affects pure AI companies.

As open weight models continue to match frontier models, the race to $0 will continue to benefit supporters of running open weight models locally.

It won't be free for Apple to use Meta's model in the future.

Even in the unlikely case that does happen, there are plenty of other open weight models like Qwen, GLM, and Mistral to name a few

Google, who is far more qualified, and is getting a measly 1 billion dollar per year refund on that $20 billion dollars they currently paying Apple for a default search position on Mac OS. Meta, whose software prowess is questionable, when compared to Google or Microsoft isn’t even in the game they are getting how should I put this Apples charity/tolerance which is all OpenAI and Google are currently getting.

One is getting a $1 billion refund from what they already pay Apple, and the other two are getting nothing from Apple.

Meta won’t be getting anything from Apple in the future, other than tolerance/charity.

How big is the local LLM market? I have trouble imagining Apple will move enough units to make $20k/ea worth it for them as a business segment.

They have been sold out of a lot of units, due to people developing AI, so apparently, it is significant.

I think the bet is that it will get bigger, both at the consumer level, and with orgs eventually discovering ready-made solutions for having their own on-prem inference.

100,000 units a year would be $2 billion annually.

And usually, most of a company's revenue comes from the heaviest spenders.

So figure there are some out there who will by tens of these because they have some "idea to change the world". (Whether true or not.) Some startups, etc.

I'd wager, across planet earth right now, it should be trivial to push 100,000 units per year. Probably closer to 500,000 units per year. So that's an easy $10 billion a year.

$2-10 billion is meaningful for any company. Even one as large is Apple.

Especially if it's a first step towards a stronger foothold in the AI inference space.

They can then expand down market to grow the pie further.

For example, with truly dedicated inference boxes.

And given the demand for $7,000 Sparks and sold out $5,000 GPUs, the market for these $20,000 machines is likely much higher than my 100,000 unit lower bound. (Earth is a big place. It doesn't take much excitement to generate massive sales in a space with currently insatiable demand.)

The only question you need to ask is whether or not Apple will do anything to design and engineer memory in-house, the money, and the capability is there that is the challenge. because you don’t need to select $20,000 computer if you get that under control by bringing it in-house.

In short, Apple needs to roll up there sleeves and get busy the three hyenas of memory and that nameless memory company in China will not sell you memory when you need it at a reasonable price. In the past Motorola said no, IBM said no, and Intel said no. This is another one of those times for Apple to adapt by designing and engineering around the competition.

The memory constraint isn’t design. It’s fab capacity. Apple has exactly 0 fabs. By the time they hired domain experts and built fabs and tuned to the point of reasonable yields, the crunch would have been over for 5 years, and apple would have wasted 20 billion with no return.

Probably worth it just as an investment into their developer ecosystem. They could probably sell at a deep loss and still justify it.

Now scale that up to Nvidia and you've understood the entire AI market

I wonder how many units Meta bought to convince Apple to do a collaborative advertisement.

Still cheaper than the equivalent Nvidia GPU setup btw.

The NVIDIA GPU can be changed as new models become available, without throwing away the whole computer.

And the Mac Studio can continue to be used as a regular computer. (Hilariously over-qualified, I suppose. But, no, I got a maxed out, 128 GB Mac Mini earlier in the year—when you still could—and I find it a snappy daily driver in my 'man cave' downstairs.)

While failing to satisfy existing Mac Pro demographics, now slowly migrating to Windows and Linux worstations, for all kinds of professional cards they require.

Naturally there are those that will put up with a Studio and thunderbolt cable spaghetti.

Thunderbolt still a thing? I coulda sworn it was all USB C now.

> Thunderbolt™ technology is more than a USB-C connector. Although it uses the same oval shape connector as USB-C, Thunderbolt™ technology certification mandates higher minimum performance and capabilities for cables, PCs, and accessories. You benefit from the best-in-class product capabilities to deliver the fastest cable solution for connecting to power and to all your docked accessories and monitors.

https://www.intel.com/content/www/us/en/architecture-and-tec...

The lifespan of those AI GPUs is like 5 years, I have macs from 2008 that still turn on and function.

So what, my Timex 2068 from 1986 also turns on and functions, does not mean it is useful in 2026.

Apple playing 3d chess here -- they can just absolutely kill it in making the HW available for everyone to run models at home, and will do a better job than Nvidia especially when M7 comes out which is specifically designed for AI workloads.

That is generally how hardware gets sold yes

I think they're sandbagging Qwen pretty heavily by benchmarking Meta's model at 4-bit, against Qwen only at native BF16 (and nothing else shown). That's not a quant most users would choose.

I get that this is cross-brand marketing for Apple's partner, Meta; but really Muse Spark isn't even in the top *ten* models I'd consider if I had a 512 GB M5 in my paws. Or a cluster of them.

> cross-brand marketing for Apple's partner, Meta

Partner!? I don't see where you get that from the article, and Apple & Facebook have traditionally not been best of friends (see everything iOS implemented to limit Facebook's overreach).

I'm guessing they chose Muse because 1. It's not Chinese so it'll look good for Trump's AI cronies (Apple is in the administration's spotlight) and 2. It's fairly recent and thus on people's mind.

I mean that they're doing joint marketing on this particular thing. Muse Spark marketing features Mac Studios quite prominently, and Apple's material spotlights Muse Spark.

Eh, Apple spent years suing Samsung while also buying parts from them for their products. You can be a partner and an enemy at the same time at the high levels of the game.

They clearly list out the quants in Table 1.

Most of the following tables show relative speedups; not tok/s numbers.

I agree no home AI users should really use bf16^; but I see two defensible justifications: (1) showing that their new hardware's speedups scale to full precision too [crucially; NVIDIA is focusing more and more on lower quants], and (2) trying to sell to the crowd running IQ3XXS or something on 16GB.

^q8 ~= bf16 is a good general rule of thumb. And friends don't let friends quantize KV cache.

For what it’s worth, Muse Glimmer is by far the most usable model on my 32GB M1 Max. It gets stuff done very competently while Qwen never gets done reasoning.

FWIW, single M5 Ultra with 256GiB runs DeepSeek 4.1 Flash 0910 just fine (~3200 tok/s prefill, ~45 tok/s). M5 Max w/ 128GiB runs GLM 5.3 Flash also fine.

I also benchmarked both on quality benchmarks (MathArena Apex / GPQA Diamond etc) to make sure the quality is what you get from the official numbers.

GLM 5.3 Flash on 128GB Ram would have to be 2-bit right?

I'm no expert but it seems the consensus is you get a suboptimal experience under 4-bit. Please correct me if I'm mistaken here

MoE, especially these Chinese ones trained on low-bits already is actually quite quantizable to lower bits, unlike smaller / dense models.

Also Expert MoE offload (to SSD), which still doesn’t let me (64GB) run a flagship but if you have 96GB you can.

The problem with SSD offload is the prefill time suffers, which is not really obvious.

MoE weights are much friendlier to low-bit quants than dense weights, because only a few experts are active per token so the per-expert error stays local. But the trap people fall into is that once weights are down to 2-3 bits, the KV cache and activation buffer at long context become the dominant term — further weight quantization stops moving the number. Memory-planning for MoE is really a context-length problem wearing a quantization costume.

I think that is true for GLM-5.3 to some extents, this is not true for DeepSeek 4-series models as they compress KV aggressively.

I wouldn’t know since I haven’t found a model to test it on, but in my experience with smaller MoE models (35b A3b), they are the only choice for my M3 Max because prefill times are reasonable (I don’t even need to quant my KV cache, which would mess up MTP anyways). The only reason dense models (3.8 27b) are unusable for me are the high prefill times (mitigated if prefix caching is working correctly).

Not exactly your average "desk", but still neat.

  In this post, we show how to run the latest frontier-class model — Muse Spark Open, with nearly a trillion parameters — right from your desk, using the latest version of MLX, released today. On a cluster of four Mac Studios, Muse Spark Open generates over 36 tokens per second (see Figure 1), well above average human reading speed. Beginning on a single Mac Studio (up to 512GB of unified memory), then scaling out across multiple Macs with the RDMA over Thunderbolt driver, below we share an overview of the range of models you can now run on Mac Studio.

Yes, I'm very glad they're doing this, but as of 2026, it's a bit academic. Pricing for the 512GB Mac Studio hasn't even been announced yet, but the 256GB version starts at $9499. So when you start talking about four of those puppies, it's substantial.

Someone else will do the math on how many years it would take to spend $40k on Muse Spark Open via OpenRouter.

There was the M3 Ultra 512GB Mac Studio, though.

…which is in the region of €20k on eBay, so the calculus doesn’t really shift…

It was only like $14k new. Why are you people acting like nobody has this stuff? People were already building clusters when that thing first came out. The research isn't bunk just because the refresh hasn't landed yet, my god.

Maybe you (or i?) missed the point - it wasn't that nobody has those machines, it was that at $40k for 4x the 256GB versions is a substantial price, and so the parent parent was suggesting that it would be compared against OpenRouter. Ie how long would it take someone to spend $40k worth of credits on OpenRouter?

I've had the same thought on self hosting for the past 2 years. Eg i could invest $10k into hardware or spend those on credits. I chose credits, because by the time i spend 10k i suspect the market will have dramatically shifted in requirements, compute, etc.

The substantial price is one reason I'm disappointed that open models are still getting bigger and bigger, rather than more efficient in any way.

I just also think that plenty of people buy clusters of Mac Studios anyway.

Haven't smallish (16-32 GB VRAM) open-weights models been getting better and more capable over the last few years? Seems like what you can run on 'a laptop' today beats the pants off of whatever you could run on that same laptop two years ago.

I guess. I found that Gemini 3.1 Pro and 4 Argon are by far the best at what I like to use LLMs for. Even 3.8 Flash doesn't compare, and I'd imagine something I could run on my local hardware (M4 Max 128GB) would be just as disappointing as the rest, at least for now, even if better than earlier models. One of the reasons I've been hoping to see diffusion language models eventually.

> I chose credits, because by the time i spend 10k i suspect the market will have dramatically shifted in requirements, compute, etc.

I agree, at least for inference credits. But anecdotal counterpoint: about ten years ago I put together a system using about $1600 worth of new and used components. Including electricity that TCO has been something like $14/month. I'd be hard pressed to find 128GB of RAM, 12 cores of x86_84, and 40 TiB (raw) worth of disk, and unlimited ingress/egress traffic for rent at that rate. That's what, $0.02 an hour, rounding up?

Yes, a hyperscaler's rent-a-computer would get faster and be more energy efficient. Possibly more carbon efficient over it's lifetime, although I have no way to quantify that. But in raw dollars it would have costed me absurdly more to rent. On the other hand, my machine has sat cold and unused for multiple months when I didn't need it to be running, and I don't consume all of it's resources around the clock. So maybe I could have gotten away with a lesser amount of local compute in a Raspberry Pi or similar, and an equally small cloud box for $5 a month or something. I dunno. I don't even own a laptop or desktop, and try to use 'cloud' services whenever practical. (Edit: always giving priority to privacy in those circumstances; only data I don't care about goes up.)

There are tradeoffs. I don't use my local server for LLM inference, it's far to inefficient for that; so I'll happily pay OpenRouter per token as needed. So I do think that hosted compute for rent does have a purpose. I guess all that's to say that I don't see it in black and white; there are times to invest in local hw and times to ride on the optimized bleeding edge of what other companies are willing to host in big datacenters.

This turned into a bit more of a ramble than I intended.

Apple’s article isn’t targeting people who bought multiple M3 Ultras before the RAMpocalypse - those folks can be safely assumed to already be on board with running large models locally.

I thought the article was about showing this specific model can be run locally, not convincing people about running models locally in the first place.

Indeed; my point is that people who spent $40k on an M3 Ultra cluster are mostly already doing this or similar, and will likely be ahead of Apple’s article.

It seems Apple figured they could sell more Mac Studios on the basis of frontier local AI models, profit to be made while users haven't figured out the math yet.

Don't get me wrong; in the long term I see the price of local inference falling as their chips continue to get 15-20% faster every year, and meanwhile the "intelligence per billion parameter" chart is rising. There will be a magic point where they cross, and "good enough" reaches your phone. That's when Apple starts to drink Nvidia's milkshake.

Apple and NVIDIA prices are not going to fall.

Their competitors will. Anyone locked into these two ecosystems are willingly going along with it.

It’s not about the price of Apple products falling—I was commenting that the price of inference will continue to fall (chips getting faster). If “good enough” inference is in your pocket, this benefits Apple and harms Nvidia.

I got your insight as you intended, and I agree that good enough models able to run on commodity hardware will hurt Nvidia.

We aren't far from that point.

Apple’s problem is memory the biggest question is whether or not they will design and engineer around it their recent history says they will.

Puts into perspective how cheap the frontier models are via subs or even API.

You can't really compare API prices though. The cards that they are running frontier models on in the datacenter can service hundreds of simultaneous requests and are at nearly 100% utilization. This thing would sit on your desk and be used to service a single user for a couple hours per day.

It is the locally ran models that get me excited. They have an immutable user agreement and a focus on privacy.

Subscription / on-demand polices are a moving target, along with cost.

I foresee the ML market replicating the video streaming services market. Join us at a low rate and no advertisements. Six months latter, advertisements for everyone, unless you pay more $$. Now $$ will still show you ads. We are raising the rate to $$$ with adds ...

The main subsidy, from OpenAI, is ending 10/26. It’s still “cheap” for now compared to all the other non-open weight frontier models, but it won’t last since OpenAI’s stated goal is to make it so that there’s no much of a difference price wise between on demand API calls and subscriptions

But there are definitely people out there that will need to run these models locally (contractual or legal requirements). By validating a Mac Studio as an option for running local models, it might be the more appealing option.

For most of us though, you're right -- the OpenRouter option is far more cost effective.

Hilarious how they managed to put out an entire press release about Meta’s technology without mention Meta other than what is in the code references.

Still weird, and against brand, for them to lead with this particular model from that particular company. The groans are still earned.

I find it bizarre of Apple to do anything publicly in partnership with Meta. I always thought the reasons Apple snubbed Facebook were out of values and principles not business. That could be wrong. Now that Meta has a line of business that Apple can finally charge a platform tax on there is new reason to start sleeping with them.

Its still weird though.

The datacenters are not full of Mac studios but Nvidia hardware.

They are known for macbooks.

And they are not willing to build proper apple servers for all the build systems people have.

They did, and few people bought them, so they stopped.

I worked at a company where one of the pieces of hardware we were building at the time used a PowerPC CPU (this is back in ~2007). We had a rack of Apple xServe for build and test. The xServe was nice pieces of hardware compared to the x86 kit we had, more like Sun gear in their attention to detail and ease of use.

At my current company we compile for x86 and devs can do a full build and test environment on their laptops (which are 95% MacBook Pros) but it is unclear what will happen when Rosetta support is pulled. Now the interesting thing is we plan to move our stuff over to target the AWS Graviton (and other cloud ARM cores) as those instances are cheaper for generally the same performance (for what was are doing anyway) so the point will be moot. There will be some crossover point where having build and test servers in a rack will be cheaper than doing it in the cloud. At that point having Apple ARM based servers would be a win assuming we can build a binary that works for Graviton, et.al.

I believe you will still be able to use Rosetta 2 from VMs, if local builds using linux (say via a devcontainer) are acceptable.

They discontinued the X-Serve because they had no control and they canned Nvidia and AMD in time. The last straw was Intel their no on the iPhone chip led to the creation of Apple Silicon.

They also had no’s from Motorola and IBM before Intel. All of those conflicts contributed to Apple creating Apple Silicon, which in turn led to in-house modems, and several in-house custom chips for other Apple devices down the road, I anticipate it will also lead to some move in the memory area.

Apple sells a Mac minis to companies though - to the point that they were supposedly asked to avoid redesigns when possible so that new models could slot into existing rack solutions.

> I always thought the reasons Apple snubbed Facebook were out of values and principles not business.

Apple is showing a masterclass of how to tow the line perfectly. Or more candily "have the cake and eat it too".

If you ask anyone, what's one company that'll stand up to government requests, most reasonable people would say Apple. Yet, Apple is extremely strategic when it comes to China. It thrives in that country, without being associated with CCP.

Same here - they are happy to partner with Meta, without getting any of the bad "aura".

To be clear, all companies do this just that Apple alone has a successful strategy here. Tim Cook personally gifted the current adminstration with a gold plaque, something none of the other "dirty" CEOs did. Such a public gesture, yet nobody would think that Apple is in bed with the current administration. Its marvelous to see this happen in real time.

Apple also was one of the largest contributors to the ballroom. They're not maintaining much of a "moat" between the company and the administration.

> If you ask anyone, what's one company that'll stand up to government requests, most reasonable people would say Apple. Yet, Apple is extremely strategic when it comes to China. It thrives in that country, without being associated with CCP.

You can stand up to government requests when it is legal to do so. Eventually you have to decide to abide by the law or leave the market.

They don’t lead with it. The title of the actual article is “Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio”.

The submitter changed it to “Apple ML Research: Muse Spark Open with MLX on Mac Studio”

Second paragraph: "In this post, we show how to run the latest frontier-class model — Muse Spark Open, with nearly a trillion parameters..."

It’s not a press release and not about Meta’s technology.

The title (“Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio”) doesn’t even mention Muse.

It’s more like an advertorial for their tech.

For the press, you had the release from meta here: https://research.meta.ai/blog/muse-spark-open

...and Zuck and Ternus high-fiving each other here: https://x.com/finkd/status/2110778025629347999

"We're excited for mommy and daddy to be playing nice again," Meta AI chief Alexandr Wang told Axios here: https://www.axios.com/2026/10/15/apple-and-meta-bury-the-hat...

Tim Cook, smiled. with OpenAI and Google a smile is easy when it isn’t costing you anything on the Apple side so John can smile away in addition, the AI models are getting smaller and more powerful by the day something else to smile about if you’re Apple.

That first bar chart showing between 15-36 tokens/second is for a 8,192 token context. Thats not nearly enough context to be useful these days

Agent swarms have figured out how to mostly work around context window limitations. There's no reason the same ideas can't be applied on a smaller scale, at least in principle.

Could you please paste some links about this that i can further read? Really curious

To be clear, at 0.75 words per token, that's about ~6,100 words (roughly ~12 pages), which I suppose could be useful, but as other commenters have pointed out, you'd need some kind of orchestrated swarm/hive that could smartly fit payload to within that envelope

> peak power consumption is about 200W for the Mac Studio with M3 Ultra, and 300W for the Mac Studio with M5 Ultra — a fraction of the power used by a common toaster.

(Emphasis added.)

I run my toaster for hours on end too. Love my sandwiches.

Once a month, I run my toaster oven the entire night to take advantage of off-peak energy rates.

Imagine if you had 4 of them!

It can offset the cost of a space heater in the winter!

Bad comparison, if I needed to make toast I'd use a Pentium 4 not an M5 Ultra

A 800B model that performs way worse than other 300B ones. Well...

What are you using to determine performance?

Most people are probably looking at: https://artificialanalysis.ai/models/comparisons?compare=mus...

This is still a welcome model, even if I won't be running it at home

Muse is significantly faster and more token efficient. That is pretty impressive considering how much larger it is. There is a lot of potential even though this release won't shake up the standings.

Am I right that investing in your own ML hardware only makes sense when you are dealing with private data and confidential sessions, right? (Not counting development or tinkering work).

If you think of it as API access vs temporary hosting running the same open model, it makes sense to run the hardware yourself whenever you are sure it can be fully utilized.

It's a good article which showcases direction of travel, i am sure mac studio will get more and more powerful in coming generations but for now it's way to expensive for local setup.

yeah especially because it's not very easy to configure everything sensibly for a multi-use case setup, they really need better mac containers on the mac itself

Many thought Apple should be competing with the frontier labs while they see themselves as competing with Nvidia.

Right now, they're competing with neither.

Very cool

Has anyone adapted the open Muse model to work like a decision model?

Sort of like Laya but based on muse spark

I am super excited about this.

While right now Mac Studio is not a budget device and it certainly costs more than the subsidised subscription, the promise that one day we can run these models on commodity hardware is exciting.

So Apple is the NVIDIA of open weight in this case

Apple is the AMD of open weight. No LLMs of their own, but a large appetite for partnerships.

Anyone know why they went with Meta models?

Apple has too many arbitrary, pointless grudges for Ternus to take them all seriously.

Google partnership for Apple Intelligence, Meta partnership for local models, and allegedly NVLink Fusion support for M8 GPUs. Maybe the White House told him that it's time to start burying hatchets. I don't get the impression that Apple's prior AI strategy under Cook made any real progress.

That makes sense I also assume there’s better allyship with Meta than X long term geopolitically and they are already in bed with google on search

openai and anthropic are too chaotic to bet on would be my guess

Not pointless, all of that led to Apple Silicon, and you seem to be ignoring the fact that it’s not costing Apple any money. None of their hookups. This so-called cooperation is costing Apple a dime. But it is costing the other side billions of dollars.

Smile, the models are getting smaller and better, every generation of Apple Silicon is getting better in the opposite direction. The only problem is the cost of memory and that is within the design engineering ability in-house it is also within their financial capacity, so if I’m John or Tim, I too would wake up with a smile because I can fix the one thing that’s giving us trouble memory because we have the talent and the money to fix it in the next four years.

Apple Silicon copied AMD's mistake. RDNA and Apple Silicon are both the slowest GPU architecture for AI because they spent 6 years optimizing raster performance over compute. Now there's 4 generations of Apple Silicon Macs that rely on compute shaders for any kind of efficient inference. Even the M5 GPU is less programmable than Nvidia's ARM GPUs from 2017.

ARM datacenters were a trillion-dollar market that Apple made ~$0 off of. The Mac Pro could have fit that niche, but Tim Cook chose to depreciate it instead of letting the PCIe connectors be used with Nvidia cards. He cut off Apple's nose to spite Nvidia's face. It's nonsense, especially since they're crawling back to Nvidia to license their interconnect, in tacit recognition that it wasn't a real grudge in the first place. My guess is that Ternus doesn't have Tim's money allergy, maybe at the White House's request. In any case, the grudge is over, and Apple forfeit ARM HPC altogether.

> because I can fix the one thing that’s giving us trouble memory because we have the talent and the money to fix it in the next four years.

By all means, continue. Which X account or YouTube channel convinced you that Apple could replace SK Hynix in 4 years?

Yes, the traditional Mac Pro customer now has pivoted to Windows or some Linux powered enterprise workstation powered by Ubuntu, Red-Hat, SuSE.

> The Mac Pro could have fit that niche, but Tim Cook chose to depreciate it instead of letting the PCIe connectors be used with Nvidia cards. He cut off Apple's nose to spite Nvidia's face.

Apple Silicon only has support for 16k pages via DART, rather than the 4k pages typically used by AMD/Nvidia. The requirement to coordinate a major change of drivers to understand 16k pages is why both PCIe GPU and eGPU do not work (or rather, did not work - TinyGPU is a compatibility layer that adds eGPU support and is used for AI workflows).

The Mac Pro's biggest redeeming feature for a data center was that it was rack-mountable. The ARM version was still mostly an empty case. It remained mostly for its ability to support PCIe cards for video and audio production use cases, such as live broadcasting.

> especially since they're crawling back to Nvidia to license their interconnect

Licensing the interconnect for broader compatibility. They support other interconnects as well.

They went with several open weight models. Presumably they did that because they wanted their research to be reproducible (once the 512 GB M5 Ultra actually hits the market). Using say a Gemini model they have access to only due to their partnership with Google would not be as useful for people to understand what Apple did.

Preparing to upsell those very expensive Macs coming later this month, are we.

I -- love -- an article -- full of -- em -- dashes.

If used correctly, I find the em dash one of the most elegant and reader-friendly ways to structure certain parts of a text — that it got such a bad reputation lately really is a shame!

Em dashes—love ’em!—but they do. not. need. personal. space.

If you type the following into Microsoft Word:

> In this post, we will show how to run the latest frontier-class model - Muse Spark Open, with nearly a trillion parameters - right from your desk, using the latest version of MLX, released today.

You will get

> In this post, we will show how to run the latest frontier-class model – Muse Spark Open, with nearly a trillion parameters – right from your desk, using the latest version of MLX, released today.

Not everything with em dashes is the product of generative AI... some of it is the word processor correctly following the rules of grammar and punctuation.

Do note that those are en-dashes and not em-dashes.

Reply on news.ycombinator.com