Hacker News Reader

Comment by bmulholland | Hacker News Reader

Intel Mac Pros max out at 1.5TB of RAM, and these are max 192GB. Seems like a large drop. I've heard that Apple Silicon is more memory-efficient, maybe because of efficiencies from the tight integration with macOS. Is that enough to make up for the difference?

View on HN

Apple Silicon isn't more efficient in terms of memory capacity, there's really no way to do that unless you go for some crazy memory architecture that nobody wants to program for.

Apple Silicon has a lot more memory bandwidth though, maybe that's what you heard?

Apple silicon has better support (hardware instructions) in saving / restoring memory. Helps with swapping "unused" things to the background and back.

What does a hardware instruction for swapping do?

Compression/encryption/hashing acceleration.

A reverse case being that x86 has a memcpy instruction and ARM didn't until recently.

I think the parent comment meant there isn't a way to make memory more efficient when it comes to app data. And code takes up so much less memory that x86 vs ARM code size isn't going to make a measurable difference.

In any case, 1.5TB is only going to help a few app niches. It is hard to even reliably predict which kinds. Even, say, editing 8K video could be well-enough accommodated with much less memory if file system bandwidth is high enough.

Unlikely. 1.5TB of RAM is the kind of crazy amount that you'd use if you were handling some immense workload and you need to hold a huge amount of data in memory. You couldn't just cut the amount of available RAM down to a fraction and expect things to continue to work.

Not only that, it's shared memory with the on-chip GPU. Isn't this a big downgrade in both memory capacity and GPU capability?

One thing I was surprised to not see any comments from Apple on is whether GPUs will still be supported in the PCIe slots. It'd require work from either AMD or Nvidia to get an ARM64 macOS driver so maybe that's the holdup there. It's hard to compete with max builds which have many GPUs in raw power without the ability to go beyond the built in GPU. That said, 192 GB of 800 GB/s VRAM is pretty crazy at the price point.

It could be a positive for GPU capability - such as models that would be constrained on GPUs of say 24 GB.

This will be a positive development for LLMs like Llama.

Yes, you only need faith.

Reply on news.ycombinator.com