It's remarkably difficult to maintain signal integrity/etc if it's not very close to the CPU at the signal rates they are getting to.

This is already a serious problem.

For places where you are willing to give up power efficiency, or latency, or both, this is not as big a deal - this is why you see CXL/etc.

But for a laptop, where people want power efficiency, latency, etc, this is a serious issue.

Because the signal integrity across the pins and traces becomes a serious bottleneck.

SODIMMs are already "past" the limit, requiring 1.35V for modest DDR5 speeds at terrible timings, hence vendors are trying to migrate to this:

https://www.anandtech.com/show/21069/modular-lpddr-becomes-a...

I suspect regular desktop/server DIMMs are getting close to their limit as well

Essentially, the closer the RAM sits to the CPU, the faster/lower latency/lower power it gets. This is especially important in graphics heavy chips, where the wide, high frequency busses can eat loads of power. In fact, I wouldn't be surprised if Apple transitions to something "new" like a newer generation of Samsung's Wide I/O if they can get memory vendors to produce it in bulk.

Lets compare. The m3 max has up to 128GB of ram, 400GB/sec bandwidth, and uses that memory for both CPU and GPU. It fits in thin (under an inch thick) laptops and is efficient enough to have great battery life on the order of 10-20 hours depending on use cases. So efficient that performance is identical when running on battery or wall power, unlike many of it's competitors. The power savings is even more impressive once you consider you don't need a discrete GPU for many use cases.

The AMD Epyc has 12 channels of DDR5-4800 and manages 460GB/sec. It's a physically large chip, can handle a ton of ram, but also uses a ton of power. You can fit a crippled version (without all the memory channels) on a micro atx motherboard that's several inches thick. Normally you'd need an ATX, EATX, or rack mount motherboard. It needs several 100 watts for just the socket, not to mention the numerous dimms (12 minimum for 12 memory channels). It's the kind of thing you see in a data center, after you put ear protection on.

If you compare the M3 max, which fits in a laptop or small desktop, to Intel and AMD competition that fits in a thin laptop you end up with 1/4th the memory bandwidth. The competition either matches the power use and is much slower, or is as fast or faster at a huge power penalty. AMD and Intel laptops need discrete GPUs to compete and often require wall power to run at full speed.

All in all the M3 max is an impressive chip, fast, and very efficient. But part of that is the soldered in dimms that allow for great bandwidth and low power use.

M2 pro/max/ultra bandwidth is 200/400/800 GBps.

Ryzen dual channel bandwidth is 100GBps. EPYC 8 channel bandwidth is well under 400GBps.