The M2 Ultra supports up to 196GB of RAM in the products Apple offers it in.
The Ultra may support 192gb but that also comes at an insane cost, if AI is going to become mainstream and Apple want to do this on device, the base models are going to need more Ram, processing power on these systems is not the bottleneck, the RAM is.
The “stopgap” is using the existing M2 Ultra for this, versus a chip specifically designed for AI server duty.
I don’t think the person you responded to is talking about on-device processing. Apple definitely wants to do as much of this on-device as they can of course, that’s just kind of tangential to this discussion on which server chips they are using.
I'm talking about more normal machines, all have 8gb including 2 out of the 3 MacBook Pro models, meaning you have to drop two grand for a MacBook Pro with 16gb memory and even that is low for AI.
They are using M2 Ultra on the servers because that has enough power / memory, the issue is there baseline machines don't and only have 8gb, hence needing to offload to a server and it's going to be a stop gap until they put more ram in the standard machines.