[edit] Oops, the Spark is $4,000, only the Ascent is at $3k now. Strix Halo systems vary from slightly slower (6%) to the same (on systems with LPDDR5x-8533, like the HP laptop).
> NVIDIA ConnectX-7 NIC these days often sells for $1500-2200 in single unit quantities, depending on the features and supply of the parts. At $2999 for a system with this buit-in that is awesome.
My naive analysis is: A high end Mac should be able to run each layer of an AI task about twice as fast because of the memory bandwidth. And the data going between layers is tiny enough to run over thunderbolt or even normal ethernet.
Is there an AI use case that prefers 250GB/s memory bandwidth plus 25GB/s interconnect over 500GB/s memory and 2GB/s interconnect? Are there other major use cases that prefer it?
For inference you can probably get by with 2GB/s assuming you can split the layers up nicely.
The interconnect can be a bottleneck for inference but only for networks with loads of activations and large batch sizes, or if you are doing tensor level parallelism.
So the bandwidth is dead even between AMD and Spark.