Right, especially when it's not clear if such a machine will be needed a year or two from now to run a model with the equivalent performance.
At work I've been successfully running llama2 variants on Mac M2 systems with more than adequate throughput for small experiments and examples. Can also boot a 4x Nvidia T4 instance on AWS (g4dn.12xlarge) to larger models.
I feel you'd need to be doing inference on a 24/7 basis for at least a year to justify the price of this machine.