While I’m very excited for an LLM powered Siri that has access to my calendar, email, and messages, it’s worth noting that llama.cpp is using metal for acceleration and the M2 Mac Studio is one of the best values out there for running very large models like llama2 70B derivatives or Falcon 180B. The shared ram/vram allows you to load huge models and it’s got decent tokens/sec performance.
In the last few weeks, the initial prompt evaluation step got much faster with llama.cpp so everything that depends on it like the python library and lmstudio are faster as well. I’m very happy with my purchase at this point and it seems to keep getting better.