Hacker News Reader

This is doing inference on the CPU, right?

On the mac, we have enabled Metal support.

What is the best way to confirm that the GPU is being utilized? I can open Activity Monitor and see the usage spike, but surely there's a better/direct way.

Try asitop. For me it doesn't indicate anything but CPU is in use.

https://github.com/tlkh/asitop

Reply on news.ycombinator.com