Thanks a lot for sharing. Haven't tested Strix Halo myself. Did you consider DGX Spark as well?
What is your target use case? Curious what feels suboptimal about llama.cpp + Vulkan so far.
What is your target use case? Curious what feels suboptimal about llama.cpp + Vulkan so far.
I do lots of different experiments, synthetic data generation along the lines of Magpie is one of the things I wanted a local machine for, as well as just general access to a decent sized LLM to try different things, without having to spin up a cloud machine each time.
I would prefer PyTorch / HF transformers to llama.cpp as I fine the latter less flexible if I want to change anything.