Part of the problem we have is some of those desktop GPUs don't have power and flops data. We're toying with the idea of writing a CLI to run/publish these but obviously a lot of work.
Cheers for engaging!
https://en.wikipedia.org/wiki/List_of_Nvidia_graphics_proces...
If you know what any given model uses for BW per token and flops per token, you can roughly infer for many models until you get better benchmarks.
There is also:
CUDA: https://github.com/ggml-org/llama.cpp/discussions/15013
HIP: https://github.com/ggml-org/llama.cpp/discussions/15021
Vulkan: https://github.com/ggml-org/llama.cpp/discussions/10879
Metal: https://github.com/ggml-org/llama.cpp/discussions/4167
These three are measured against the old model of Llama 2 7B, but it also means the numbers are without any per-model quirks that reduce compute and memory BW usage, thus more or less generically applicable.