When I tested the A6000 against the H100, there wasn’t that big of a boost from the newer card. Perhaps GPU operations weren’t the bottleneck in that case.
Yes, but the point of a review with benchmarks is that it is expensive and time-consuming for a customer to acquire the hardware just to run their own benchmarks on.
Stable Diffusion and various LLMs are available pretty easily.
A simple benchmark that this version of stable diffusion/llm was used with these settings and this is how long it took to produce image/we got this many tokens/sec would be a nice comparison that you are in a good position to do with access to all the hardware.
And anyways, it’s not that expensive. You can rent the same gpu from cloud providers for a few dollars per hour. If you are serious about buying a GPU it is an extremely small cost in terms of time and money compared to the price of the gpu itself.
The alternative is trusting an online review running some training or inference code which is likely not comparable to what most people are doing.
The big three cloud providers don't offer things like the RTX 5000 32GB or the 4090 do they?
AWS will happily rent me a H100, A100, V100, K80, A10G, T4, or M60. Not to mention a Trainium, Inferentia, Inferentia2, Gaudi or Qualcomm AI 100.
And don't forget to benchmark each one of those in a 1, 2, 4, and 8 GPU configuration; and with a variety of batch sizes; and with and without distributed training. Remember to work out the performance-per-dollar for on-demand, reserved, and spot instances, times three different cloud providers. Now, to compare to on-premise pricing we start by calling our air conditioning and backup generator vendors...
Starting a VM with a cloud provider you already know and use may be extremely fast, but adopting a new cloud provider isn't.
Even if your organisation is so unbureaucratic you can get a new provider set up without any due diligence work, every cloud provider comes with their own oddities. Will they have a wacky set of default firewall rules? Will they only offer Debian not Ubuntu, if you want an instance with the CUDA drivers already installed? Will they insist you learn what a 'provisioned iops' is before letting you start an instance? Will some GPUs only be available in certain regions? Will accounts have quota limits that vary by region, availability zone and GPU type, some of which default to zero?