At this pricing level, with this amount of RAM, I suspect a lot of use cases will be with ML and GenAI. Benchmarks for these use cases would have been interesting.
If you're making, say, an ML-based on-premise CCTV system and you need to run several large ResNets at the same time? And you don't want to go rack-mounted, as some sites don't have a data centre? And you want the longer lifecycle and guaranteed spare parts availability of an enterprise product line? This could be the card for you.
Admittedly it's a rip-off, but the Workstation/Quadro line always has been.
Honestly I'm not sure how healthy the workstation market is right now - with the rise of work-from-home and hybrid working, I don't see many people using huge desktops any more. And when Adobe puts a powerful generative AI feature into Photoshop, they don't expect users to upgrade to powerful GPUs - they run it in the cloud, so it works for users with puny GPUs and Adobe can get that sweet sweet recurring revenue.
"Customer may not [...] provide commercial hosting services with the SOFTWARE. [...] The SOFTWARE is not licensed for datacenter deployment, except that blockchain processing in a datacenter is permitted."
So if you were e.g. an ad agency artist using a 4090 at your desk to generate images for commercial use - that's fine, because it's not in a datacenter, and the commercial services you're providing with it aren't hosting.
I've never heard of nvidia taking any enforcement action, or defining precisely when an office becomes a datacenter - I suspect this is mostly to ensure the big cloud providers don't offer 4090s.
Reading my comment again, obviously it wrong and I should have been more careful with the wording.
The worst thing that can happen is Nvidia declining any warranty repairs on these cards as they've been used outside of the intended use. But it's definitely not a legal issue.
And EULAs in general are just not enforceable outside of the US.
When I tested the A6000 against the H100, there wasn’t that big of a boost from the newer card. Perhaps GPU operations weren’t the bottleneck in that case.
Yes, but the point of a review with benchmarks is that it is expensive and time-consuming for a customer to acquire the hardware just to run their own benchmarks on.
Stable Diffusion and various LLMs are available pretty easily.
A simple benchmark that this version of stable diffusion/llm was used with these settings and this is how long it took to produce image/we got this many tokens/sec would be a nice comparison that you are in a good position to do with access to all the hardware.
And anyways, it’s not that expensive. You can rent the same gpu from cloud providers for a few dollars per hour. If you are serious about buying a GPU it is an extremely small cost in terms of time and money compared to the price of the gpu itself.
The alternative is trusting an online review running some training or inference code which is likely not comparable to what most people are doing.
The big three cloud providers don't offer things like the RTX 5000 32GB or the 4090 do they?
AWS will happily rent me a H100, A100, V100, K80, A10G, T4, or M60. Not to mention a Trainium, Inferentia, Inferentia2, Gaudi or Qualcomm AI 100.
And don't forget to benchmark each one of those in a 1, 2, 4, and 8 GPU configuration; and with a variety of batch sizes; and with and without distributed training. Remember to work out the performance-per-dollar for on-demand, reserved, and spot instances, times three different cloud providers. Now, to compare to on-premise pricing we start by calling our air conditioning and backup generator vendors...
Starting a VM with a cloud provider you already know and use may be extremely fast, but adopting a new cloud provider isn't.
Even if your organisation is so unbureaucratic you can get a new provider set up without any due diligence work, every cloud provider comes with their own oddities. Will they have a wacky set of default firewall rules? Will they only offer Debian not Ubuntu, if you want an instance with the CUDA drivers already installed? Will they insist you learn what a 'provisioned iops' is before letting you start an instance? Will some GPUs only be available in certain regions? Will accounts have quota limits that vary by region, availability zone and GPU type, some of which default to zero?