Hacker News Reader

Would you be so kind to post these results? How much faster, which models etc.

I found some charts here showing A100 roughly doubling a 3090: https://bizon-tech.com/blog/best-gpu-for-deep-learning-rtx-3...

With the 4090 being a fair amount more powerful, it might give the a100 a challenge for about 10% of the cost. That’s still about 10k hours of renting one with this service though.

Sure if you are using tiny net(by today's standard) and small batch sizes, A100 is just twice as fast compared to 3090 due to overhead of framework and memory movement. A100 has 312 FP16 TFlops[1], compared to 40 and 82.6 of 3090 Ti and 4090 respectively[2], and has 3x memory. Also it has 10x inter GPU communication bandwidth.

[1]: https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Cent...

[2]: https://images.nvidia.com/aem-dam/Solutions/Data-Center/l4/n...

I believe you are wrong. The relevant metric is tensor flops and 4090 has 330. I believe 312 you quoted for A100 is tensor flops.

Reply on news.ycombinator.com