I can run 10 GPUs on my model training runs and finish in an hour now when they used to take at least 2 or 3 days, and it took absolutely no work on my end. It’s been absolutely wonderful for my productivity. The price doesn’t matter either because compared to how much the people are being paid at these sorts of companies, it really doesn’t matter. The boost in efficiency is so much more important.
I’m pretty sure for the next Tensorflow 3.0 update, they’re rethinking towards a Pytorch style with more dynamic computational graphs.
Do they know if tensor cores in the these processors are just good for interference or are they similar to the more pricey models (floating point precision)?