This, not to mention one could get the GPU usage on the V100 way higher by training with larger batch sizes, which would also make training much faster.