You compared each model fixed number of times (images/sec). What was the training accuracy difference between FP32 and FP16? Do they match?
I would like find out the efficiency gain when training he same model to same accuracy with different floating points types and the same model size.
Mode | Top1 accuracy, % | Top5 accuracy, %
-------------------------+------------------+---------------------
Fp32 | 58.62 | 81.25
Mixed precision training | 58.12 | 80.71
FP16 training | 54.89 | 78.12