Interesting no mention or discussion of FPGAs for DL Neural networks.

"Our enhanced NPU on Stratix 10 NX delivers 24× and 12× higher core compute performance on average compared to the T4 and V100 GPUs at batch-6, despite the smaller NX die size."

"Results show that the Stratix 10 NX NPU running batch 6 inference achieves 12-16× and 8-12× higher average energy efficiency (i.e. TOPS/Watt) on the studied workloads compared to the T4 and V100 GPUs, respectively."

https://users.ece.cmu.edu/~jhoe/distribution/2020/fpt2020.pd...