You need at least 10x over an established approach to have any hope of surviving in the mid-term. The results are for batch-size 1 as well. GPUs are throughput optimized. So I wouldn't be surprised if an A100 chip would be able to outperform this for batch size >4. I still think this is a really cool architecture though.