This is a very impressive work. I am wondering why it was compared against PyTorch (just on the CPU) and not against Tensorflow (on CPU and TPU) or do I miss something?
I wonder if the point of this article is that other tools can be used to access TPUs other than the nominal Tensorflow-based workflow. I.e. the article is to show that another workflow exists to access the capabilities of the TPU, which are (to some degree at least) already known. Keno Fischer says [1] they'll have other benchmarks out soon.
[1] https://twitter.com/KenoFischer/status/1054932277824774145