Yet in my experience PyTorch is much nicer to use. Its API feels very natural and easy to extend -- it feels very Pythonic.

TensorFlow's API, on the other hand, seems to get in my way whenever I try to do anything new that isn't already built into one of its higher-level APIs (e.g., Keras). I frequently I find myself fighting with TensorFlow's API.

For iterative R&D/exploratory work, I find I'm more productive -- and happier -- with PyTorch than TensorFlow.

What TensorFlow should do instead: do dataflow-analysis, like any modern compiler, and figure out the dataflow-graphs at compile-time.

OR... take PyTorch's approach and use dynamic graphs. I bet there's not even a significant performance penalty associated with dynamic graphs, as the tensors are usually quite large and consume most of the computation-time anyway.

My point: why use a tool that's founded on a wrong design-decision?