However, I still feel kinda nostalgic for those warm summer days, writing my first autoencoder in Theano and using it to make blurry versions of photos.
-----
0. Theano had some truly gnarly exception tracebacks, which made it difficult to identify if you'd made a truly incompatible model or if one of the layers/operations just needed a datatype specified.
1. I wish more companies did something like this. It's not necessarily innovation, so much as having a corps of people experienced with some wondrous but unwieldy technology and then giving those people the opportunity to reimplement it "right". Maybe there are many companies that attempt it, but instead produce a clone with problems of its own? Or maybe you need someone like Jeff Dean supervising the project to make sure everything comes together?