RNNs in TensorFlow: Practical Guide and Undocumented Features | Hacker News Reader

RNNs in TensorFlow: Practical Guide and Undocumented Features

wildml.com

178 pointstim_sw13 comments

Related - does anyone know of a super basic, from-the-ground up example of RNNs with TensorFlow? Even this is a little more advanced when it comes to RNNs (and I've personally found that the official TensorFlow documentation on RNNs/LSTMs is a little spare in terms of how the official example actually works).

Maybe http://r2rt.com/recurrent-neural-networks-in-tensorflow-i.ht... ?

Among the ones I saw this one is the best http://r2rt.com/recurrent-neural-networks-in-tensorflow-i.ht... The first thing I tried to do in Tensorflow was RNN and I had quite a hard time with other tutorials.

In addition to the other link provided, I've found this blog post to be helpful:

http://mourafiq.com/2016/05/15/predicting-sequences-using-rn...

His code successfully can successfully predict continuous functions, but would this also work for predicting the next value in a time series? If not, what is the best ML-based approach for that?

I tried to use TF to train an RNN recently and ended up using Keras, which is a layer on top of TF/Theano that made it much simpler, and I hear Theano's RNN implementation is better atm anyway.

This is the most straight forward TF example I could find, but it uses google's skflow library: https://github.com/tensorflow/tensorflow/blob/master/tensorf...

One of the things I realized when trying to improve my Keras model is that Keras seems to have a lot of good defaults, so it's easier to get something working quickly, and then dig into all the options, rather than having to start from scratch.

I agree Keras is the easiest way to get into RNNs and deep learning in general.

I would like to start digging into Neural Tensor Networks, will Keras be also a good choice?

I'm not familiar with Neural Tensor Networks, so I'm not super sure. From the bit of reading I did, it seems like they're related to recursive neural networks somehow.

I couldn't find any implementations of either concept on top of Keras, so I probably wouldn't recommend it.

Your best bet is probably to find an existing implementation and study it in whatever language/framework it's in.

Depending on how related recursive neural networks are, I found a few implementations of those in theano, etc, and I'd probably suggest starting there.

If they're not super related, Keras may be a good option, since the learning curve isn't very steep to get started, but you'll probably have to assemble the architecture yourself and since it's an abstraction it will really depend on how easily NTNs can be expressed in that abstraction.

Isn't that the same thing? You use tensors to build a computational graph.

Maybe worthwhile

https://bleedingedgepress.com/tensor-flow-for-machine-intell...

code repo

https://github.com/backstopmedia/tensorflowbook/tree/master/...

seems like a work in progress, I have some quibbles with coding style, and I didn't get far into RNNs... but it is super basic from the ground up, and I got a lot better with TensorFlow.

If anyone is interested in RNNs for Java and Scala, we've documented them here: http://deeplearning4j.org/usingrnns

Are there any tutorials on generative RNN's? Or is that a trivial task once you know how to implement an RNN?

Reply on news.ycombinator.com