It's worth noting that any TF user (including those on NVidia Macs) can get significant speedups (20%+) by recompiling to use all the features of your CPU.
TF even warns you about this when you start up.
TF even warns you about this when you start up.
Not all TF components are GPU enabled anyway. The default LSTM isn't and you need to use CuLSTM. That is nice but doesn't support the GoBackwards operation so you can't use it in BiLSTMs.
GPUs are great but there is lots done on the CPU too.
Edit: To clarify, the comments on the LSTM implementations are actually on the Keras/TF LSTM. Not sure what the native TF one is like.