Isn't video prediction a substantially harder problem than text prediction? At least that was the case a couple of years ago with RNNs/LSTMs. Haven't kept up with the research, maybe there's been progress.