> Do you think there's any other architecture that would've had better results -while still not taking too long to train-?

Yes! A 5th or 6th order word level Markov chain with smoothing.