Last night I was running run_translator.py script and found that their scripts not actually allow people training models from scratch.
But hey, I was able to read the code, fix that small thing that needed to work for my case and ran my experiment.
I could never do that in AllenNLP. Go figure.