You are on to something there. For the syllables I'm actually using a rule-based model from Python's "pyhphen" library: https://pypi.org/project/PyHyphen/

I am not totally happy with the results but have not had a chance to train my own