I had a verbatim transcript already, and I used CMU Sphinx to do the alignment between that and the recording.

With raw audio only, I'd probably use a transcription service that provides timings, or software like Trint to do machine learning first, and then clean up by hand, like you did.

The software I wrote to handle the playback is here: https://github.com/vitorio/hyperaudio-lite