I recently ran across a similar tool for aligning subtitles based on a reference subtitle file called aligner:
https://github.com/kaegi/aligner
It seemed to work pretty well in my testing. Maybe it could be useful to look at?
https://github.com/kaegi/aligner
It seemed to work pretty well in my testing. Maybe it could be useful to look at?
Either way, I also thought this was a cool project and will be thinking more about whether any ideas can be borrowed.