Thanks! Having the click seek to it was my ideal. Unfortunately the player is an iframe, so I'm not sure how to hijack that. I could just do HTML5 audio or something, but then we don't get the benefit of Simplecast's analytics.
We're about to push a fix for this. It will also work with URLs, so you can go to /episode-title/#1:23 and it'll go there. Makes sense to share a particular question and answer with someone.