Google are aware of the problem and are working on a fix: https://www.reddit.com/r/Chromecast/comments/1j7caui/comment...

Still very annoying.