Seen this behavior and thought it was a browser-specific bug since it seemed to appear and disappear based on which browser I was using. Definitely an anti-feature. Most annoying.
Now if it could just forget I finished watching* something and removing it from the continue list automatically.
* Finished watching generally means fell asleep watching for me.