I struggle with the same, and I have exactly the same thought process.

Occasionally, I have a filtering session when I am going through everything and narrowing them down, however extra items are appearing until the next filtering session and the same goes again. It is a vicious cycle.