This was reassuring to hear from someone else, because I've had this exact feeling about books I read, films I've watched, conversations I've had, work projects I've completed, etc. This is true even in cases when I was completely engaged in, for example, reading the book, and the book left a positive impression on me.
I've always felt guilty about this, especially when I see others who don't seem to have the same problem when they talk about the books they've read, etc. I've also found that recall can be greatly improved by repeatedly talking about the specific topic with multiple people.
The strange thing is that I have an excellent memory for certain things - information about people and relationships. In light of our evolutionary history as a social species, perhaps this is not so surprising after all.