How do you do the collection per paper? Is that manual or automatable?
This is probably an automatable task, but it's generally not too much of a burden to do manually. I add a 'needs-review' tag to papers I've identified in a literature review, and categorize it when I get around to reading it.