I have a raspberry pi gathering Reddit comments with an Amazon link in them every hour, then exposing them on Shop By Sub categorized by subreddit, sorted by upvotes.
Same with OP I put my Amazon referral link in there.
It's been decently well-received by Reddit, but I have a hard time getting recurring users. It seems like they just forget about it by the time they need a Reddit recommendation again.
Best of luck in the space OP!
Using the PushShift API, I started backwards in time and searched for a bunch of different regexes I thought were important, slowly creeping my way towards the present.
Using the Reddit API, I wrote a script to do a deep dive into specific subreddits, so I could index more popular ones first.
It's mostly just patience to get the amount of data I've got :) (and PushShift can't be overstated, it's very useful).
A mailing list isn't a bad idea. I'll have to figure out a way where the user can see a sign up box, but have it not be intrusive. I've been thinking of adding infinite scrolling to the most recents on the homepage, too, just for folks who are curious as to what weird things Reddit is talking about