Hacker News Reader

Comment by joezydeco | Hacker News Reader

I'm a huge fan of Apple's refurb store. But it can be a cat-and-mouse game because as stuff comes in and out of stock, the website will change minute by minute.

There is a site called refurb-tracker.com that puts out an RSS feed based on changes to the store inventory. I used some tool to turn that into a twitter feed @MacRefurb.

It's been running so long I don't even remember how to log in and change it. And I forget what tool I used to make the RSS->twitter conversion. But I still have 185 followers, 4 years later...

View on HN

And for this reason I wrote myself a script to wget the Macbook pro 15" refurb page, and parse it for: Price, manufacture year, and matte screen option. If it matched my criteria 3/3, then it would email me.

One morning I got that email and I no longer had an excuse :D

I like what you did there, great stuff! I do wonder how many personal bots are out there, scraping sites for individuals' own gain like this.

I remember a good few years ago, getting frustrated at having to wait six weeks for a driving test. I wrote a little script that ran every half hour, logged into the DVLA's site (in the UK), entered my details, and looked for recent cancellations. If it found one, it sent me the details, and I'd madly scrabble together to try and book the slot. After I failed my first test, it was even more useful :)

I used to use Yahoo! Pipes to scrape every major craigslist location in California when I was on the hunt for a certain model of car or motorcycle. I wasn't the only one, either, because eventually the RSS service started getting shut down every month when some arbitrary RSS bandwidth number had been exceeded.

It was beautiful, transitioning from sifting through CL by hand, to having carefully filtered ads delivered to my inbox within an hour of being posted.

I use IFTTT.com to do the same thing on CL.

I know I do it all of the time. Often I connect it to a twitter bot as well so others can use the information. Using python with tweepy and twill makes it very easy. Once I set up a script I just run it in cron every 15-30 min and forget about it.

I've done this too, scraping a doctor's site with flu shot information back when we had a flu scare a few years ago - my wife was worried about missing the notification for our daughter, so I whipped up something to ping me when more shots became available.

I've used Yahoo! Pipes and Boxcar.io iPhone app similarly to parse RSS feeds to find a fridge-freezer on Freecycle and a good apartment.

Reply on news.ycombinator.com