Hacker News Reader

Comment by josephwegner | Hacker News Reader

I really appreciate giving a 3 week heads up before moving to a new frontend structure. It's a nice gesture, but I have this horrible feeling that there's only about a 10% chance that my Hacker News app gets updated in time.

I know you can't not iterate because people are scraping, but it does stink. At least this will make everything more future-proof going forward.

However, it may be nice to give a bit more heads up than 3 weeks. I know a lot of apps can take ~2 weeks to get through the review process for iOS.

View on HN

Some sites actively punish scrapers by constantly, purposely changing their markup. So giving them an API and 3 week head start is leaps and bounds above and beyond what can be expected. When you operate a scraper, you are always on the defensive when it comes to site updates suddenly breaking your app. They should be so lucky to get this 3 week notice. It is all on them if they can't turn it around in time.

As someone who poured countless hours into meticulously scraping the HN markup and faces the prospect of having to port all my code with dread, I'll probably be pleading for an extension alongside you.

A preview release / staging version would help those of us with scrapers update it, without having a so much downtime / scramble when it's finally released.

Maybe they can put the new renderer on a different sub domain for a few weeks after their 3 week deadline. One to beta test the interface and two to give devs a bit more time to convert.

We'll probably start with logged out home to a percentage of traffic and work our way to updating the rest of the site and handling the full traffic over a few weeks.

If people need help converting their apps, I believe the Firebase guys have offered to help. Contact us at api@ycombinator.com and we'll connect you.

Love the staged roll out idea. You could probably also use user agents to recognize scrapers from real users (could also be helpful for slowly rolling out to older browsers).

Might be easier/quicker to use the API to generate your own HN clone with HTML in it's current format, then point your App at that

Reply on news.ycombinator.com