Hacker News Reader

Comment by paxys | Hacker News Reader

The author paints it as some kind of conspiracy, but the simple truth is that RSS waned in popularity once push notifications became popular and widespread. Every user's RSS reader polling every website for changes every few minutes for all eternity is horribly inefficient and unfeasible to do on most platforms and devices today (especially on mobile). Pub/sub is the clear way forward, but it's just sad that there is no accepted standard for it.

View on HN

>Every user's RSS reader polling every website for changes every few minutes for all eternity is horribly inefficient and unfeasible to do on most platforms and devices today.

1) It's not that inefficient, especially if you recalculate your feed only when it changes and use HTTP statuses (particularly 304 Not Modified) correctly.

2) Most people then and now would go through an aggregator. So they'd really be talking to that instead of the source feed.

3) There were push notification standards out well over a decade ago, including RSSCloud and PubSubHubBub.

> 1) It's not that inefficient, especially if you recalculate your feed only when it changes and use HTTP statuses (particularly 304 Not Modified) correctly.

Besides, assuming you're not hideously overcomplicating things, you'll just be serving a static file off disk. If your server is a raspberry pi, you ought to still be able to serve some 1000 requests/second if that is what you are doing.

Assuming most RSS readers poll at most once every 15 minutes (which is a really high rate), you'll need at least a million subscribers before you need to get a second raspberry pi.

Here is what I found looking at my server logs a few months ago:

     Feedly: 7min
     Feedbin: 15min
     Bloglovin: 30min
     Dreamwidth Studios: 30min
     Feed Wrangler: 30min
     NewsBlur: 30min
     BazQux: 40min
     inoreader.com: 1hr
     theoldreader.com: 2hr
     pine.blog: 24hr

https://www.jefftk.com/p/looking-at-rss-user-agents

Also worth mentioning the obvious, feedly etc is updating all the users on their platrom with a single call to your site (every 7 minutes in your case). No doubt they scale the update frequency by how popular it is on their platform / frequently updated / however often the ai decides it is needed.

Yeah, I ran a niche aggregator for a few years and we polled ca. 3m feeds a day. Every time a feed had an update since the last time we'd polled it we'd subtract a percentage of the polling interval with a floor of about 5m. Every time there wasn't an update we'd reduce the rate until a ceiling of about 24h. Very simple and no doubt you could do better but better wasn't necessary.

Should also depend on whether a push extension is used, then the reader can fetch the feed less often. The feed of that blog seems to be a plain RSS feed without that.

Everyone used to "ping" sites whose only purpose was to provide feeds of when blogs changed, so all the aggregators would use that as a starting point anyway. But really it isn't that necessary - there are few feeds where getting things to your subscribers that urgently matters enough that you necessarily want them to hammer your site the moment you publish.

I'm not aware of any push functionality in RSS -- something needs to poll the feed, no? Or is there an RSS extension that uses long polling or something?

PubSubHubbub/WebSub (the latter is effectively the slightly modernized W3C standardization of the former) is the standard solution for that. (Feed declares where to register for updates (hubs), readers register their callback URLs with hubs, publishers tell hubs when something changed, hubs run through the registered callbacks and ping them with the update. Thus needs a server-component for the reader side.)

I'd assume that somebody who misses RSS will also be happy with pulling only few times a day and not every few minutes. That's why push became so popular but with a different subset of users. Everybody goes through a push-positive phase but some (like me) come to a conclusion that this is unhealthy and distracting in the long run - hence the fond memories of technology like RSS.

Switching back to Linux after ten years on Mac, I had completely forgotten how much I love it when my computer does _not_ notify me about anything. I have no notification system installed. It’s been a few months and I haven’t missed anything, but I feel more relaxed and focused while using the computer. Notification are mind pollution.

I like notifications on linux. It allows me to know when a livestream starts or when a person I like tweets immediately instead of me only realizing hours later.

Haha, I suppose this illustrates how well Linux can work for every different preference, but both of those sound like exactly what I don’t want at all. Being called to pay attention to something, immediately, because some other person did something, not directly interacting with me, is the antithesis of how I want to spend my time.

>The author paints it as some kind of conspiracy, but the simple truth is that RSS waned in popularity once push notifications became popular and widespread. Every user's RSS reader polling every website for changes every few minutes for all eternity is horribly inefficient and unfeasible to do on most platforms and devices today. Pub/sub is the clear way forward, but it's just sad that there is no accepted standard for it.

Polling isn't a requirement. If your reader is polling it's strictly because the author wanted to implement such a feature; that's it.

There's also ETag and last-modified which are used when checking and have very little overhead (most especially in the case when the content has not changed).

I think it was the sunsetting of Google Reader that triggered the downfall. I don’t think I ever cared about push notifications for feeds. When you have your morning coffee, open up Reeder, refresh the feeds and read through a few articles.

Push notifications have a cost too, now you need the server to maintain state of all clients and notify them on changes. This has huge implications at scale, especially if it's extremely cheap or easy for new clients to spin up. Think about every single browser tab now requiring a server to maintain some state on the backend--that's a nightmare with billions and billions of users and tabs. Pull-based architectures have the nice effect that less popular content has almost no cost to continue serving and just falls into the background forever.

I don't think it was even just a technical issue. Linear feeds of all content just died out to be replaced with sites like Hacker News / Reddit where you have ranked ordering and comments.

As much as HN hates the out of order feeds, that's what the average user wants because most content is boring.

> Every user's RSS reader polling every website for changes every few minutes for all eternity is horribly inefficient

I think you're greatly overestimating this inefficiency. Especially for websites properly implementing the HTTP protocol. 'Push' requires a lot more complexity and it's also outside of the user's control. For fuck's sake, you can just serve a static file. We've been doing this for decades.

Talking about control, my hypothesis is that it was the main driver of the downfall of RSS. It started when Google killed Reader and Facebook took control of what you see in the timeline. Users picking and choosing what/when to read is not compatible with the way the major gatekeepers of information drive engagement.

FWIW my RSS reader is set up to only refresh manually when I open it and pull to refresh, and I like it that way. It's a much nicer way of reading than having everything yelling for my attention all the time.

I've noticed a healthy amount of websites with pretty granular RSS, for example the auto parts outlet rockauto.com has RSS feed for every make, model, and production year that they service.

I'd wager this saves them some unpaid overtime. It seems that a service which provides correct, useful RSS nearly obviates the entire hassle of dealing with bot scrapers and the resource drain and health insurance premiums that come along. You can easily serve any volume of RSS with cloudflare or nginx.

It is quite true that mobile clients scraping a url is inefficient vs pub/sub. Maybe you could just uses mqtt to trigger a GET or something.

I spent about five minutes considering what a world with facebook, instagram, ebay, amazon RSS feeds might look like. The experience felt like goat staring.

I was part of a startup that aggregated items for sale years and years ago (we were sold to LookSmart for a pittance in the end) and we did RSS feeds for everything. Every page spat out a list of items, in fact and the website and the RSS and atom feeds were all generated from the same middle layer. A huge part of our inventory came from RSS feeds too.

That's not true, but might be a common misconception. RSS rode the wave of the realtime hype at that time and has multiple standards for push notifications. RSSCloud und Pubsubhubbub are the two immediately in my mind, note that PuSH got turned into WebSub.

Multiple standards is the same as no standard. Yes there are many ways to do push-based RSS, but there is no single way that everyone uses.

Wordpress does support both. You can make a blog on wordpress.com right now and you get both standards activated by default in the RSS feed. That takes care of half the web ;) (well, it would if it's also true for standalone Wordpress installations, which I don't know). And all relevant readers support at least PuSH. This is really not a standard problem, and I doubt the concept itself is part of the adoption problem.

No, not having a way to subscribe when stumbling over an RSS feed was the main problem in my eyes, and the existence of a feed not being highlighted in browsers anymore. Firefox dropping support was not only an act of treason against the free web, it was also very effective in making it very hard for new users to understand how to use RSS. Not that chrome is any better. Here lies the problem, and that's where it could be solved.

Add into browsers an icon when a site has an RSS feed, and let that point to a foundation managed site that points to RSS readers, where users can select their favorite. I'm 100% certain that would have a huge effect on adoption numbers.

As this comment thread shows very well RSS is not dead at all, but it's not in the mass market right now and that could be changed easily enough.

Push notifications are a replacement for RSS in the same way that cars are a replacement for public transit.

This is a nonsense comparison. RSS is about presentation, not about real time interaction. And it would not be hard to have RSS feeds support push notifications, or other modernization.

RSS is a universal and accessible way of presenting content, that is what is so inconvenient about it to publishers.

The business model of most platforms today has RSS as competition. It completely undermines analytics, and their ability to fine-tune presentation, for the benefit of the end-user.

So of course GM will say that their gas guzzlers make electric trains obsolete.

XMPP is an accepted IETF Internet Standard with PubSub functionality: https://xmpp.org/extensions/xep-0060.html

Sites like Movim (https://movim.eu/) offer an user friendly front-end for this protocol (building upon the ATOM format). You can login using any XMPP account and also self-host Movim instances.

True! Android notifications were even originally built off of gchat and XMPP

I don't agree with your premise or conclusion. Its far easier for a website to just publish a dead XML file that any client can read at their own pace, as frequently as they would like. Also, anyone can create a pub/sub 'middle man' that can implement a push system for clients who want that. You can have the best of both.

I don't see such a high correlation between push and rss. Different scope, purpose, environment and user base.

Reply on news.ycombinator.com