Hacker News Reader

Comment by pwf | Hacker News Reader

It seems like this will use up a crapton of bandwidth with its 'push' style notifications. If someone with a million Facebook followers makes a new post, one entry is made in the database and then users pull it down as they visit their own pages.

If a million people decide to 'camp in my tent' (?), my server is suddenly pushing out gigs of data every time I make a post.

View on HN

Status.net/OStatus uses a similar PUSH model, basically it scales with number of followers, and yes, if you have thousands of followers the load will be significant. As a data point, I've been running a status.net instance since last December, during which time I've had 50-80* followers, have made an average of 12 posts/day, have followed 100-190* others, and my instance has transferred a grand total of 10,279MB * over that entire period.

followers and followeds gradually increased over the duration. I have this number handy since the connection is reverse-proxied via pagekite.net, which is metered. Over the last few months, I've had all static content offloaded to another server, which reduced the bandwith used, I'm not sure by how much.

You definitely seem like what I would consider a light user then. My wife has 1000+ Facebook friends, and posts 15-20 times a day. If I round to 1GB/mo in your case, then she would probably use around 20GB/mo... which is actually not bad.

Thank you for the numbers!

Even worse are pictures and videos. Rather than including a URL to it, the 'push' includes the base64 encoded binary data. Hmm.

Posts have different "views" one of which is meta which would just have a URL pointing to the photo instead of including it in the post. See: http://tent.io/docs/post-types

Ah ha, I missed that. Good work!

Good, I was going to say, after evaluating PuSH I would have assumed that some things can be heavy pushes and other things can be light pushes.

We anticipate follower counts similar to the levels seen on Twitter today. Managing 1 few hundred followers isn't too difficult, thousands can be managed by cranking up the dynos on heroku. But yeah, Justin Bieber is going to need a hosted solution to handle that kind of load (like he has now with Twitter, YouTube, etc).

We also have a setting in subscriber settings that control which types of posts are pushed in their entirety to different followers vs just a notification being sent vs nothing at all. So blog entries and status updates might get pushed to all 1M, but you probably wouldn't push an HD video update to all of them without some more serious server architecture.

What arguments do you have against a 'pull' style for notifications? Something like a GET to /latest that passes in the last time the client checked for updates and only gets back the URI of the last 10 or so posts since then. An HD video post could contain simply a URI that points to the video, so the video isn't even downloaded until the 'app' within tent asks for it.

This way you could have a path with all the videos on it that you could proxy off to a server with more bandwidth. It would get the user a lot more control over how their content is accessed.

Edit: And what if my server is down when one of my friends makes a post? Will I never see it?

Pull-style requests actually use up quite a bit more bandwidth than push notifications, because every client and server has to query every few seconds instead of just getting notified when there is a new post.

Posts have 'views', so a video or photo post would be pushed out with the 'meta' view which would include a URL to the content instead of the content inline.

I don't think so, considering that not all of my followers are going to be using the service all of the time. If the server just blindly downloaded everything always, then yes, it would be more bandwidth. But with proper caching strategies and servers only requesting new posts when clients are active, I think you could save a lot of bandwidth by not pushing everything.

While the example of Justin Bieber makes it seem to make sense to push the cost of following to the followed rather than the follower, I am not at all sure this makes sense in the general case. It puts the user at the mercy of spammers & bots and ISTM making the follower bear the cost of following would disincentivize this kind of bad behaviour.

What about allow a publisher to delegate a subscription endpoint via something like pubsubhubub? Then publisher just has to push to the aggregator, everybody does their pulls from them.

That only moves the load problem, doesn't solve it - given a decentralised system you're likely to be responsible for load on the aggregator server as well.

Well, in a decentralized social network you'd have to make sure the tentpole you are living under is able to hold the weight.

This is decentralized. In my expectation this network is not about a few celebs having millions of followers. I'd love to see it become peer to peer on the human level. Friends who are actually real friends, or at least people you have met and have had a real human interaction with.

Reply on news.ycombinator.com