Hacker News Reader

Comment by apeacox | Hacker News Reader

I wonder how 2000 servers can host 800.000 users. It means that, on average, a server can host 400 users. I really appreciate this project for several reasons, but it (sadly) looks like a waste of resources with these numbers.

View on HN

Does, not can. If you look at lists, there are tons of tiny instances: experiments, some people using their own instance as their private homepage, or just one for a few friends. People being able to do that is a strength of a federated system, and it's far from clear it actually is "wasting" something without looking into it in a lot of detail. It's not like all of these instances sync completely with each other.

The biggest instance has 250k users, and the top 10 instances together have 600k user (75% of of those 800k). I don't have info how these large instances map to actual servers, but I don't think Mastodon currently has a lot in the way of features to split an instance across machines.

(Data from https://instances.social)

You can split Mastodon to several servers easily. Postgres, Redis, Sidekiq, Streaming API, Puma, Nginx and Media Storage can all be placed on different servers. Even multiples for each. You can read here about https://medium.com/@Gargron/scaling-mastodon-1becde463090 how Eugen had Mastodon.social running a few months ago.

Systems like Twitter, Facebook, etc use tons of resources scaling up. There may be some duplication amongst it but spreading things out also distributes much of the load.

Has the internet really regressed so much that developers would also make the argument that it's a better idea to have one or two email providers, for instance, than have it be a distributed system? What about many wordpress instances, etc?

Instances are not equivalent to servers. Just because you have 2000 instances, you don't need 2000 servers to run it. Just like you don't have a single server per WordPress install.

For example, I have 100 Mastodon instances running using 8 VMs at https://masto.host/

I can tell you that a VM with 2vCPUs can run 30 small instances easily (CPU load average under 50%) as long as you have 1GB of RAM for each.

So, if you have a server running, I am sure you can add Mastodon to it and the only thing you will notice is the 1GB of RAM usage. If you want to install it for a small group of users, just find a sharing hosting space like mine or go with a $3 VPS and you will be more than fine.

Even if you can potentially run several instances per server (with enough ram), I think that an usable one should run on a dedicated server. Considering that a Rails app doesn't run like a webserver with php installed, just like you can do with WordPress or php apps in general. Also, I suppose 1gb is required to run the instance, but it isn't known how many users can host decently.

I think I understand what you are saying but depending on the use case it is not only usable but it runs great.

I have instances with a couple of thousands of users and it runs smoothly with shared resources.

It really depends on what you want to build. If you want to create an instance for a small specific group of people or you want to open it to the masses. I haven't tested concurrent users but I can tell you from all the instances I run, nobody ever mentioned an issue of slowing down when experiencing heavy traffic.

I have noticed that cPanel has been pushing RoR https://documentation.cpanel.net/display/ALD/Ruby+on+Rails and I haven't tested it or know of the security implications but I am pretty sure that with enough tweaking one could install Mastodon on a cPanel server with other apps running in PHP.

Anybody can open up a server. The bulk of users are going to sit on the first few hundred servers (I imagine there's a handful of popular servers for each language) and then the rest are just instances that people host because they can or for their group of friends to hang out on.

  (I imagine there's a handful of popular 
  servers for each language)

As far as I can see, there's a handful of servers, full stop. Some level of criticality exists and is yet to be achieved here.

Edit: gosh no, I'm completely wrong. I looked at the instance list at https://joinmastodon.org/ but didn't see there was a scrollbar!

Are you trying to say that there are too many or too few users on each server?

Either way, 1. If you take out the 5 largest servers, the average drops significantly. I don't have the numbers available now, but the average may go to 50 without the top 5 largest servers. 2. The biggest barrier to getting more people to set up servers is that it requires a lot of resources (a.k.a. more money going to your favorite VPS provider), with the reputation of a fiddly install being a close second.

You don't need a lot of resources to set up an instance. Mastodon runs easily with less than 1GB of RAM and 1vCPU. Also, you can even install it in a shared server, like you do say a WordPress. I have a hosting service dedicated to Mastodon https://masto.host/ where I host around 100 instances in shared environment in a Cloud of 8 VM.

Reply on news.ycombinator.com