Comment by dragontamer | Hacker News Reader

Comment by dragontamer | Hacker News Reader

Lemmy has become one of the major "lifeboats" during the #RedditBlackout even this past week. This has lead to interesting performance issues and questions... as well as human-level scaling issues.

On the technical side, this https://programming.dev thread (itself, a Lemmy instance) asks how Rust may have affected Lemmy. The ongoing discussion covers a wide variety of topics.

The #1 post right now is frome snowe@programming.dev, the owner of this instance. He took disk-measurements of the server and published them, demonstrating that Lemmy's current bottleneck seems to be RAM (proven by the change in Swapping behavior / Disk-io before-and-after he upgraded RAM).

--------

I do think you can see the benefits of federation in this manner. Users from many instances: https://kbin.social, https://lemmy.world, and https://beehaw.org... and more (all different Lemmy instances) have joined in the discussion and have made a high-quality thread.

---------

Note: I am bullish on federation / Fediverse after playing around with it this past week. I'm less bullish on Lemmy itself, which has proven to be a buggy, laggy mess. Still, there's good discussion happening on Lemmy now (albeit slower due to fewer users). Communities are springing up (and will be dying), and I'm sure instances will spin up and die too. But the idea of federation works, especially for bringing together the discussion.

I think work (and play) needs to happen on Lemmy, kbin, Mastodon, and other Fediverse instances to figure out what is the best software, and best experience for users. Its not quite ready for prime-time, but the core feature set / Minimum-viable-product demo here is extremely exciting to me.

> Lemmy's current bottleneck seems to be RAM

Rust seems like a sound choice in that regard because it's not garbage collected, so you're not wasting some RAM overhead just to avoid needing to do extra work, and it doesn't have C-style struct packing issues. If you don't specify the layout you need for a user defined type, Rust will assume you are OK with it re-arranging your data structure to be smaller / faster.

If you're willing to put in some work you can often go further with this in Rust than in other "systems programming" languages too. For example C++ Strings have the "Small String optimisation", exactly how this works varies but e.g libc++ (on a modern 64-bit server) String is a 24 byte data structure which can hold up to 22 bytes of textual data inline without yet needing a heap allocation. So if you do a lot of work modifying small amounts of text, Rust's built-in String isn't as efficient as a libc++ String, but the CompactString type from a popular Rust crate is also a 24 byte data structure yet it holds a more impressive 24 bytes of text before needing heap allocation.

What happens with the content if posted on an instance that shuts down? Will it be kept alive by other instances?

This has already happened in practice, as https://beehaw.org and https://lemmy.world defederated due to some (hopefully temporary) issues.

I don't wanna go into the drama / discussion here but lets just call it some nerd drama + lack of moderation tools that can solve the issue. (Beehaw.org is very confident that if Mastodon-like tools existed, they'd be able to refederate and get the things they want. The issue is being blamed right now on Lemmy's relatively new codebase that's missing some features that Mastodon users are familiar with)

https://Lemmy.world is continues to host the copies of the "old Beehaw.org" material, allowing Lemmy.world users to talk. But the "federation" has shut down, you can only talk with local users (because Beehaw.org serves as the "True copy". So upvotes / downvotes, and federated messages/comments are coordinated by the Beehaw.org server)

In practice, this means that all the Beehaw.org communities has become kinda-sorta useless on https://Lemmy.world. But, the copies of the conversations (up to the point of defederation) do exist and remain on your local (erm... more-local?) server.

It might still remain in the cache of another instance, but there is really no permanence build into the Fediverse. If an instance goes down, its content and user accounts go down too. There is no quick&easy way to migrate content to another host. There are no crypt-ids (e.g. PGP keys), everything is refered to as user@host, even message-ids aren't even globally unique, e.g. these two refer to the same post:

* https://beehaw.org/post/598023

* https://feddit.de/post/881008

Fediverse still has a lot of work to, right now it feels more like a quick hack than something that could significantly improve the Internet in the long run. If an instance goes malicious, you have all the same problems as you'd have with an unfederated server, after all federation is completely optional in the Fediverse.

Just wanted to pick up on this minor point:

> current bottleneck seems to be RAM

RAM exhaustion is often a secondary side effect of the true underlying bottleneck.

It could be that RAM used for temporarily processing an HTTP request, say, would be freed quicker if the database returned sooner. Maybe the database is the actual bottleneck. And the actual root cause is an indexing / partitioning issue.

Reply on news.ycombinator.com