Hacker News Reader

Comment by relativeadv | Hacker News Reader

"The simplest explanation is that they saw this coming a mile away (who honestly didn't?) and this is engineered martyrdom."

I think its more boring than that. Their platform was built by amateurs who didn't know what they were doing. Things like soft-deleting posts and auto-incrementing post IDs. This is why it was so easily crawled in a day before the shutdown.

https://arstechnica.com/information-technology/2021/01/parle...

View on HN

This is the real answer. It was very clear they had no competence to make an app of this scale. It would have collapsed under its own weight before long.

I mean, Reddit was pretty poorly built in the early days prior to getting popular and went down frequently due to load.

It's easy to blame incompetence, but startups are startups and many things are hacked together in a hurry when you're growing fast.

Off topic but what's wrong with using auto-increment?

The most common criticism I'm aware of is that it leaks information about your system and its users. For example, you can gauge growth by how fast ids grow, and relative age of accounts. The former is potentially financially valuable. Often the latter is exposed properly through the UI, but the that can be a choice, rather than exposed due to an implementation detail. There are likely other criticisms as well.

https://en.wikipedia.org/wiki/German_tank_problem#Historical...

That's a great example, isn't it? :)

Auto-incrementing ids makes guessing subsequent or previous items trivial. There was a post here a few weeks ago about it -- https://news.ycombinator.com/item?id=25244872

That doesn't seem like it would matter much on its own. You can pretty easily guess HN item IDs, too. e.g. https://news.ycombinator.com/item?id=25244871 and https://news.ycombinator.com/item?id=25244873

Of course you’re right — nothing matters on its own. It’s one possible factor in a defense in depth strategy —- https://en.wikipedia.org/wiki/Defense_in_depth_(computing)

It makes it easy to crawl. If you're trying to get all of the user profiles for a given website and all of the URLs look like foo.com/user/123 it's easy to download all of them, whereas foo.com/user/1234-abcdef1234-1234 is much harder to guess.

You can use auto-increment IDs as your internal keys, but don't expose them as external identifiers.

There's nothing particularly wrong about it. Twitter does something like this with a bit of timing thrown in the mix, which is why some smart-asses have made self-referencing tweets:

https://www.spinellis.gr/blog/20090805/

I believe they also for a long time had a public API^.. and I'm pretty sure they soft-delete posts (bonus that they could revive the tweet down the track if it was a 'mistake'). I don't think the Twitter team are amateurs though.

Having monotonically increasing IDs for non-distributed systems removes the possibility of duplicate IDs. In distributed systems ID conflicts are unavoidable, and you might have less conflicts if you generate random IDs, as well as it becomes difficult for an attacker to guess IDs.

^ likely it would have been rate-limited, which isn't a problem if you've got a large pool of users you can authenticate with

> Having monotonically increasing IDs for non-distributed systems removes the possibility of duplicate IDs.

In parler's case, having monotonically increasing ID typed as signed 32-bit integer _guaranteed_ duplicate IDs once they reached 2.1 billion unique items. This is well-known failure mode, and yet it took them by surprise.

https://twitter.com/adrianbowyer/status/1348761214428569602

Okay.. I concede, 32-bit is a rookie mistake. I didn't know that. Wow. Did they not account for millions of daily active users in a Twitter competitor...

> what's wrong with using auto-increment?

1) It invites resource enumeration attack

https://www.cequence.ai/use-cases/enumeration-attacks/

https://owasp.org/www-community/attacks/Forced_browsing

2) they ran out of ints. Stop me if you think you've heard this one before

https://twitter.com/sarahmei/status/1348474968527360001

https://twitter.com/adrianbowyer/status/1348761214428569602

Reply on news.ycombinator.com