Hacker News Reader

Comment by chris_wot | Hacker News Reader

I don't think WMF staff are credited enough for the work they do in keeping Wikipedia running. They seriously know how to scale, I think the only ones better than them are honestly Facebook and Twitter!

View on HN

I don't know if I would go that far, but WMF also don't have nearly the resources of Facebook or Google or Amazon, and at their scale nothing is easy anymore.

Google as well?

You know, Google are so much in my life I don't even notice them!

Google beats everyone :-) they are so far ahead that I didn't even consider mentioning them!

Can't caching a page with varnish and memcache handle this?

They cache the bejeezus out of their pages. Problems come up when a lot of people want to edit a page, an inherently uncacheable operation.

It's possible to use stacks to 'cache' writes in scenarios like this.

Writes to the same object go in the same stack, iterate over stacks, pop the first item, write it, clear the stack.

It works miracles for ephemeral data like wikipedia edits.

If you have extremely spikey load on servers, stacks are also a great replacement for queues, admit that during the deluge some portion of queries will timeout and go unanswered, instead of trying to process queries that are likely to timeout, simply process the first query on the stack and don't waste time processing the ones bound to fail.

Doesn't caching writes from multiple different servers potentially cause consistency and durability concerns?

I know MongoDB still haven't marked the bug [1] reported by Kyle Kingsbury [2] that found stale reads on all consistency and write concern levels...

1. https://jira.mongodb.org/plugins/servlet/mobile#issue/SERVER...

2. https://aphyr.com/posts/322-jepsen-mongodb-stale-reads

Yes, but this is wikipedia, the entire premise of it is eventual consistency. The idea being that the most recent update to a page is the correct one.

Good point.

So... did you read the article?

Varnish does handle the mass of logged out cached requests. However, because they were having such a high amount of page edits per second the cache in Varnish would only by valid for about a second. Then a flood of logged out users hit the servers at the same requesting the uncached page to be rendered. The PoolCounter extensions keeps the web servers under control and by throttling requests for page rendering.

Correct me if I am wrong, but I thought Varnish has support for limiting concurrent backend fetches to the same resource.

Across a cluster of varnishes? I think that limit is per varnish.

No, definitely not across a cluster (although that would be quite nifty). Even on a single node that would reduce the thundering herd effect substantially.

Maybe that's a new feature request!

Why not just set the min TTL to a few minutes, or even a minute, for anonymous users? Is there enough usefulness in ~seconds of delay on article edits vs minutes to warrant a much more complex design?

Reply on news.ycombinator.com