Hacker News Reader

Comment by dalore | Hacker News Reader

What if instead of looking up each user in the thread to see if they are a friend, they just serve the same page to all users. Then a bit of javascript loads up your friend list from the server and uses dynamic css to change the style of your friends username to the friend style.

That way everyone can share the cache page, and the dynamic icing is separate.

View on HN

Probably better to send the friend list as JSON down the wire with the page otherwise you'll double the number of requests (not something you usually want to do when trying to scale).

You could use the client-side store (supported in a bunch of common browsers) as a cache.

Or cookies. 4 kB is a lot of friends. Invalidate them every time someone adds a friend.

Both of these however make cache invalidation (in case of multiple browsers/computers) hard—nearly impossible even.

Cookies suck for that sort of thing because they are sent over the wire with every request.

Smart. Still, the db request SELECT name FROM friends WHERE source = %(source) should probably be pretty cheap given proper indexes, and could be done inline, whereas lookups for each person are likely much more expensive. You could still send the JSON in the same request by composing the cached page with the dynamic friendlist. Now all we need is for someone to make it happen.

*Pardon my broken SQL, I don't actually know the language.

What I was suggesting is do the SELECT name FROM friends query as part of the initial page load, not as a secondary AJAX request. Store the results as JSON that gets put into the page, then use JS to manipulate the comments client side. As you noted it should be cheap and as others noted, quite cacheable (friends don't change frequently).

Unfortunately if you are sending the friend list as part of the cached page then you won't be able to use that cached page for all users.

I believe the original comment was that each comments page should be cached globally and then each user can have their friend list & dynamic data fetched separately and added to the page using javascript.

Ah, my bad then. I don't think the comments pages cache well though--the votes are always going up and down. My understanding that in addition to pulling all the comments for each request they were also pulling the friend relationships for the current user with all the participants.

Who cares if the votes are say 1 minute out of date?

Even caching a 2000 comment thread for a minute would significantly speed up that page.

I think it would be confusing to post a comment and have it disappear after a refresh. Same for votes.

The page would be regenerated on each new comment, of course.

Votes should be done the same way friends' lists are: on the client side, via javascript. The cached page with its comments would include Javascript to grab the most current scores for all its posts (one request) and modify the DOM appropriately.

You could if using SSI (say, with nginx or varnish).

The comments page would be cached globally; the json friends list would just be inserted as the cached page is served to the user.

Why would you use JavaScript to request the friends list JSON and then use the DOM to insert a stylesheet? Just CSS would be fine if you're changing the color of friends' usernames and not doing anything that needs JavaScript.

The idea is to remove the part of loading friends from the page loading. By delay the friends JS to run only after the page is loaded, the page loads faster, and can be loaded from cache. The cached page doesn't know your friends. You then have the JS load a page with friends as a separate request. This shifts the request from the loading of the page to after the page is loaded, meaning the page gets loaded faster, but missing the small feature of displaying friends.

Keep a counter of friends for each user. Most users will have zero friends, no need to look them up.

Also, if that's such a bottleneck, create a separate webserver in C++ and keep friend relationships in RAM in some sort of an efficient data structure, something like Map [UID]->[set of friend UIDs]. Queries should be lightning fast (even if you pass a thousand UIDs in one query).

That is essentially what memcached should be doing now.

Plus, the friend list isn't that dynamic, so even it can be cached. (but on the client side)

They could add that feature to Reddit Gold, both boosting their subscriber base and reducing their load. (But who knows; maybe there would be tremendous backlash. I'm just suggesting this because I don't personally use the feature.)

Reply on news.ycombinator.com