Hacker News Reader

Comment by pwaring | Hacker News Reader

It's how Greymatter and a few other systems used to work as well. The main problem was determining which pages should be updated, which usually resulted in the entire blog being rebuilt. This took several minutes if you had a few years of archives, and if anything went wrong part-way through the process you had to start it all again.

View on HN

I've ruminated in the past about a better blogging system based on updating on-disk caches based on POST and not GET. I think everybody who watches Wordpress trip over and burst into flames when you so much as breathe on it has.

The main problem to solve is inter-page dependency. The rest is not that hard /famouswords.

You can get too carried away with scalability though. I run half a dozen WordPress blogs, plus some static sites and email for multiple domains (with AV and spam filtering) within 500MB of RAM. I don't need to implement any caching because I already have sufficient resources.

Get this: http://wordpress.org/extend/plugins/w3-total-cache/

It's called a caching plugin, but it also compresses and minifies stuff. Any site can benefit from loading faster. You can turn it on and let it go. It has sane defaults.

Thanks for the info, after hearing people praise wordpress as the holy grail of blogging, I found it to be very very slow. Hopefully this plugin will help

I found that a major contributor to page generation times on Wordpress is the time taken to load and parse all the PHP files. I installed a PHP opcode cache (APC) as I recall, and it made a huge difference, because the parsed representation of all the Wordpress code is cached in memory (or disk). I'd do that before installing any Wordpress caching plugins, though total cache has other nice features, like minification. Plus it has three different levels of caching. You can cache entire generated pages, and/or objects and/or DB results.

WordPress is usually praised for usability, not performance. To make WordPress faster and more efficient, you need a caching plugin like W3 Total Cache. It caches your pages in static files, caches queries and results in memcache, and rewrites css, images, and js to CDN paths if you have a provider. In front of this, I would have Varnish cache as much as possible in memory, for even faster access. W3 Total cache can even invalidate your Varnish cache when pages are updated.

Cannot give Varnish enough praise, it can tubo charge even the most expensive / slowest loading pages and uses minimal resources.

> I don't need to implement any caching because I already have sufficient resources.

Define "sufficient resources". It might be enough for your normal traffic, but without caching those blogs will crash from the /. effect (unexpected traffic spike).

Sufficient = enough for normal use. My point was that people tend to get worked up and spend time on caching strategies when there's no need for the vast majority of sites.

Even with caching, I suspect Apache would fall over if I got slashdotted.

I agree there is a balance between sane and insane efforts at caching, but I disagree that no caching is a viable choice. Apache and Wordpress with no caching on a smallish VPS will fall over really quickly. This is unacceptable when you can get huge benefits very easily by installing a widely available plugin like WP-Super Cache or WP-Total Cache. It's almost no work to get that set up, and will help you survive a mini-slashdotting. That's going to let you serve an order of magnitude more traffic than usual.

An easy solution like those plugins is such a big win for so little work, I just can't agree with "it's good enough for normal use". The last thing you want is for your blog to fail when you are getting a traffic spike.

> I don't need to implement any caching because I already have sufficient resources.

I want you to print this out and stick it somewhere out of sight. When one day you are linked to on Reddit, Slashdot, a major news site or the like, pull it out for a hearty laugh.

There's a risk vs effort balance involved in these sorts of decisions. The risk of any of my sites getting Slashdotted is sufficiently low that it's not worth spending time preparing for such an eventuality, and I'm prepared to accept the consequences if it does happen.

Installing WP-super-cache or W3 Total Cache takes about 5 minutes and will do the most effective 80%.

Also, I think W3 Total Cache is an order of magnitude easier to install than WP Super Cache.

I'm not sure I understand that.

Around 2002-2003, I built a gigantic, sub-optimal, regex-heavy Perl script for automatic updating of webcomic sites (it's still in use on several sites I know of, ugh).

Each day of the archive had its own separate HTML page generated from a template.

I never got around to the complexity of supporting selective updates, because brute-force regenerating the entire archive every time the script ran, even to several thousand comics, took just seconds on anything but Windows (whose file I/O was just too bloody slow, but even then the run only took a minute or so).

This was on a shared hosting platform, where the provider probably oversold capacity, and without access to the command line. I suspect part of the delay was waiting for the CGI script to send data to the browser.

Reply on news.ycombinator.com