4chan is an image board. None of the content is there long enough to be considered "Highly requested.". And like half the posts have jpeg's and png's attached.
Besides, how does caching solve the "My bandwidth bills are killing my wallet!" issue?
Besides, how does caching solve the "My bandwidth bills are killing my wallet!" issue?
I'd argue that all of the text could be served from 1 nice box if you wanted to(multiple boxes make it more complicated but not that much more). Send the post to each box, add it to a table/indexes in memory and write the post to disk and backups for recovery purposes. Then either update and cache every page the new post affects, or mark the pages dirty and update and cache them the next time they are requested.
Done, all pages served out of memory super fast, what am I missing?
As far as bandwidth bills, most browsers observe cache settings and won't re-download what it has already downloaded. His complaint is about getting hit too hard serving the html/text not the images.
> In the past 30 days, CloudFlare has proxied 1,331,004,996 page views from 4chan
CloudFlare is not caching his html, thus the performance problems, because his backend is probably dog-slow.
> His complaint is about getting hit too hard serving
> the html/text not the images.
Congratulations, you just massively blew out your bandwidth bill. The cheapest option is to host your static content yourself, especially if you're serving over a petabyte of it per month.
Sounds like it is?