Hacker News Reader

File-based storage may be "storage-friendly", but it isn't retrieval-friendly. For example, finding the list of all tags your blog uses requires a linear scan over every document in the system. Not fast. (Visit http://blog.jrock.us/. It does this. It takes a long time to load.)

View on HN

You can cache cache that in-memory! Because you've got a persistent app server.

How do you know when to invalidate the cache without doing a scan over all the files?

(Answer: hope your filesystem updates the directory change time when a file inside changes, or that your app server is inotify-ed of the change. Both are more complicated than querying a database.

If I used KiokuDB for Angerwhale, it would only take me one key lookup for things that currently involve traversing the directory tree. Caching is nice, but it is not the solution to all problems.)

You know when you update your data. Therefore, you invalidate your cache when you update your data.

Timestamps help alot if you're paranoid, also.

Reply on news.ycombinator.com