I've a different but related use case. I want to make a 5 GB file in a specific format, which requires that the data be sorted. I want it to run on my laptop machine, which has 4 GB of RAM. I dump the unsorted data to a sqlite database, tell it to use up to 1 GB of cache, create an index on the sort field, and pull back the sorted data, which I can stream directly to the file.