Hacker News Reader

I would be very surprised if there is anything in there that is not in reocities yet but I will certainly do a comparison.

I'll have to run that against all the deletion requests as well for this content because chances are fairly large that lots of it has already been removed at the request of the owners.

View on HN

Have you tried de-duping files to save space? I imagine there's a finite number of MIDI files and "under construction" GIFs that could just be symlinked to save a ton of space...

Space is not the problem, I've got plenty of that. The bigger problem is that the underlying filesystem is having some problems dealing with the total number of files.

Total disk space usage on the array that holds reocities is about 10.5 T (that includes a master copy though) and runs to 233 million files.

Have you considered implementing a NoSQL solution to store the files? I've successfully done something similar with HBase.

No, but that's a good idea, I'll definitely look in to that when the next round of reocities improvements rolls around.

Reply on news.ycombinator.com