In which things are mapped, but also reduced [alternate Wide Finder in Closure]
technomancy.us
technomancy.us
Does This Actually Work? · Not quite, at the moment. I mean, it works fine for the first 50 million lines or so of the big dataset’s quarter-billion, keeping my eight-core 32-thread SPARC T2000 maxed, and I mean smokin’, you can see the 50M blocks that Paralines is reading go by pop-pop-pop. But then it hits its heap limit and descends into garbage-collection hell and thrashes its way to a miserable single-threaded standstill.
I'm hoping he will eventually arrive at a clean solution that can handle the full 43GB or so of input.
1- http://www.tbray.org/ongoing/When/200x/2009/11/11/Clojure-Re...