You may be interested in an SQLite version of the dataset that is 553 GB vs. the 908 GB JSON: https://archive.org/details/2015_reddit_comments_corpus_sqli...
The storage format of a dataset can make a big difference in memory usage.
The storage format of a dataset can make a big difference in memory usage.
No comments yet.