Hacker News Reader

OK I'm glad I wasn't the only one reading this going, "wait, this data lake thing does seem like wild overkill"

100GB ? You could store that in RAM easily these days .

I would say no for anything less then 50-100TB today.

We do have to keep in mind she was fired in 2018 and this was probably few years before that .

Keeping 2015 as the reference frame a bare of minimum of few TBs to consider any lakes or puddles

Given the "data lake" ended up being only 4 weeks of work, that might be a slight overstatement of what was being designed.

This sounds arbitrary. I wouldn't set up a lake just to generate reports, but why 100 GB?

you could load that size in memory these days in a consumer machine let alone a server .

AWS these days have 24TB RAM servers available on demand . 100 GB data in even dozen different formats require just regular database .

A data lake just for generating reports on 100 GB of data which are going to be used by very few and built by even fewer seems overkill .