Show HN: Get 70 Years of Historical Weather Data via API
oikolab.com
oikolab.com
This is a historical reanalysis data (ERA5 from ECMWF) post-processed for fast time-series access. The primary dataset is a gridded (0.25 x 0.25 lat/lon) dataset covering the entire world so the data can be extracted for any arbitrary location. Later this year, a higher resolution dataset (0.1 x 0.1 lat/lon, about 9km x 9km) will also be added.
I've been working on this because I believe having access to historical weather is important for us to understand how climate is changing where we live. I'm not a programmer by training so it's a little rough around the edges but I believe it's the largest repository of weather data outside the primary sources, with 50+ weather parameters covering 70+ years with hourly resolution. It should be about 10~100x faster than getting the data directly from the source, and without having to deal with NetCDF/GRIB formats.
All in all, it's about 100~200TB of data that's been post-processed/chunked to Zarr format, with about 1TB/day on-going. Current users are mostly analysts who occasionally need large volume of data and don't really have time to go to the primary sources. We've just rolled out a feature to request multiple locations also request regional data (returns the data in NetCDF format) although most users seem to be happy with querying for single locations at a time.
Any comments would be greatly appreciated.
Colour me genuinely shocked that this much weather data is available to the public, fantastic project!
Most weather data what we encounter from apps are actually public data - they're just cumbersome to work with. There is so much there but not as widely used as they should be.