Hacker News Reader

Comment by 1659447091 | Hacker News Reader

There is a github of the factbook for anyone that just wants JSON or markdown files:=> https://github.com/factbook

"A cache for datasets for the country profiles from the World Factbook in the original (1:1) format from the cia.gov website"

https://github.com/factbook/cache.factbook.json

View on HN

Hi there, thanks for linking this! My GitHub and website both link to and use this source! I just thought putting it in a SQL database and making the entire 1990-2025 queryable was needed since I couldn't find one anywhere :)

it is a lot of fun and rewarding to do this! I've done it several times for medium-sized datasets, like wikipedia dumps, the entire geospatial dataset to mapreduce it (pgsql). The wikipedia one was great, i had it set up to query things like "show me all ammunition manufactured after 1950 that is between .30 and .40" and it could just return it nearly instantly. The wikimedia dumps keep the infoboxes and relations intact, so you can do queries like this easily.

Do you have a write-up of this somewhere? When I last looked at the Wikipedia dumps, they looked like a mess to parse. How were you getting structured information?

unfortunately, i consider it proprietary.

You'd presumably have to run some part of the transclusion pipeline to properly handle template/module/page transclusion.

Ahhhh yes thank you for this link! I'm working on a project and my agent referred me to the factbook. It said all this great savory information was made available on the factbook website. I thought great but when I went to check it was all very basic. Truly I thought my agent was hallucinating!

It wasn't! Note to self: also check archive.org in case there is an internet archive for any sites an agent might reference.

I checked out this repo -- it has the information useful to my project. Thanks for sharing this!

Reply on news.ycombinator.com