Hacker News Reader

sbermanon Show HN: The Internet Portal – Warp to a Random Website, Using ICANN CZDS Data

Thanks! I don't have very much experience with web design, but I did my best.

sbermanon Show HN: The Internet Portal – Warp to a Random Website, Using ICANN CZDS Data

I'm glad you like it!

sbermanon Show HN: The Internet Portal – Warp to a Random Website, Using ICANN CZDS Data

I'm currently working on a way around this similar to what you suggested.

sbermanon Show HN: The Internet Portal – Warp to a Random Website, Using ICANN CZDS Data

You must click the button each time to request a website. The loading page is just a placeholder, so reloading that page will not bring you anywhere. I'm not sure if that's what you were doing, but hopefully that helps.

sbermanon Show HN: The Internet Portal – Warp to a Random Website, Using ICANN CZDS Data

Thanks, I appreciate the feedback!

sbermanon Show HN: The Internet Portal – Warp to a Random Website, Using ICANN CZDS Data

Some links won't work, and that's unavoidable. I'm working on a way to redirect to another site if the first one is broken. In the mean time, just press the button again for a new link.

sbermanon Show HN: The Internet Portal – Warp to a Random Website, Using ICANN CZDS Data

Thanks for the feedback! I'm thinking about ways to prevent loading broken websites. I'm not sure it's possible to filter for only a certain type of website though, I think there are way too many sites for that.

sbermanon Show HN: The Internet Portal – Warp to a Random Website, Using ICANN CZDS Data

Yes, I'm thinking about a way to implement that. It's too many domains to filter in advance, but it might be possible to redirect the user to a new site if the current one is dead.

sbermanon Show HN: The Internet Portal – Warp to a Random Website, Using ICANN CZDS Data

It's true that this doesn't list all the websites that are registered, nor do all the domains lead to a working website. However, I think that most of the invalid websites are not caused by NS entries. As for the Zone File Access Agreement, it prohibits uses that allow the access of a significant portion of the data. An immense amount of time would have to be spent scraping data to get any portion that could be considered significant.

sbermanon Show HN: The Internet Portal – Warp to a Random Website, Using ICANN CZDS Data

I started this project after doing some research about how DNS works and learning about the CZDS, where any interested individual can request access to DNS zone files. I realized that I could turn this into a website, especially since I couldn't find anything similar on the internet. I used their Python API to download all the zone files, then wrote a Python script to scrape them into one file with only the domain names. I then stored these in a MySQL database on my web server, and used AJAX + PHP to retrieve and redirect to the domain. One thing I think is cool about this is that it gives you a sense of the websites that constitute most of the internet, not just the most popular ones. And unless you've clicked the button over 200 million times, you are almost certainly going to get a website you've never seen before.