I am running my personal web crawler since September of 2022. I gather internet domains and assign them meta information. There are various sources of my data. I assign "personal" tag to any personal website. I assign "self-host" tag to any self-host program I find.

I have less than 3k of personal websites.

Data are in the repository.

https://github.com/rumca-js/Internet-Places-Database

I still rely on google for many things, or kagi. It is interesting to me, what my crawler finds next. It is always a surprise to see new blog, or forgotten forum of sorts.

This is how I discover real new content on the Internet. Certainly not by google which can find only BBC, or techcrunch.