Hacker News Reader

Comment by bioemerl | Hacker News Reader

I'm honestly amazed that out of the top million sites, which probably includes a ton of tiny tiny sites that are idle or abandoned, only ten percent are offline.

View on HN

Yeah, I'd expect a list of 1,000,000 "top" "sites" to contain much more than what can be called a "site," especially in 2022 when the internet has been all but destroyed and all that's left is a corporate oligopoly.

How many are placeholder pages thrown up by registrars like Network Solutions?

If they're placeholder pages, they're not dead. Those 10% are not responding at all; the requests aren't reaching any HTTP server.

at least from his computer/script. A number could have been blocked simply detecting him as a bot.

Not all placeholder pages will forever stay placeholder pages though. Some may get sold, become a site, then stop being a site again. Some may not get sold, come up for renewal and be deemed unlikely to be worth trying to sell anymore (renewal is cheap for a registrar but the registry will still charge a small fee).

Of course the vast majority with enough interest to make this list will either be sold and be an active page or still be an active placeholder but I wouldn't rule out there being a good count of pages towards the lower end of the top million being placeholders that were eventually deemed not worth trying for anymore.

Exactly. Afaik, there's even a whole auction based side industry trading expired domains. When you see a placeholder page, the original site is for all intents and purposes, dead. The domain just happens to be interesting enough that someone wants to stick an ad on it rather than let it resolve to nothing.

Yabbut a placeholder page returns status 200. We're discussing URLs that don't return any status at all; they are unresponsive. They have no placeholder; they aren't a "place".

A site that is occupied by a placeholder page is a 200, as far as that "top million" database is concerned. It might as well be a working shopfront or blog.

How is "top" defined here? If they were dead, wouldn't they fairly quickly stop being "top"?

EDIT: the article uses a list sorted by inlinks, and I guess other websites don't necessarily update broken links, but that may be less true in the modern age where we have tools and automated services to automatically warn us about dead links on our websites.

I can expect large SEO spam clusters of "sites" with many links inside a cluster to make them look legit. For some time such bits of SEO spam were on top of certain google searches and enjoyed significant traffic, putting them firmly into "top 1M".

Once a particular SEO trick is understood and "deoptimized" by Google, these "sites" no longer make money, and get abandoned.

Blows my mind that my blog is 210863rd on that list. That makes the web feel somehow smaller than I thought it was.

Eyeing you jealously from my position at 237,014 on the list... We're almost neighbors, I guess.

Reply on news.ycombinator.com