Hacker News Reader

Comment by Hedja | Hacker News Reader

It's worth mentioning that indiehackers.com is a JavaScript-only website and doesn't work with JS turned off. Google's spider can execute JS, while others do not. This is why indiehackers.com doesn't show up, not because of the search engines themselves.

I have a similar issue with a side project of mine. JS-only websites solve this with server-side rendering or static generation.

View on HN

When I think of a search engine I think of it as two things.

1. A web crawler 2. A search index

Or as a good boss used to say garbage in garbage out... the quality of the engine is as much a function of the index’s ability to rank as the quality of the input from the crawler... the fact that none of the other engines crawl with JS enabled is a huge competitive advantage for google

Well sure, but you could also just make a website with a modicum of thought and then all search engines will have no problem.

Anything that loads no resources unless I have javascript enabled (and is a general purpose website, not some 3d toy or whatever) is trash.

I mean, clearly this site has value to the OP so it is obviously not trash.

What value a service provides is what matters in the end. Implementation details are secondary.

Pulling gems out of trash does not make it not trash.

Bing also renders JavaScript [1][2]. Though there's not a lot of information out there about the details like what browser engine they use.

[1] https://blogs.bing.com/webmaster/october-2018/bingbot-Series... [2] https://searchengineland.com/bing-crawling-indexing-and-rend...

> Google's spider can execute JS, while others do not.

I’m honestly having a hard time seeing how this isn’t a bug in Google search.

1 point by jefftk 0 minutes ago | edit | delete [-]

A search engine is trying to predict which pages will be helpful to you in response to your query. This means it should take "how does the page look to users" as input, which today means executing JavaScript.

(Disclosure: I work for Google, not on search)

Contrapoint: if you keep enabling hostile behaviour, you'll get more of it.

Non-JS web indexing as a default would vastly dimnish the utility of JS-only pages.

It is a feature, not a bug, because it provides value to customers.

I separately think sites that only work with JS enabled are not great, but that's a problem with the system and not with individual actors.

On the other hand, in the same way that Google is (claiming to be) trying to make the web fast with AMP, they could make the web fast by refusing to index pages that don't provide a non-javascript experience. (Yes, you can make a slow page without Javascript, but it sure is easier with it)

>> Google's spider can execute JS, while others do not.

So Google is enabling the excess of JS we see today? Sites wouldn't do that if it couldn't be indexed by Google.

Yes, they would. They'd just use NextJS or similar to push down a static skeleton that minimally covers the crawler. (My current company has to provide data to the Facebook crawler and that's what we do.)

Nothing prevents a site from doing that and just loading JS afterwards. Or from still being basically read-only without JS. Which might be an improvement...except within epsilon of zero people care in the first place and Google is building software for a rather larger proportion of the market than that, so it's an improvement for what's barely an audience.

Nobody is really "enabling" it except for a browser. And that's not changing. The shirt-rending is becoming tiresome.

I'd be curious to know why your company specifically targets large-market crawlers like that, instead of just making things crawlable. It feels like leaving money on the table when you overfit your stack to one market.

It's a shame you find it "tiresome" when people insist on retaining their own opinions, but your exhaustion is not particularly relevant.

We need to build shareable landing pages. We share via Instagram and Facebook (for now). We use NextJS because they do provide static rendering that covers what we care about and are likely to care about anytime soon.

And you can have whatever opinion you want; nobody’s saying otherwise. But the constant whining about nothing — and it is nothing — is exhausting.

Yet another reason to support other search engines. I haven't used Google in years, and I haven't had any real problems.

Reply on news.ycombinator.com