Hacker News Reader

Comment by RC_ITR | Hacker News Reader

The progression that people almost always forget:

1) New system comes out that indexes/controls/regulates a naively created dataset

2) Data consumers adopt that system and experience benefits

3) Data suppliers learn rules of system and take advantage of it to improve the positioning of their data, thus breaking the intent of a system built on the assumption of naive creation

4) Users complain about the broken system

5) New entrants realize that the original system actually solved the core problem really well, and there are no easy ways to solve the 'gaming the system' problem

6) Flawed system remains the best available option indefinitely

It's like entropy, there's just no fighting it.

EDIT: And to extend this beyond Google - do you see a lot of long text blocks in 7 second tik tok videos? That's because the creator found a way to game the algorithm.

View on HN

Honest question: what else besides web search has followed this progression?

Spam and bot detection could be close to that model ?

Email spam? I'd say that's basically solved.

For bot detection, when was the golden age? I'd say that, rather than follow the above description, it started as non-existent and there was basically a smooth descent downward to where we are now.

3) ban abusive data providers?

How would that even work? Define the border between SEO and abuse.

> Define the border between SEO and abuse.

Perhaps there is no border to be defined since nearly all SEO is abuse?!

The only acceptable SEO should be "provide website content that is more relevant/interesting for the viewer".

Let's say someone writes a blog article. They publish it on Wordpress with a bunch of unoptimised defaults. Some time after, someone comes along and says "Hey, the search engine will rank your post more highly if you include the name of the post in the URL instead of just having it be blog.example.com/posts?id=1234", so they change the URL. Is that SEO? Should that result in the site being de-listed/banned? It doesn't meet your definition of "provide website content that is more relevant/interesting for the viewer".

And where is the line between "provide website content that is more relevant/interesting for the viewer" and SEO? If I realise that (to take an example from something found on Google) I could take the sentence "Identify the best customers and convert more" and rewrite it as "Marketing automation helps you identify the best customers and convert more", is that SEO because I'm intentionally adding a keyword, or is that making the content more relevant to the user by being more explicit in what I'm saying?

To my moral standard, all of this is manipulative if the intention is to manipulate the Google ranking instead of seeing an improved Google ranking as a result of delivering better service to the visitor.

This does of course not imply that these things can be proven, which implies that no action can be taken. Not everything that is immoral is punishable.

I'm sure there are better examples, but this specific one actually does make the experience better for the reader. People often return to good blog posts and it's a lot easier to remember a title than a numerical id. If you can remember the title and know the url of the blog and it has a simple naming convention, you can bypass the search engine entirely and just go straight there. Sports sites are really good about this. <URL>/<LEAGUE>/stats always gets you the stats. <URL>/<LEAGUE>/standings gets you the standings. If you're curious how a team is doing or who currently leads in scoring or whatever, you know where to go without needing assistance from search.

Well let's make this real.

Sites with people hired to optimize the robots.txt file are ranked higher than those without. If we ban all websites that hire people to optimize their robots.txt, then a lot of the web isn't indexed and the search results suffer.

Or if we ban every creator on tik tok who posts a lot of text in a short video, then a lot of creators are going to be de-platformed and moved elsewhere.

The point of what I wrote is that 'humans are really god at following rules, so much so that they often are able to manipulate rules to their advantage, without breaking said rules.

That's why this is such a hard problem to counter.

You're ignoring the only insightful part of the article: if a search engine succeeds at all, it cannot ban anyone, because some antitrust bureaucrat somewhere will cluck about it.

If this were true, wouldn't Google display all million pages that are vaguely relevant to my query on the first screen?

Their only job is to stack order the internet. Down ranking a site to the 1,000,000th result is the same as banning them. They necessarily do this all the time (like for every search with more than 10 potential hits).

Reply on news.ycombinator.com