Google dominating search, internet needs crawl neutrality, what do you think?

fastcompany.com

2 points·cheinyeanlim··8 comments

Amazing that getting some press on this issue, which is very, very old, took some former Googlers who are trying to compete with their ex-employer. Monopoly in search does rely in part on cooperating websites, who allow Google to crawl but block others. Why. Because Google pays them or helps them get paid. Or at least, Google is the only way the website operator believes her site can be found.

If all websites offered sitemap.xml files (links to these files are found in robots.txt), then "crawling" would in many cases be unnecessary.

Regulators and policymakers need to step in if they care for competition in search. The market needs crawl neutrality, similar to net neutrality.

“What do you think?” is redundant since the point of submitting links is discussion.

On HN we submit the original titles of the articles.

"What do you think?" is the starting point to lead the discussion. I think is ok to further elaborate as long as not manipulate to bias the origin purposely. Just my personal POV, be flexible, no hard feeling.

I’m referring to this site’s guidelines: https://news.ycombinator.com/newsguidelines.html

Thanks for pointing out. In fact, I read it many time before I start to submit, but can't really find any statement mentioned that ONLY "original titles of the articles" is allow.

From the guidelines:

> "Please use the original title, unless it is misleading or linkbait; don't editorialize."

Please assist to amend the title if you think the question “What do you think?” offend the HN's community rules.

Reply on news.ycombinator.com