Comment by yaakov34 | Hacker News Reader

Comment by yaakov34 | Hacker News Reader

Parent Full thread

Hey, I come here to offer useless kvetching and complaints, not solutions :). Seriously, if I knew how to solve it, I'd be the CTO of Google.

But I'll make a couple of guesses. I think they need to go for a lot more human input. It goes against the grain of everything they do, they want to solve everything algorithmically. But SEO is just too good at abusing algorithms. I think they need to get more humans involved, and force the SEO guys to tone it down.

And one tiny specific suggestion: drop the strong signal of a search term being in the domain name. Maybe leave it for proper names, but drop it for all generic words, please. It used to be a good signal. It used to be that when you saw your search echoed in the domain name, you knew you found what you wanted. Now, I cuss and skip it, because I know it's spam. It's been abused beyond belief.

EDIT: Yes, lots of people suggest "going social" on the problem. That's a lot harder for most people to manage than just using a search box. And searching for "trusted social group for reviews" is just going to take you to mytrustedsocialgroupforreviews.com. Guess what's there.

We tried doing something similar at XMarks (a bookmark sync service) when I worked there. We figured bookmarks were a good source for mining quality websites and hired some very bright people with a search background to build it. For some searches, it gives pretty good results. Here's the one it gives for VPN providers:

http://www.xmarks.com/topic/vpn_providers

It was an uphill battle for us, which ultimately never worked out. Early on, we did a usability study with users to see how they use search engines and the people we interviewed tended to think that google was more of an expert than themselves and if they were having trouble finding things, it was more their problem than google's.

We ended up trying to take our data and tried decorating it directly on the google search results page to start getting usage, but it was really hard to get any traction and build any momentum. For most searches, Google is good enough and only a small number of our current users started using our website for search/discovery.

We ended up spending a significant amount of time decorating the ads within search results and after a considerable amount of logging and number crunching, found that decorating those ads not only increased the clickthrough rate of ads we decorated, but also increased the overall ad clickthrough rate. That's a huge win for a search engine and worth a lot of money, but ultimately google just did it themselves, and for other search vendors that had ads on their pages, they were basically handcuffed to google's contract which wouldn't allow them to use our technology.

TL;DR - Competing with Google is hard.

The conflicting interests involved in building a search engine are a fascinating subject. This is probably a bit deep in the comment tree to start a discussion of it, but it's certainly true that it doesn't do you any good to present wonderful academically vetted search results, if the users don't click on the dang ads.

For the particular search here - yes, you do better than Google, but you don't have a million hungry SEO guys trying to game you. So it's not a fair comparison. If you get as big as Google, it will be a different story. Of course, if you get as big as Google, you might not care :)

Why not sell this data to Google and other search engines, instead of competing directly with them?

Note that I've used XMarks / delicious in the past to get more relevant results. When a link has been bookmarked by many users, it is more likely to be interesting. It was extremely useful when approaching an unknown subject.

I think one reason why the search product didn't gain traction was because it interfered horribly with the original goal of Foxmarks -- a service to sync bookmarks.

Speaking as a former user of Fox/XMarks, I saw the product go from a simple bookmark sync, to a bloated extension that tried to do more than what I had downloaded it for.

I do think that the data could have been useful, but it would have been a better idea to create a completely different product than to bundle additional features onto an existing one.

> Drop the strong signal of a search term being in the domain name [...] It's been abused beyond belief.

I whole-heartedly agree—but people still search for "facebook.com login" and similarly trivial queries. If Google broke returning "facebook.com/login" for these it'd lose a lot of its value for these people.

I can only imagine how many links facebook login has across the web from their widgets. Don't think we have to worry about them falling out of the top spot for that query, in fact if they did google would likely get a visit from the DOJ soon after...

The Facebook login page did fall out of the top spot last year in favor of a ReadWriteWeb story about changes to the Facebook login page.

The comment thread rapidly filled with deeply confused people complaining that they couldn't figure out how to log in on this new page.

Here's a link to a RWW story about what msbarnett is talking about: http://www.readwriteweb.com/archives/web_illiteracy_how_much...

It surprised me at first, but I really forget what it's like to be tech illiterate.

Search term in domain name should not have as much value as it currently does. Along similar lines, I think link anchor text is way overweighted as it's fairly easy to game for commercial searches and not at all common for the type of "editorial link" model pagerank assumed.

As it stands, link buying is rampant: as risky as black hat techniques are for real businesses, if you're building diverse spam web properties through a scalable model, it's not really risky at all. it's not like google can give you a permaban.

> That's a lot harder for most people to manage than just using a search box.

Hey, sounds like a business opportunity. "Join our bogon-buster network! [fine print: invite only, your activities and reviews will be rated by other members.]"

Of course, after awhile, you'll need another layer on top of it to make sure that your bogon-buster networks aren't full of spammers themselves.

Yes, you then have the same problem suffered by most social networks.

I actually think a similar problem has been partially solved by Twitter already. Twitter grows but the content curation is done by the people you follow therefore new users don't affect your interactions on the site unless you let them. Of course, there's gardening work which has to be done by the user in getting to this point but it would probably be less work than browsing some of Google's search results.

Additionally the intelligent thing would be to mine the social interactions of authoritative users to provide public data for users that don't want to engage with the app.

edit: Of course, it's possible to become an authoritative spammer in which case as a power user you will still need to garden the group of users that you interact with.

So why not adapt the Twitter model to this? You can choose who to include in your Bogon-net, and maybe set different confidence requirements; e.g., "Show me only results that at least 50% of my Bogon-busters have rated, and only show me results that have a 75% approval rating."

The problems I already see are a) having to compute all those scores, and b) I'm unlikely to rate more than a handful of sites a day - I think Google tried that with their browser toolbar, too.

Reply on news.ycombinator.com