The next Google
dkb.io
dkb.io
And that's by the way why Google is still successful as well. Because it literally still is a simple box where you put a question in and it gives you answers without needing to do anything else. The only way to beat that is to make it even better while not making it more complicated which is very hard to do.
This is the thing. I'm the power user in my circle. People comes to me for suggestions about pretty much anything involving tech. Sometimes just because they see me with different stuff.
So if I'm not the average user I may look for other options, as other people like me may do. If I find such options reasonable for the average user, that will be my recommendation to them.
Google is very good at this, but this is exactly what I _don't_ want in the "Next Google". I want a _search engine_ for the web, not an answers engine that tries to know what I want better than I do.
Search the web. Give me links to websites. This seems obvious to me, but everyone is trying to be like Google.
I'm convinced the death of the web and independent forum communities is largely the fault of Google's lousy search results no longer actually returning real web results.
The audience of HN is extremely skewed towards preferring systems that allow tinkering but that's not what the market wants.
Some walled gardens, like facebook, are decaying. Others, like tiktok, are vibrant, still fresh and new.
Walled gardens are a natural result of the pursuit of capital. If you burn VC money, but create no moat (it turns out the walls of a walled garden are also a moat, weird huh), then as soon as you introduce a clever algorithm that introduces 50% more ads to extract profit, your users will all leave.
As such, a business is incentivized to build these walls and moats.
How do we avoid this?
Well, we do have examples. Mastodon and other open source projects eschew walled gardens in favor of free software ideals. Web3 embraces a certain "decentralized" vibe which lends itself in this direction. Universities, and other public-ish institutes like DARPA, created the original internet and many of its technologies.
Unfortunately, open source projects will struggle to advertise or find users. They do not have the initial capital to get as much momentum as the VC-funded alternatives. Web3 seems surely doomed to end up also building walled gardens for the crypto-anarcho-vibe is only skin-deep, and many a regular business is now highly involved.
This leaves government entities. The government is the one group that is both well funded enough, and has motive to create protocols which prioritize the user's freedom over profit (after all, the users will pay taxes regardless of how high the walls are).
It seems to be out of fashion these days for the government to actually do anything though, so perhaps there is no more chance of that than of a free software project managing the same.
That's also why "Google's search results are soo bad." They're not. For the bulk of Google's visitors, they're good enough.
Anyway - you’re right, nothing at all wrong with wanting choice. I think the point being made here though is that “layperson gravity” / mass market appeals / lowest common denominator is going to mean that tuneable web search will be a niche product, forever. Even if we’d both like that niche.
"Elitist" feels pejorative. Also a not-inaccurate way to describe a system which presents high barriers to entry.
Not trying to be awkward, but even back in the early days of Netscape, it wasn't like you had to recall any arcane commands to use most search engines.
Sure, they all had limitations then, such as not necessarily knowing similar word senses to create more nuanced searches, but that was a fairly level playing field.
Is it simply that it was initially a bit obscure and not everyone had found out about it? That's not really a barrier in terms of difficulty - as soon as people got into the "in crowd" they could use it just like the rest of them.
When netscape came out if you were lucky to have ppp access setting it up on windows 3.1 was difficult but once you were up as long as no one picked up the phone you were fine.
But even this was too difficult for the average person who lived in an aol world where the internet was limited to aol.
Why is publishing documents so complex now?
my site is all html, why not.
Cost was one. Hardware wasn't cheap and it would seem to have no practical use except to satisfy one's curiosity. A PC for the kids cost about half of our family's monthly gross income. A modem would be 10%. And the phone bill caused tourette's like symptoms more than once.
AFAICT, quality personalized search results isn't the goal for Google here.
I think the point of all that is generating revenue via advertising sales.
And while providing high quality search results might once upon a time have been a goal, both as a goal in itself and a tool to drive user adoption/engagement, that's no longer necessary as they have a (relatively) captive audience and a (relatively) captive customer base (advertisers). As such, quality search results are no longer all that important.
I'm not a Googler, IMHO, YMMV, etc.
Now they have millions of cell phones collecting data and the GPS location of everyone offline, they run extremely popular DNS servers to see what websites people go to, most websites (including educational and government websites) include google's trackers so on most websites every page loaded will ping at least one of google servers. They've got people uploading their personal and work documents to their cloud. They are swimming in data collected from sources outside of web search. Between that and the lack of actual competition it's no surprise they aren't investing in making searching the internet better for people. In some ways they profit from search results being terrible. If anything beyond the first few results on the first page is filled with spam and irrelevant websites the top few results to any search become even more valuable.
That depends on whether or not you agree with Google's definition of "better". What is better for Google is a set of results that make you most likely to click on an advert. Failing that, they want you to click a link, see the site, and either click an ad there or quickly return to Google where you'll click on an ad. The worst outcome for Google is that you'll click a non-paid search result, like it, and stay on that website.
This is the fundamental problem with search engines - if they work well and give you exactly what you're searching for first time then they won't make any money. A lot of what Google does is subtly trying to give you results that look great but really aren't.
But to be honest the internet is still the internet. The web still exists. Any lamentation at the loss of the "old" internet is that you don't have more angry uncles spewing political rhetoric on your motorcycle forum.
Do you want the unwashed masses in your specialist forums? I certainly don't. Seems to me things are working pretty well. I do worry about the next generation but to be honest, the same was said about me.
Instead of complaining about the state of the web just be the change. Host your site, run your forums, live your life. Stop worrying about how other people should live their lives. Take some youngsters under your wing and show them how a keyboard works instead of a touch screen.
This x1000.
The only way to ensure that future people can continue to maintain things is to teach those who are interested.
While I think we need to scale back the opaqueness of tech to the average user[0], there will always be some who just know more.
[0] By which I don't mean "more configuration options", because the average user doesn't want those.
I mean make it easy to access the source - and just as importantly make it easy to use changes to said source, so those who think, hm, I want this, can learn to fix it themselves, or find someone who can.
Having the choice between "unwashed masses" and "curated to suit political agenda" I would firmly choose the former.
Before I get attacked on political agenda angle, please understand that it takes trained professional actively trying to remain objective to have impartial moderation. Many professional discussion moderators fail here and you just cannot expect neither unpaid volunteers (doing things out of passion which is not impartial in the first place) nor paid employees enforcing company policy (again very much partial) to remain objective.
The goal of moderation should be to allow for multiple angles of thought to emerge and not get drowned. Give me the tools to see those non-mainstream positions.
The unwashed masses are often filthy and malicious (on the internet).
Moderation, of the governance and oversight meaning, is unforunately very necessary to keeping any pool from becoming a cesspit. By some combination of technical, volunteer, and participant effort (membership fee, or service requirement), areas can be kept decent.
Without this, decent and would-be participants are run off by spam, scams, trolls and harrassment, political rants, religious rants, etc.
Admninistration web sites are now forcing you to use google(blink/geeko) or apple(webkit) based browsers. Asking nicely to restore interoperability with noscript/basic (x)html browsers does not seem to work, namely keeping the door open to alternatives requiring a reasonable effort of development, and not the army of devs of big tech.
This requires regulation I guess.
I'm not sure that's true. Just to pick an example, basically any type of product search now leads to auto-generated spam/borderline-spam websites that scrape reviews from Amazon, which are themselves often fraudulent.
I guess you could say that's "good enough" but only in the sense that someone being scammed at some point consents to something that makes them a victim of the scam.
For many searches, you'll see "reddit" autocompleted to the end of the search because Google's organic results are not good enough and people are trying to weed out the spam somehow, albeit in an imperfect way that restricts results to one site.
I guess you could call that good enough.. I call it morally irresponsible.
I can't lab it up now, but that is the gist. And DDG has it while Google rarely has the same level of it.
They could start penalizing sites that show different things to googlebot and a real browser again.
The could do a better job penalizing sites that just copy content from other sites.
They could penalize ad-laden sites.
They could stop prioritizing youtube results over text content.
They could allow you to blacklist sites again.
They could do so many things.
None of this is curation, put priorization of search result is exactly what a search engine does.
Have you read any of Google's advice to site owners? It amounts to "provide your users with what they're looking for, and do it with a fast and easy to use site". If you think SEO is an easy hack to get any site to the position 1 for any search, you're just ignorant of the landscape. All the "SEO" crap you see about keyword density or whatever, is just an industry trying to sound different and like they've got the key to position 1.
> They could start penalizing sites that show different things to googlebot and a real browser again.
Google is ridiculously good at this. Would be keen to see examples of a site that does this in a meaningfully negative way that ranks well for searches that matter.
I don't fully agree with all of your other points but I can see why you'd mention them.
And the people trying to get their garbage spam sites to rank high are wise to the common tricks people use to attempt to get better results. Just today I accidentally clicked on a result that from the title looked like it would lead me to reddit. I should have paid attention to the URL because instead I was sent through a bunch of redirects to some terrible spam site.
For products, one of the worst offenders is Target. Yes, Target. They show up near the top of about 60% of my searches, including for things they simply do not carry. But hey, they carry a similar product that you might be interested in! (not)
SEO results make Google money, but they also make the results suck.
Anyone I watch older than 50 years old is constantly frustrated by google, they just don't know what's good, what's not. I think they use it because it's the "norm" and that is the only reason.
And as far as everyone else.. I have no idea to be quite honest. I have a fairly big handful of non-techie friends who think google sucks but is good enough so there is no real incentive to change.
With any habit people don't change it unless they have a huge reason to. No one stops eating double cheeseburgers until their doctor literally tells them they are going to die. I think that's what we're up against here.
Hey, now wait a second there, you young whipper-snapper. I'm sitting here reading HN while I install Debian on an XCP-ng VM, and I hear I can't understand whether Google results are good or not? There are plenty of us over 50 that actually lived the early internet and watched it first-hand transform into what it is today.
Plus the topics were very Linux, DRM focused. Digg had newer more interesting tech topics
And somehow, there is still enough good, new content to keep the conversations fresh. Some of the best are when someone shares a personal story, then there are tens or hundreds of follow-up posts -- generally supportive or inquisitive.
I've had a number of replies get hammered, unfairly in my view.
Yet the signal-to-noise ratio benefits overall if we eat a little Type I error now and then.
Yes, and now imagine these same people, most of whom never bothered with basic google search options (eg. excluding entire domains, mandatory keywords, ...) presented with a search engine with a plethora of knobs and levers to tune it to their liking.
How well will they work with that?
I’m sure that’s what Yahoo! used to think about Google before Google replaced it.
I’d say the for the bulk of Google’s visitors the results aren’t merely good enough, they’re insanely great.
Now, that said, the G monopoly is horrible and needs to die.
What pushed Google over the top for me was the sheer size of their index...
That's an excellent demographic point.
A counter-argument is that the move from port 80 to port 443 is the elitist drift. Instead of innocently putting information out there and sharing in an egalitarian fashion, we now have certificates and gatekeepers and lions and tigers and bears, oh my!
Most of my search results at this point look like a spam ridden inbox from the mid-2000s.
Google is a drug dealer selling to the digitally crack addicted. The death of cocaine was meth, and the death of the web was google.
I know I get on people's nerves on HN by harping on user needs and FOSS usability and interface design and such, but I think we as a group need to start taking user needs much more seriously. Open-source alternatives will always be alternatives until without taking usability and the overall experience seriously.
Many, if not most FOSS software developers choose some commercial tools— count how many MacOS and Windows laptops you see at OSCON or FOSDEM. Consider how different the cost/benefit analysis would be for people without the most basic requisite knowledge to reason about software problems, let alone troubleshoot, or throw in a PR to address them directly. Commercial companies don't get these seamless experiences by magic— it involves research, design, development and testing to deliberately remove friction and stumbling blocks rather than assuming your use case is universal, or that docs are a suitable replacement for fixing usability problems. There's nothing stopping any open source project from doing any of it. Volunteering isn't unique to coding— people volunteer to cook and build houses and clean up trash, too. But developers run FOSS and developers need to deliberately incorporate those other perspectives into their projects to get the benefit.
If one dislikes systems that allow tinkering then why read and comment on HN. I do not understand.
Maybe HN is not the ideal forum for discussions of "the market" (due to lack of interest and/or understanding), but maybe HN is a decent forum for discussing systems that allow tinkering.
This is absolutely true; however there are riches in niches.
The % of internet users who want X probably is lower, but I’d gander the absolute number of people who would want something like this is much bigger than it was a decade ago.
This community forgets that because we’re told by investors, the media we read, etc. to go after the biggest markets possible. “Hunt for elephants, not field mice,” they say.
Don’t forget that a niche market on the web can still be massive and small fortunes can be made building products and services for them. You might not even need to have investors on your cap table to bring products to market to sell to those niches if you know what you’re doing, but shhhh… don’t tell them I told you that.
There are no more websites worth linking to anymore... If you filter out all SEO spam there is barely few webpages left...
Google is desperately trying to hide that fact. Most of the web 1.0 can nowadays fit into small town telephone directory... You do not need mulitibilion dolar business to run web directory...
I thought like you that if even Google couldn't find anything it was not there, but after discovering marginalia I now know it is just Google that has become unusably bad.
For day to day searching I now use Kagi and for me it is easily worth 10 or maybe 20 dollars a month since it "just works" unlike Google and has a larger index than Marginalia.
For now though Marginalia gets the money since Kagi is still in beta.
This is just retrieving articles when I search, not actual websites. Interesting if you're looking for article related to search keywords I guess, but genuinely unusable for actually finding something specific.
I think it showed the man page first because that domain is highly ranked.
You’re in the small minority. You’re a power user that thinks like a software engineer, and likes to deal with data in lists.
Most users are completely happy that Google tries to answer their questions and (usually) provides the right answer front and center.
1. Yes Google search has gone to shit - even putting stuff in quotes now does not do an exact search (there is another checkbox you ALSO need to use for that). It tries to be too smart even when no user is logged in.
2. Giant mega forums like Reddit have really taken over. Instead of a dedicated forum people just go to subreddits. Personally I think it is good and bad and I still try to actively participate in both.
The "we answer your question" thing could be useful, but:
1. It should be in addition to (not instead of) the web search results.
2. The answers can't be wrong. Google often gives such a low quality answer that I no longer want to use it for this, besides asking about the weather and sunset/sunrise.
I think you have the cause and effect reversed. (Though I disagree with the meme that Google's results have gotten worse. For me, it's more useful than it's ever been. I often find the information I need without even needing to click on a result.)
You don't want that, I don't want that, and I am sure many people here on HN and similar gathering places for powerusers agree with us.
But "the next google" doesn't have to replace google for us, it would have to replace google for the average consumer (AC). And the AC likes the "simple box". The AC doesn't want to fiddle with customization options. The AC is used to walled gardens, advertising, "Apps" that are just wrappers around webpages.
So any "next google" will have to compete exactly at the "simple box" game to get the attention of the masses.
You are wrong; Open Web died or almost died because of walled gardens like Facebook, Instagram and Twitter. Billions of people who hang out on Facebook, Instagram and Twitter would hang out instead on Open Web e.g. websites, blogs, forums etc. if walled gardens didn't exist.
I kind of miss forums being the place where most discussion took place.
It is a box that gives you a screenfull of ads, some spam copycat sites I wish I could remove, and a lot of clutter. As many people on here have said before, it only gets away with it because it owns the web browser through Chrome, Android and its Apple deal for Safari.
From someone who tried the switch to duckduckgo and uses ddg as the default engine: I can't remember the last query I typed without adding "!g". Google doesn't get my queries because the service is shoved down my throat, it gets them because the alternatives I tried are worse wrt the total scope of my queries.
I didn't bother checking customization-options, the ux was hostile and the results did not give me what I want. Pass, sorry.
1. Wiki article 2. Github repo 3. API documentation 4. Introduction with Python Code(Not my fav quality article) 5. Guide in R 6. Research article
I can't quantify what is better. But Google didn't give me what I wanted but DDG and you.com did. So congrats. My only issue is seeing Medium articles above the web results. While Medium might have the answer in this scenario the top web result was correct but de-prioritized which was incorrect from my prespective.
UI is nice and fast, different too. Not sure if I love yet, but will keep trying. Thanks for the link. [1] https://towardsdatascience.com/t-sne-clearly-explained-d84c5...
Speaking of doing less, I would love to see the web be more hierarchical or semantic (but not necessarily "the semantic web" as it's currently conceived). Google itself is what made the world reorganize itself. A world where that kind of search exists will reorganize itself around search, maybe not always for the better.
Concrete example of going from a hierarchical/semantic world to a search-based one: Instead of finding your socks in the sock area, which is inside your clothes area, which is inside your "do private things" area, let's say now every sock has a trackable chip in it similar to an AirTag and you just say "Alexa where's the nearest pair of socks?"
Pros:
No effort spent on putting your socks in the sock place. Just throw them anywhere.
Instant access to socks.
Cons:
Big Tech, with all its limitations and machinations, now mediates and controls the relationship between you and your socks.
You succumbed to the temptation to slack off, and now there are socks everywhere. The Roomba doesn't even work right.
1. Much clearer delineations between ads and organic search results.
2. For anything remotely commercial it wasn't the case that literally the first 4 or 5 results were ads, pushing organic results below the fold.
3. No AMP carousel
4. There wasn't a vomitous amount of those "tidbit" sections - there were only a couple and they were usually helpful.
i know what you're saying, but this isn't exactly a triviality
People want to accomplish stuff, this means they need tools to do stuff and if a tool can be customized to do the stuff faster or better people want the customization. At my job we have paying users that requests features that indeed are work related(not moving shit around). I know GNOME-minded people will disagree and they prefer to bend their work to fit a guru-s vision.,
Now my turn to ask, why do people like you think there is a generic and basic solution that works at the same time for the casual user and for the user that has a lot of tasks to accomplish? Is there some theorem that shows this, like "The GNOME theoreme of product design, keep removing features until the shit convergence to the local minimum where you find the minimum product and the minimum set of users possible.
Obviously there are exceptions, and some tools are so advanced that it's necessary to be able to customize them. No argument from me there. But for most tools, and most users, there's just no hunger for customization. Almost nobody wants to have to manage an array of options to do a web search.
What is even more shitty is when soem feature is removed like the System Tray and first they pretend they do not understand what you mean when you say this feature is very important, then after someone wastes his time to explain a n=th time again how working people use the System Tray features at work or at home the GNOME dude finally admits the issue and offers some workarounds that are not equivalent but are "possible to do but with lot more work".
On short, some people do work on the computer, some people use search engines for work to find relevant stuff, this people do not ask features like "please use the exact same padding everywhere because I am OCD" or "please make those buttons/corners or edges smoother so I don't cut my tongue when I lick my screen", this people want the customization to do a task.
I am not sure why simple people have a SEGFAULT if they randomly end up in the Advanced section of a settings section, what do they expect when they open Advanced? Google main page has a small link for Advanced search see https://www.google.com/advanced_search?hl=ro&authuser=0 how many GNOME users got hurt by this link existing?
"Hold on", you say, "professionals want options, like user-expandable RAM etc". No, that's the misconception about the concept of a "professional". Unless you are a hardware engineer, tinkering with your notebook's internals is the absolute opposite of professionalism, its either a completely misguided waste of time and money, or a perfectly fine hobby.
Real professionals get work done. Customizing their workspace is something they feel ashamed to do, because it's procrastination at best.
Just that exactly is the workspace changes a lot, the hardware internals are hardly ever a consideration for any professional, it's easier to just buy something that works well from the start (and hardware costs peanuts).
The Titanic didn't sink because of the arrangement of deck chairs, and Mozilla didn't sink because of any features they did or didn't provide (and GNOME didn't fail to achieve significant market share in the first place because of features either). The actual problem doesn't have anything to do with the stuff on the deck at all.
It's the one-two combination of vendor-lock in and bottomless marketing budgets. Since most of the value of the Windows platform and the Web is the immense amount of stuff that's built on top of it, there's a huge lock-in effect that prevents you from even reaching parity, much less exceeding it. And in order to overcome the marketing budget and pure inertia, you need to be ten times better, not just on par.
If GNOME becomes as usable as Windows, it won't have anything to do with what they actually do in the desktop environment itself one way or the other, whether it's continuing on the road they take now, or reverting everything back to the way GNOME 2 was, it's totally irrelevant. GNOME becoming usable will be entirely because of Valve investing in Wine, combined with a whole bunch of other apps moving to Electron and shipping Linux versions because heck why not?
Unfortunately, while they are probably already on par with Windows, they aren't ten times better than the Mac:
* The Mac has a bottomless marketing budget. Good luck competing with that, GNOME.
* They've shown a lot more restraint than Microsoft has, probably because macOS is considered a niche product to round out their catalog rather than being their one and only operating system like Windows is for Microsoft. They have even reversed course on a few anti-features, like adding back USB-A ports to the Macbook Pro even though it made the laptop slightly thicker. And unlike Windows RT, they didn't lock down the ARM Macs.
* Those tectonic shifts I mentioned that made Linux usable? They also make the Mac usable, because Wine is open source and Electron is basically its own operating system. Anything truly good that GNOME does, Apple can copy it just like Chrome copied all the really good stuff from Firefox.
What would work IMO is someone with money paying the developers and the designer but force them do do customer support, you don't play with the new shit until most tickets are resolved and customers waiting for response are satisfied. maybe a paid support would help too.
Market success has almost nothing to do with product quality. Well, okay, it does, but only in the sense that you need to not actually be a total fraud. You can get away with dismal quality as long as your marketing is good [1]. In formal terms, software development is a loser's game [2].
[1]: https://danluu.com/nothing-works/
[2]: https://news.ycombinator.com/item?id=26660679
This implies that the churn you're complaining about has nothing to do with market success. You might not like it, but that doesn't mean the failure of Linux on the desktop is actually caused by it.
Too many products equate frictionless == featureless these days and there should be more power when the user needs it.
That's why we built a search extension that improves upon Google specifically in ways that it's weak https://news.ycombinator.com/item?id=30725933 http://abhinavsharma.com/blog/google-alternatives
Turns out the market doesn't know what it wants until market participants see what is being offered.
It's kind of like ad-blockers where you don't have to maintain a list of domains to block. Others do that for you. And then people could create hosted version of their simple box with all the infra taken care of.
It is not a dead end if you're building a tool.
More likely, it exploits human's cognitive weaknesses successfully with a simple way. It learns how how people get their dopamine dose. And there is no going back. You need more and more, more extreme content. More polarization. All you need to do is to open app and get that dose. Is it the same for search engine?
And people make more crazy stuff to get views. How this ends? Not well, probably.
These are firms that are meeting a legitimate need- and that’s the need to feel connected. Tik-tok provides that, and very effectively. Plenty of people get genuine enjoyment out of their product, and meaningful connections do happen thanks to Tik-tok
I don't think they fulfill some gap of the need of feeling of connected in a real way. More like a bandage. We have seen the development of Facebook, Snapchat, Instagram. Their audience is fading on countries who have used them longer time, what went wrong? How is TikTok so good that they try to adapt it on their platforms as well? No way to connect?
Short video clips which might or might not lead for a real conversation. They might offer escape from reality in your lunch break at work.
I understand the perspective of "feeling connected". It brings people together with similar mindsets on entertaining way. Or at least people who seems to enjoy similar things.
On the contrary, is it different than some oldschool cults or religions? Cults which are using psychology writings as base for feeling mutual understanding of themselves. Or religions which share same ideologies and use it as a solution for their problems?
Technology is advancing, is TikTok a modern solution for finding your role and place in the world when it does not make sense and you feel you are alone with your thoughts? Maybe it is, maybe it then fills some gap.
I agree that TikTok is providing entertainment (well, that is what dopamine usually is). It is easier to hook people on short videos which are done by global audience versus Netflix where there is a limited amount of material and they cost a lot to make, when audience on TikTok is mostly making them free and you just pick suitable ones with your algorithms for showing the other audience.
However, there are many problems in this. How it can be abused and how it creates people living on their own bubble, like people on some extreme Facebook groups. When a narrative includes only content that boosts your own thoughts, a reality can be lost. We need some research on this matter, but for some reason social media companies are doing their best to prevent that.
Someone people also really get addicted on the entertainment and cannot stop using it. Well, same thing can apply also for alcohol, but is addiction risk closer to opiates for example?
I don't really believe that executives are thinking for the best of the people, so optimizing platforms to hook users is a quite dangerous play.
This seems a little sensational. Many people (myself and most of my friends included) don't see _any_ "extreme" content or polarization. A quick scroll through my feed is largely nothing but magnet fishing, frog tracking, cats, DIY projects, and geologists talking about rocks. It's enjoyable and arguably a dopamine dose on-demand, but not necessarily a road down more "extreme content and polarization".
To clarify, I don't mean with "extreme" necessarily a bad things, just content which gives you "extreme" emotions.
What kind of TikToks are you watching? Tiktok sees I like watching Japanese videos that compare American and Japanese culture so it knows to show me more of that. Trying to say that showing me videos I like is them just giving me a done of dopamine is such a weird way to phrase it. Should they be instead just constantly show me videos they think I would dislike?
I tend to agree. It's an attempt to deal with bad design by putting in switches, options, and knobs to tweak. That's because 1) it's easier than focus groups, A/B testing, and taking video of users using the thing, 2) design takes some artistic talent, and 3) it lets the programmer blame the users for the problem.
This is a vice of open source people. It's why Linux on the desktop has never taken off. "Just edit /etc/conf/foo/bar/prefs.txt" is not a design. Nor is "On the visual side, you can modify everything about the way things look, even being able to write your own custom CSS."
I'm critical of Google's search, but this is not the way to fix it.
Consider just few more search options, alongside "News", such as "Scholar" and "Noncommercial", for when you're overwhelmed by crap. And "Popular", for when you want the crap. Don't add another interacting dimension of tweaking.
I personally benefitted a lot from the ability to like the reddit app once and then see more real reddit results?
When really innovative stuff is just about removing steps. If a process has 10 steps, remove as many as you can, and now you have something truly innovative on your hands.
I feel like Google did exactly that. Pre Google steps: search for XYZ, wait, check first link, second link, spam, spam, check next page. Post Google: search for XYZ, get XYZ.
Or worse - Search for exact term: get a page full of "Missing X - must include X" links hidden in tiny text below a result, click "Must include X" get the SAME FUCKING RESULTS again, click tools, click the dropdown, select verbatim, finally see decent results
I don't know how to easily fix that though. Simply "crawling the entire internet" is still not a simple problem, let alone doing something more useful with the result than google can. Ahrefs is an interesting business but not what people mean when they say the next google. "Machine learning" but I think google is all over this already (and has been for years).
Google does a bad job at getting user feedback about results while reddit does better so people search reddit, maybe a hybrid is an opportunity.
Crazy from a privacy perspective, but potentially very seductive in terms of convenience.
And that's great if it works! The problem is that once it fails (and, at least in my use-case, it does so quite often), working with it becomes an absolute pain.
I, too, would prefer an omniscient box perfectly answering my questions. But it clearly doesn't exist. And a box with screws to adjust so that I can eventually find what I'm looking for is the second best thing.
Google has preferences also.
It doesn't need much--a single button with "Remove this site from future results".
That would solve 99% of the problem because it would destroy SEO optimization since whole swathes of people would have different criteria due to the blocked websites. It would also tell Google which websites suck if they didn't already know.
Alas, it would also destroy their ad revenue because once a site got overly scummy people would start delisting it.
What TikTok shows to me is that users want specific things when they go to a site: for TikTok, they want fresh entertainment, plain and simple. I go to HN when I want tech discussion. I go to reddit when I want aggregation of niche topics. I go to costco.com when I'm looking to shop.
Google is frankly a horrible experience for ecommerce, it simply cannot compete w/ the likes of Amazon or any retail store website, really. Being a search portal, it's fundamentally incompatible with the concept of evolving through permanence of a hivemind; every new search is like reseting a would-be community to zero all over again, so you cannot gradually build up a collective commons on Google like you can on HN. Youtube has recently gotten pretty bad with cycling fresh content on the front page. For stack overflow sort of stuff, any other search engine does more or less the same. For trivia, Siri/Alexa are fine substitutes. The Google search properties have become mediocre on average; they're not particularly great at any particular thing.
Are many of these domains walled gardens? Yes, think of things like Doordash/Uber Eats grocery catalogs, a ton of brick and mortar shops are choosing to integrate with these delivery apps, and these catalogs are completely invisible to Google. News SERPs on Google are often garbage since they just link to paywalled content half the time, might as well just get a subscription from the actual news outlets. Etc.
IMHO, the next Google is already here, and it's everybody else wisening up to the simple business fact that they need to own the top of their funnels.
That aside, a search engine needs to be really focused on privacy. Customising it implies that the user needs to log in and remain logged in. I'd much rather no do that with a search engine, and I don't want its behaviour to react to my identity either.
It's often said that people don't want more choices, they want to be confident in the choices they make. It's the strength in simple product lineups like good, better, best. Googles lone search box is good example. Apple's product line also tends to be a good example at times.
I think it really depends on the product.
If it's something like Photoshop or Vim which are used by professionals for multiple hours a day and productivity is to be maximised, customisation is not just a nice-to-have but a necessity. Power users appreciate it too.
You're absolutely right in 90% of the cases, though I'm not sure which box web search would fall into.
Google's simple box is not attractive when others can provide the same quality in search results.
site:reddit.com {search_query} site:twitter.com {search_query}
I hope to have this as default at some point because some of the indexed results by default are just terrible.
You are so on point with being opinionated though...
I agree, BUT I would like to share my experience moving from Google to Kagi. Google serves me primarily ads, whether it labels them ads or pretends the results aren't ads, they're mostly ads. I see the same horrible domains pop up frequently, and there's nothing I can do about it. Kagi gives me the ability to remove these domains from my search, and it is INCREDIBLE. How many times has Pinterest barged into your search results?
Now, one could argue that these are domains which no one wants in their search feed. So perhaps this could be solved by being less beholden to shareholders and an advertising model. However, what about domains which most people want, but I don't? For example, I was finding increasingly polarising political content coming from MSNBC. I didn't like their "human" angle on news, and I didn't appreciate their extremely polarising commentary. So I want to remove all MSNBC results from my search feed. I understand I am probably in the minority. Perhaps my political views don't align with the typical customer. Could a well designed search provider anticipate this need of mine without giving me some kind of method to tell the provider that I don't want to see MSNBC? I'm not convinced so.
So I think the answer here is both. Design a search engine well, and give users the ability to customise it.
It's better to focus on the core basics and perfecting it than to add many features and try to support and maintain them especially if they are features that only 1% of the users use.
The old Google was able to do an exact search using Boolean expressions. Man, I miss that all the time....
It's not rocket science to provide a user interface for this. A combination of GUI configuration, keywords, and specific URLs for these cases would go a long way here without bothering the regular user in the slightest. Yes, it's probably slower than in earlier times, pure search volumina and its handling by AI weighting etc. considered. But there is also a "don't want" on the part of Google and the other players.
The useful ideas in here is the "just my data" search, linking into external providers across siloes. Of course, Microsoft, Google & Apple already have this as long as all of your data is held in their ecosystem.
For example: best hotel pool in vegas. Seems simple enough. Circus Circus actually has a waterslide, but if you dive deeper (read a hundred reviews manually) you find out parts are often shut down, and the hotel itself is quite trashy and smells bad and has bad service (explains the really cheap room rate). But do you find that information on blogs? No way, they just include all the hotels with pools and copy in the verbiage directly from the hotel websites or other blogs.
There's probably an actual traveler blog out there that tells you all this and has great information, but it's hidden by all the SEO optimized trash blogs.
And this example can be applied to so many things we do all the time. Try to find a product you want on Amazon without spending half a day sorting through reviews and trash blogs.
So, personally, I think the future will be actual human service. I'd pay a few bucks to call up a service to answer these questions definitively for me.
It was called ChaCha: https://en.wikipedia.org/wiki/ChaCha_(search_engine)
Acquired by Google in 2010 and ...
What are the chances that I cumbents will stand by idly as the "next Google" just takes over? Best case I can think of is the incumbent imitates the upstart and becomes better.
Source: worked at ChaCha.
Customer: How is the newly opened Spa?
Hair dresser: Other customers said the water was cold.
And yet Travelocity (and others) would have told you how terrible Circus Circus is in five minutes or less.
22,000 reviews, 3.2 stars out of 5, among the worst in Vegas (among major hotels) with a gigantic number of reviews. Its room rate alone helps you to begin immediately forming a good conclusion about its quality. This isn't a subtle thing.
Oh, but that's not realiable, one might say. Yes it is. It does a great job of approximating the quality of the hotel in question, and it's an exceptionally easy and fast means to narrow with. It isn't a perfect approach (is it really a 3.1 or 3.3 star quality?!?) and doesn't need to be, it just needs to let you know that Circus Circus is garbage, and it does exactly that.
- good coffeeshops
- tacos
- Korean grocery stores
- places that have traditional style al pastor tacos
- finding a specific coffee brand at a local grocery store
- finding EDM songs similar to a particular song (dullscythe)
- hot chicken
- canolis
and a bunch more similar things. I would think if you posted to the /r/vegas subreddit asking about the coolest hotel pool in vegas, you'd get a bunch of up to date info.
Reddit's own search has become less reliable so I search via DDG like `<query> 'site:reddit.com'`, falling back to `!g <query> 'site:reddit.com'` if the former doesn't get me anywhere.
Not sure what you mean... It's always been an absolute pile of shit
Reddit is changing fast though. It's becoming more walled-off every day as they push to become the next Pinterest, and it's increasingly plagued with spam and bad actors.
Google brought this upon us by lifting SEO spam websites to the front page while pushing down helpful websites written by humans without incentive, like Ask MetaFilter.
Example: check r/sandiego for [insert type of taco] suggestions. There should be thousands of options if you're actually from San Diego but you'll get 5 or 6 at most that stand out a lot, and 1 or 2 that get the most up votes or responses.
Now that may in fact be the "best" (depending on your and their definition) but you might also find that it's just the restaurant that has the most interesting social media presence, or the one you go to in order to signal that you're "one of us". Maybe that's exactly what you want, maybe it's what you want to avoid, but either way it's important to know the mood and trends of the subreddit as well as general reddit culture (and don't tell me it doesn't exist).
Edit: A couple more real examples off the top of my head:
- In some circles, everyone recommends getting a Shure SM7B + a Cloudlifter for recording voice. It's not a terrible option but not really better than many other mics (many of which are higher gain, removing the need for the separate preamp), and even if selecting an SM7B, depending on your audio interface you may not need the Cloudlifter. It's just a preamp, not magic.
- On Reddit's /r/NewZealand, everyone loves Whittaker's chocolate and hates Cadbury. Whittaker's is locally owned and honestly is better than Cadbury, but from reading the discourse you'd think Cadbury was completely disgusting. Any sort of moderate opinion (e.g. liking both) is not allowed.
However, if I was looking for recommendations about a good taco place when visiting San Diego, I would bet that the groupthink recommendations of an active San Diego subreddit would be vastly superior to what you'd get on Google.
Unfortunately, this points to gamification of reddit recommendations as being a major target for the same folks wrecking web search results with their SEO blogspam garbage.
One way to stop this is sort by controversial. It allows you to see what the hivemind votes down at the top.
I also find city subreddits can be much more polarizing and needing controversial sort vs interest subreddits. The only common thing city subreddits share is geography so you get wildly different opinions which controversial sort helps with. The subreddits dedicated to an interest are much more susceptible to the reddit answer.
The other option is search for an /r/askvegas or similar -- if a city subreddit has a dedicated subreddit for questions you can usually get better answers on the asking subreddit, but they are closer to the reddit answer than a general city subreddit will give you.
Also important to at least try searching at first. So many people thought they had a super original question but it'd really been asked and answered to death already.
Maybe that's what would fix Google. Up&down-votes for web results. Would probably be the easiest way to penalize bad sites and promote good niche pages. But after they removed downvoting in Youtube, I wouldn't get my hopes up.
Duh.
I love the idea. However…
You have invented an influencer hellscape. Not to mention the complete subjectiveness of so many things.
I can imagine the first two steps of their process working for a human-assisted search machine. Specifically, the "finding sources of information" and "filtering information" steps. But, I'd imagine the human workers would still need more complex / configurable search tools than Google (which is probably what the workers in the study used to find their sources).
Imagine trusting HN, visiting Amazon and getting all the reviews from other HN users ...
So...a travel agent? Nothing beats talking to a person one on one who is actually trained to do this stuff and personalize it to their client.
We've come full circle.
I get what you're saying but, does anyone really need the "best" here? why not "good" or "great"?
All review site have become some version of "link to amazon and include their description as your in detail review"
> best hotel pool in vegas.
How is finding the best X at something from I assume hundreds if not thousands of choices, a simple question ? In particular, would asking any human give you the actual answer ?
At best we’d be in the same situation as looking at guide books or Michelin guide kind of rankings, created with human oversight but static in nature. More probably we’d get the most reputable hotel with no actual evaluation if it is effectively better than the myriads of others.
And that’s without digging into what you value in pool experiences vs what any evaluation system took as criteria to rank them.
I think this is an example of how we are trying very hard to simplify problems that are inherently complex. And sometimes it kinda seem to work, so people start forgetting it still is complex and rebuilding it from scratch means reapplying magic on it again.
See [1] for Vegas, for example.
I could see a "virtual assistant" version of this where you pay fifty dollars and send a request like "Reservations in Vegas for these dates at a hotel with good pool."
https://www.oyster.com/las-vegas/hotels/
You can also get some decent mileage out of Booking.com and checking reviews for keywords. That won't help you quickly find the best hotel pool, but it will help you find out what people think of a specific hotel's pool (e.g., Circus Circus).
- who pays for the service (ads? users pay? Average user will never use a paid service if a free one is available)
- how to resist attacks against the algorithm (Google has been fighting spam for decades)
- how to personalize without invading privacy, e.g. Google had an option to search through your email in Google search...it's gone now, I wonder why?
Whatever replaces Google will be doing something that Google can't without causing them other problems. The first thing that comes to mind is make them choose traffic vs. advertisers (I don't know, if I had an idea of how to, I would not be writing this), but they're big enough that other wedges could start chipping away at their margins.
The article is titled "The Next Google." I was responding to that, not "A Profitable Also-Ran".
I've been using it for a couple weeks now on my work laptop and for programming-related searches its been great so far. And the usual annoyances that show up at the top of google and DDG don't show up on Kagi (geeks4geeks, etc).
Pull out phone -> Safari -> type "Walmart" -> click on the map -> Maps app opens and starts guiding me
When I switched to Kagi, the flow went like
Pull out phone -> Safari -> type "Walmart" -> top result is Walmart.com... no address to nearest Walmart to be found -> Close Safari -> Open Maps app -> search for Walmart
And it got so annoying to have those extra steps. I know I can change my workflow and get used to it. But it wasn't just directions. It was other basic searches. Like if I needed a phone number for a local business. It drove me crazy. I really hope Kagi gets better at those sort of things. I want them to succeed. But it was just too much friction for me.
or https://you.com/search?q=pyspark%20filter%20array%20element
Since disabling it this way removes the `sendBeacon` function from existence, you should just be prepared to properly handle its absence.
I know little about front end JS dev, but the docs say it’s just a way to send an asynchronous request (no response) with a guarantee that the the request will be sent out, even if the page starts unloading. That’s apparently not a guarantee with the two other to send requests - XmlHttpRequest and fetch api.
I believe products can work that way, offer premium service and features for the very few that need it and a basic service to anyone else. In the end, the free tier is cheaper than what you'd have to spend on marketing otherwise.
Chances are there will be both.
To me the fact that Kagi is not currently VC funded is huge for me as far as adoption. Every customer facing VC funded startup I've worked at inevitably starts to institute increasingly anti-user practices while grinning and talking about "customer first!"
I know it's a huge ask, but if Kagi remains privately funded/non-VC I'll happily pay the moment I can.
I've only been using the service for a short while now but have been enjoying it a lot. The ability to blacklist domains has already dramatically improved my search.
I'm afraid what happens to 1Password since they get big funding recently.
Overall it seems very rare anyone ever considers this an engineering problem. Really, what's stopping you from running a search engine?
But if you must know:
First you need to collect a lot of content from the internet. From many different sites. With very different types of code structure. Broken html. More often than not behind some SPA JS code. Behind robots.txt files and bot protection efforts.
So the first problem to solve would be building a crawler at scale. That is able to crawl anything your users might want to visit but don't know of yet.
Then storage and retrieval. You need to store and update all this content your crawler collected. You need to enrich it with meta data and organize it for efficient retrieval. So that you can surface it to your users when they use your search engine. Indexing, structure, build g connections between content pieces. A lot of interesting things to think about.
Then there is the front end. Make it easy to search, to refine. Surface relevant content for search queries.
OH maybe I forgot, but you probably need to do a bit of engineering to make your system understand the users' search intent.
This is relatively straightforward for a limited search and document space up to a few million entries in your DB. A few million documents should be doable with off the shelf parts.
Bigger than that. I would applaud you if done with orders of magnitude lower than Google. Anyone would.
There are some problems that aren't as big as they seem. Parts of an SPA can't be reliably linked to anyway even if you find interesting text there, so you can just leave them out of the index.
Likewise, there isn't as great of a need to keep a fresh index as it may seem. The odds of a document changing is proportional to how frequently it changes. This is a bit of a paradox, where even if you crawl really aggressively, the most frequently changing documents will still always be out of date. Most documents are relatively stable over time. You can actually use how often you see changes to a document or website to modulate how often you crawl it.
The bad HTML is quite manageable. You really just need to flatten the document to get at the visible text. Even with really broken formatting, that's manageable.
The storage demands are also not as bad as you might think (most documents are tiny, sub 10 Kb), there are ways to lessen the blow on top of that. Both text and indexes can compress extremely well. Since you're paying for disk access by the block, you might as well cram more stuff into a block.
Most of the crawling concerns, in general, can be gotten around by starting off with Common Crawl (even if I do my own crawling, which also is finnicky but manageable).
> This is relatively straightforward for a limited search and document space up to a few million entries in your DB. A few million documents should be doable with off the shelf parts.
Right, so shouldn't the question be how to find the documents that are even candidates for being search results? Most documents are not ever going to be relevant to any query ever. Get rid of that noise and your hardware goes a lot longer.
I'm running a search engine on consumer hardware out of my living room that can index 100 million documents. Go a bit higher budget than a consumer PC, and you've got 5 billion. That goes a long way.
What qualifies? What defines signal, what noise? I agree, that a lot (probably nearly all) pages will receive very, very little traffic/search requests. But are these therefore not relevant?
> I'm running a search engine on consumer hardware out of my living room that can index 100 million documents.
That's extremely cool. I would love to know more. To me an impressive feat already.
> What qualifies? What defines signal, what noise? I agree, that a lot (probably nearly all) pages will receive very, very little traffic/search requests. But are these therefore not relevant?
Now this is a proper difficult problem with (probably) fairly subjective answers. I do however think it's something that warrants serious investigation. It's probably a decent candidate for a machine learning model combined with some manual tweaking for sites similar to wikipedia or github that have absurd amounts of parallel historical content.
Developing heuristics for this is a bit of a hobby horse of mine. It feels tantalizingly almost doable with just a little bit more resources and time than I have.
> That's extremely cool. I would love to know more. To me an impressive feat already.
Yeah it's at <https://search.marginalia.nu/>. I've built all the software myself from scratch in Java[1], and I'm doing my own crawling and indexing. The machine it's on is a Ryzen 3900X with 128 Gb RAM. Most of the index is on a single 1 Tb consumer grade SSD.
I do use a MariaDB database for some metadata, but I think it will have to go as its hardware demands is becoming a serious bottleneck.
[1] Despite using Java, I should say regarding the index. This is approaches sunk cost at this point. Building a search engine index is not something Java is at all suitable for, its limited low-level I/O capabilities is incredibly handicapping.
Worth pointing out that Lucene/Solr, the biggest open source player, is also Java!
* You can only allocate on-heap arrays of 2 billion items.
* On-heap arrays have a massive size overhead in terms of GC book-keeping.
* You can only allocate off-heap memory map 2 Gb at a time.
* This also goes for memory mapped areas.
* You have no control over the lifecycle of mapped memory and off-heap memory. They get cleared if and when the GC feels like it.
* You have no madvise capabilities
* The language barely acknowledges unsigned types
No you didn't. All good. And I learned a lot from the extended answer. So I am thankful for the explanation.
> Developing heuristics for this is a bit of a hobby horse of mine. It feels tantalizingly almost doable with just a little bit more resources and time than I have.
I can totally understand the feeling. There are quite a few things that I'd like to go deeper into either at work or in private. But alas time.
> Now this is a proper difficult problem with (probably) fairly subjective answers.
I agree. And I don't have answers ready. A lot boils down to preference. Personally, for example I prefer written content over video. Except in a few areas were I like (some) explanatory videos. To me it comes down to the question of how easy I can skim the content when I am looking for an answer.
On the other hand - for deep immersion into a topic I use multiple media formats.
In terms of web search I sadly nowadays need to sift through a lot of seo-fied content that is there either to build a (personal) brand or to attract clicks for advertising revenue/affiliate revenue.
So in principle I agree with you on the noise problem. Still I also believe that there are real great gems to be found in the long tail. When I still feel like I came late to the party, but when I started out in the web in '97 there were so many lovely, quirky sites. So many places that people had put a lot of time, energy and thought into. And sites so packed full of information that I came away not only with more knowledge, but in awe that somebody would give this knowledge away for free.
There also were quite a number of horrible sites (my first ones probably included). So there was a noise vs. signal problem back then. Maybe not to the extent today, though.
> The machine it's on is a Ryzen 3900X with 128 Gb RAM. Most of the index is on a single 1 Tb consumer grade SSD.
Call me impressed. Sounds absolutely cool.
So even with a raid setup for redundancy this is doable.
May I ask how you decide to add me content? Do you follow links? Do you use other search engines' results as a starting point?
I could probably shoot many more questions, but don't want to be a nuisance.
Thanks for your time already.
I initially did basically a DFS-walk originating at a few websites I liked, with some filtering criteria that deprioritized websites that didn't look too interesting. Now that I have a fairly comprehensive mapping of the space I want to index, I use a few factors like frequent outbound links from highly ranking domains to inform which new sites to index.
> I could probably shoot many more questions, but don't want to be a nuisance.
No worry, I love to talk about this stuff.
And I learned already a lot in this thread after the explanations unfolded.
The initial statement sounded exactly like the armchair "experts" one so often encounters. Actually this was for a long time the first time that there is a person with substantial experience in the problem space behind such a statement.
Maybe - but [marginalia_nu](https://news.ycombinator.com/user?id=marginalia_nu) isn't an armchair expert - they've actually implemented theor own publically available search engine - which is linked in their profile.
Should be said I do wear through this SSD at a pretty alarming rate. I'm at 193 TBW on this disk since I started using it as an index less than a year ago.
I do have a bunch of mechanical drives I use for archiving and as intermediate working areas as well, but the index itself is on an SSD.
The solution for o both of these might actually be a paid service. If you have a paid service, there is a possibility of it being profitable with much fewer users. As an example, let’s say you have 1,000,000 users at $10/month, that is a $10,000,000/month which might be enough to run the service and provide a comfortable profit.
With regards to the spam issue, the fact that you have a small user base would be to your advantage. Because there are so many Google users, it is in websites’ economic interests to spend money to try to game the algorithms. With much fewer users, your paid search users may not be worth it for the sites to spend money trying to game your algorithms.
From what I'm seeing, if you could create a bot free eco system, people will pay for it.
The question is "can you make it bot free". This is gonna be the next trillion dollar company.
At the moment, spamming Google seems to be trivial with no long-term penalties if you get caught doing something nasty.
A simple rule (manually enforced on a case-by-case basis) that would ban your brand/domain for a year if you get caught breaking the rules would get Pinterest into compliance from day 1 for example.
Using ads/analytics/affiliate links as a negative ranking signal would make a lot of blogspam/listicles/clickbait disappear if their only funding method immediately makes them rank much lower below where they are no longer profitable.
This isn’t a hypothetical example - Google actually includes in their webmaster tools a “disavow links” capability so sites can avoid getting punished for bad actors trying to make them look bad. But you can imagine if the penalties were even more severe other folks may get caught up in an unforgiving dragnet with no judge or jury and no way to appeal.
My main point is that people will find ways to game the system, and usually sharp edges (“harsh punishments”) on any system will be taken advantage of by actors, and unfairly penalize others.
Not to mention, if free search engines keep devolving into an endless sea of spam, people may have no choice but to start paying. There's plenty of things out there people pay for not necessarily by choice but because there's nothing else out there that would accomplish the task at hand.
If I can, I'm happy to pay, so others doesn't need to. I don't understand why I'm in minority and most of the people thinking only about themselves.
I'm happy to pay because for many years I also had to use free services, paid by others (or ads, but ads I'm blocking however I can).
That said, I do like the feature of ignoring entire domains. Google used to allow that
A respected friend or colleague will tell you that the next Google just works better. That’s what made Google search win: it just worked better. It’s now I found out about it and probably how you did if you’re pushing 50.
We didn’t care about pagerank or know what it is. That came after we used it and wanted to know why it worked better - or we wanted to manipulate he results.
We didn’t read a blog post telling us what’s good.
We were using another search engine and Google arrived on the scene and it was just WAY better. I was using alltheweb. Friends were using other engines. In weeks, everyone smart and productive was using Google.
Google/Alphabet may be a large company now, but never forget how they started. They were just so good we couldn’t ignore them.
That’s how good the next Google will need to be.
The simplest explanation is the scene from from Wall-E with the captain of the ship discussing with the A.I. It will be fluently integrated in our everyday lives.
An input box that spits out links is _exactly_ what I want.
I think achieving this would be worth billions, at least?
We really tend to underestimate the amount of spam on the internet.
Google does badly here: it overcorrects and you get only super safe results. Yandex? Unless you get lucky, lots of spam.
You'll still get spam if that's all that matches your query, but now all it takes is for someone to make a page matching the query without the aforementioned items to outrank the spam results. You wouldn't need to append "site:reddit.com" to your queries because the (mostly) non-commercial Reddit results would automatically outrank all the blogspam and listicles.
If ads were downranked it would make a lot of spam/clickbait/listicles unprofitable overnight as they'd rank low enough that the costs of creating & maintaining the spam site/content would outgrow the returns from ad impressions.
This would be a really interesting experiment: A search engine that ranks websites by the amount of Ads and other spam that they contain.
I'd pay /mo for a good one. Why not? Currently trying Kagi.
SEO is killing Google and it needs to be addressed before it kills the internet. Having the opportunity to build an app that solves a search, instead of perpetuating a spam system, has been really rewarding... and we're just getting started.
I've been trying out Kagi as my default search engine, but will give You a try next.
When I'm researching stuff, I go thru a whole page of google result, and 'open in new tab'...90% of the time it's crap, and I really wish I could signal google 'this was useless'.
I currently recommend Kagi.com, it can block but also pin selected domains at the top of the results list.
In fact often you don't even want the information, you just want to solve a problem. I don't know about you, but I don't like learning all about air conditioners and spending time finding the product available in my area with the highest quality/price ratio that fits into my budget. I just want the best air conditioner for me.
Providing links is the only thing that scales, since you can delegate the aggregation work to specialists and you van verify the source.
I don't mean that in a derogatory sense, I just strongly believe that in 20 years we'll look back at our Googling years and wonder how we managed to find anything at all. Who know what the next pattern of information retrieval will be, but I personally think it'll be an even bigger jump than the jump from reference desks and encyclopedias to Google.
Edit: okay, maybe not 20 years. Decades, perhaps.
https://horses.extension.org/what-is-the-carbon-footprint-of...
> The digestive process of horses produces far less methane than the digestive system of cattle and sheep. (...)
Also the Ford company’s dogged resistance to listening to customer wants (“Any colour you want so long as it is black”) hurt Ford as soon as they had to compete on anything but price, with GM finding success with “a car for every purse and purpose”. From 1921 to 1927 Ford went from controlling 65% of the American car market to just 15%.
All that to say: “innovator vision that flies contrary to common sentiment about what people want” is often overrated and can be a trap. That’s not to say that the customer is always right… but they’re not always wrong either.
The same goes for DuckDuckGo and others. All of the above use the Bing search API for the majority of their web results which for most use cases is not economically sustainable.
I do think there is a large swath of users who will pay a subscription for a truly great search engine offering, but indexing has to be at the core of this offering, at least for me. If users realized this is just Bing results with a few enhancements and additional result sources mixed in, they might not be as willing to pay for what they could technically get for free elsewhere, albeit with significantly less privacy.
That said I wish Kagi all the luck in the world. As the original dev who planned out and built the initial backend implementation in Crystal and put together their early engineering team, I can at least say they are building on rock-solid, very fast and privacy-oriented foundations, and this is a truly web-scale product in terms of the infrastructure design.
I tried Kagi as the default search engine for a few days and realized just how many little things Google does that I prefer, but never noticed until now. Not strictly related to search results, but things like converting units (Kagi has this, but it didn't always work), getting a time in a timezone (same), shopping results, the QA answers, etc. It's a ton of work to implement all that. For now I switched back, but I'll keep an eye out.
Honestly I’d be happy with a search engine that does search well, and use dedicated tools for the other widgets.
I think browsers could do a better work at using the omnibar as a command line tool to cover these cases. For instance I’d really wish it would be easier to map incantations like “/mtg intl” and have a predefined action executed. Bookmark keywords are already half the way there.
All the more reason for any serious competitor to try to do it themselves, yet no one does.
and to make matters worse, 95% or more of the web isn't even being shown to users and so I keep getting the same stupid search results: which is fine for many cases but not all cases. sometimes I need more than just another post by NBC.
I will say, google search is very good at technical searches, that's for sure. But, for example, they totally suck if you're looking information or doing research on financial matters or economics.
Unless your technical search, for example, contains jargon with a similar spelling to or other sense synonymous with a common term that Google will helpfully also consider to match your query. I think it's actually rather mediocre for technical searches now that they've gone so deep into stemming and such and lessened the ability to use operators to refine queries. And, of course, the lack of support for symbols makes many technical queries extremely difficult.
Instead of pagerank we need peoplerank. Let's say we build a new elgooG search engine that you can customize by indicating which people you trust. For example I say to trust Paul Graham about the topics entrepreneur and startup. Then, authors like Paul can register a key on elgooG and sign their online articles with that key.
When I search topics related to startups, elgooG knows I trust Paul and will try to find articles from him or anyone that he trusts, or even 3rd grade connections.
It is very difficult to fool such a search engine with SEO optimization, since the user actually indicates which people to trust.
I'm pretyy confident it would be a good solution, but I'm afraid it's very hard to start up.
If you steal my idea, please consider giving me a 1% stake (or less) in your company ;D.
Which article do rank higher: one by Melcher that is highly ranked by large number of "random" individuals or the one recommended by KingsEgg?
Maybe a combination of trust and reputation would work better, but you might still get groups of people vouching for each other's recommendations. I guess it's not an easy problem.
Glad to disagree, DuckDuckGo is good enough if not great.
For anything that isn’t the least bit obscure or technical. (Same with just about every search engine)
IIRC, someone on hn said that search engines are only good for things you already know are there, not for finding new information or different perspectives.
And I use it in two languages. Not bad.
For other results I find myself reaching to a bookmarked searx instance. I thought it just agreggates results but somehow it gives me different stuff.
And you also get magnet links :)
It's sad what today's web has become. Each year you see it degrade and there's no real fix I'm afraid.
1. Let me ban domains like pinterest, quora, stackoverflow clones, stock image sites, etc without requiring a chrome extension.
2. Do what I ask it to do. Don't be too smart. Bring back the plus sign, minus sign, double quotes, tilde which have been deprecated over these years and stop polluting the results with what it thinks I want.
3. A new feature where I can search inside the top 100 search results. Where I can narrow down the search results using additional filters like I do on amazon searching for products. So i can say "5000mah -clickbank" in the top-100 search results to weed out spam and narrow my search accurately.
A world without that choice, where everyone's experience is equally bad ... how sad that would be. Just ask your average east-European soviet-era survivor.
Here's a dead simple use case that is just unnecessarily frustrating. As a user, I want a high res picture of a buffalo. So:
a. I do an image search for "buffalo" and the results page contains thumbnails of buffalos.
b. I click the first result, it's tagged with "nationalgeographic.com" so it's probably gonna be good. Instead of the image, I get another page, but with a slightly bigger picture than a thumbnail. When you over over it, it says "3,072 x 3,072" but the image itself is clearly not that resolution.
c. So I click that image, and it opens a new goddamn tab of nationalgeographic.com's web site, with another picture, still not the promised 3,072 x 3,072! WTF!? When I try scrolling down to look for the raw image somewhere I'm hit with an E-mail signup-wall. Good grief!
d. Little did I know, if at step b. I instead right-clicked on the image and selected "Open Image in a new tab", I'd have gotten the image I was looking for. Thanks, Google, for hiding the 99.9% use case that people want to do.
The actual user experience should have been:
a. I do an image search for "buffalo" and the results page contains thumbnails of buffalos next to clickable .jpg links. The End.
(1) https://9to5google.com/2018/02/09/google-images-features-get...
Why should I be forced to look at a page full of purple links when searching for some specific programming topic? It feels like there's a bunch of potentially-fake developer blogs that copy/paste from Azure documentation, without adding anything novel.
I currently recommend Kagi.com, it can block but also pin selected domains at the top of the results list.
The dream search engine would always be a click/swipe away and have access to all of your personal data/documents and the web, if Apple wants it CMD+SPACE could really be that.
For example when I type "best humidifiers" I want a suggestion that is ideally based on accurate anecdotes of people who've experienced a brand in a variety of context and edge cases.
What google gives me is a cnet article listing Top 10 in 20XX. This is why I add reddit to the end of the search query now.
Don't see it mentioned on Wikipedia what the issues were unfortunately. Interesting though, thanks for sharing.
This is a red flag to me. I would like if this were true for a next gen anything, but in my experience truly next gen experiences - the kind that spread like wildfire and displace incumbents - have no reason to make promises about privacy or ads. Why would they, if they have a product that consumers want to use?
The person who asked 'is HN becoming an echo chamber?'
would've got their answer by now.
Anyway, Google is getting less relevant because the technology is getting better than "good enough", and any additional tech that Google adds is not really all that useful. It's like PCs. You don't need a faster one because your old one can do word processing just fine.
If I search for "what's the weather now" I probably want to get an answer "like an oracle" as you say. But if I search for "riaa vs napster" I don't want an "oracle answer", I want a list of several websites so that I can learn more about the case. I'd like a search result for court documents, but also commentary from news websites and blogs, and maybe wikipedia. I want to open those tabs and come out the other way with more information, but there was no "oracle answer" to be provided.
Compete with google? Who gives a shit. I just want to be able to hit a full text index that points back to URLs. I don't need instant PhD-tier answers for life questions.
> Some days I wonder what would stop me writing my own web crawler and building my own damn search engine from scratch.
A few things:
* Storing the index. For example if I search for "Sushi", google says there is 1.3 Billion results. That's already 19.3 GiB just to store the relationship between "sushi" and its results. (If you use UUIDs) And you need to do that for most words in the english dictionnary. And I'm assuming you're not even storing what you crawl for debugging/re-indexing.
* Website admins might throttle you. For example, discourse — a website which runs most discussion for Rust, Ocaml, and many other projects — blocked the bingbot at some point[1] due to its crawling aggressivity. Cloudflare, which is the front-facing caching service for almost half of the internet now, is notoriously anti-crawling. (It will quickly display captchas)
* grep over the web will yield you bad results. (most of the time) You want a search engine which at least groups words semantically. If I search for horses, I'm usually also interested in mares and ponies.
[1] https://meta.discourse.org/t/bingbot-is-no-longer-default-th...
In fact, I am excited about their new feature multimodal search https://blog.google/products/search/introducing-mum/
Nobody uses yahoo search today...
Why is this always a given? Yes, privacy is good, but honestly, I find what I'm looking for in the first few links about 99% of the time with Google, because of the lack of privacy. They have 20 years of search history on me, including maps searches. They know where I live and where I go and what I like and what I buy, and they can read all my email.
And I get better search results because of it.
If I search for [haircut], I get the cheap places near me, because Google knows I'm cheap when it comes to haircuts, because they've seen where I've gone before. To get that on an anonymous search engine, I'd have to search for [cheap haircut near $home_address], so now I've had to type extra words and I've given up my privacy anyway, just with extra effort.
I'm not the biggest fan of Google knowing that much about me, but I also know it gets me great results, and that's a tradeoff I'm willing to make (and lots of other people are too).
- does my current location matter for the search?
- am I trying to spend money (shopping), or trying to get information (facts)?
Welcome to my side project, a better organized directory of domain specific search engines, based on duckduckgo !bang operators https://github.com/MoserMichael/duckduckbang/blob/master/REA...
The project also adds some explanation on the role of each search engine, also the page has been auto translated into several languages.
Ideally this directory/thesaurus should help you with finding an appropriated domain specific search engine for a given task, without the need of snooping on the part of the search engine.
[1] of course they also need the snooping for targeted advertising, but that's a different story!
- The first example is not much more than Google with a styling chrome extension
- The second doesnt bring anything more than Google when you are searching about webcontent. Some ideas on filters are interesting but this is not user friendly
- You is powered by AI, so it is basicaly giving not what you want except when you ask for the general case (the case Google works pretty well for)
- Andi is a chatbot. Which is basically Google + only the first answer
Brave Search Goggles, (what most people call collections, what Pinterest calls a board, what Twitter call List) is for me what is going to be the real innovation of search Engine. They are a way to bring collaboration to enrich content, but nobody has cracked the UX to make it work yet
My biggest problem with it is that the alpha version still has errors and is weak for searches like local businesses. Also, big problem is I'm in Australia and everything is in imperial units. Having said that, however, I've been using it since it became available a few weeks ago and it has already improved a surprising amount. And the question answering if you ask specific-enough questions can blow your mind.
Then click on "View in Reader" on the results. This is not like Google dude.
People on the Discord are trying crazy questions like that and reporting what it does good or bad on. It screws some up but it is cool. It finds stuff Google doesn't.
To be honest that's impressive for complex queries
in some cases, those results are baked into the core search experience, e.g., click news or recipes on main search at https://breezethat.com
in other cases, we're doing stand-alone drops as we experiment with different curation &/or technical challenges for different topics:
- college scholarships -- wide quality variance, https://breezethat.com/b/scholarships
- golf expert -- sheer breadth of websites, https://breezethat.com/x/demo-golf-search
- gender filter -- getting the dials just right, https://breezethat.com/p/gender-filter-beta
while it is the case that we're doing curation internally at the moment, we fully expect to open that up in some way as we scale; that will likely be after our next release to (1) fix our really awful mobile UX and (2) to deploy some version of our time filter, which is a core part of our thesis about easily tracking topics over time, e.g., this thread with some March Madness examples, https://twitter.com/HeyBreezeThat/status/1511397583209570311
The earlier Google, when Google was at its best in both performance and results, was incredibly simple. It indexed the entire web. It returned keyword matched results based on relevancy. There was no customization required. No editorialized results. No Big Brothering the user against thoughtcrime or wrongthink. No promotion of The Current Thing. Just information, whether good or bad, true or false, as a search engine should be.
Something that can replicate that will have a good chance of success, but I think the longstanding challenge to both Google and competitors is how to rank material in an increasingly siloed and ideological web filled with agendas, astroturfing, SEO optimization, and spam.
You can definitely find these on Google.
https://you.com/search?q=numpy+array+filter+elements+python - results include AI writing code, StackOverflow, Arxiv papers etc.
They also started using dark patterns in the UI. For instance, some search results will bring up an obnoxious popup asking to enable location services. It blocks off a good chunk of the visible screen
This is on a desktop machine without GPS. And it sometimes pops up for search terms that wouldn't have a location associated with it.
And you can't disable it.
I'll probably take a look at Kagi when I get an invite.
- Files (Box/DropBox/Google Drive/SharePoint)
- Static(ish) Pages (Wikis/internal CMS's/Actual websites hosted on some random nginx/apache)
- Web Apps (eg Internal Learning Platforms or helper HR apps or whatever)
- Chat (Slack/Discord/Teams/??)
- Files stored on my actual laptop
- Source Control (eg GitHub/GitLab/whatever ... which have their own wikis/pages/content)
Is it just me? Anyone else feel like finding something across the above mediums is difficult?
The only company that could theoretically build a web search engine due to the amount of data they have is Meta.
Search is one of the biggest moats ever built.
You have to provide an excuse, or perhaps an experience, that gets people to come back. Just one niche, is all you need.
You're not competing with Google the search engine, or even Google company. You're competing against google the verb.
It's hard to do (1) well given Google's data moat and how good that makes them at head queries. It's rare that people stick with one of these alternatives because they struggle to be as good in the majority of simple queries.
We built a few such search engines before choosing a different route — we made something that augments Google (or Amazon, or whatever other search engine you use) and improves it in areas that it's weak in.
I posted our thoughts on this a few weeks ago on HN that generated some discussion as well — http://abhinavsharma.com/blog/google-alternatives https://news.ycombinator.com/item?id=30725933
The apps we built are a mobile browser extension for iOS and a desktop browser extension
https://apps.apple.com/us/app/hyperweb/id1581824571 https://chrome.google.com/webstore/detail/hypersearch/feojag...
It's one approach, but we're trying to be more non zero-sum about it, and consider that the search engine doesn't have to be replaced 1:1 with a better search engine.
Why? _NeevaScope_ seems like _NervoScope_. And _NervoScope_ is a lot better known term.
Bing - maybe. DDG - it used to be much more of an alternative, now a little less, with their decision to actively censor their content. Still, they - supposedly - don't collect and store information about you. That's the sense in which they're an alternative. And in Google, there is massive, manipulation of results with all sort of commercial biases. In DDG we don't know that that is the case.
> For this new generation, privacy is necessary, and invasive ads are not an option.
Hmm. I notice they didn't didn't say search results manipulation is not an option.
> Why should everyone have the same search experience?
As far as search results are concerned - so that you don't get to play games with what's more and less visible on the Internet? For the sake of fairness?
> We all have our own preferences about how things should look and work.
Oh, so you want to _flaunt_ how you'll manipulate people's search results to fit your interests? I'm sorry, I mean "the default preferences"?
So
- optional verbatim and prioritized terms (like Google once did it)
- optional logical operators (and, or)
- a separate input box for context (either, specific terms like "product", "shopping", "legal", or some assorted terms to indicate a field of meaning and/or associated concepts, time span, etc)
It's the latter, where the search engine may shine and users may evoke its sophisticated intelligence, but, please, don't reinterpret the principal search terms to concepts that maybe just related. I do understand that this may not be for everyone, nor may it be the sole interface, but there should be a niche for this.
Example, wanting to find something about the early press coverage of System/360:
- Search term: (IBM || "big blue") && System/360
- Context: news 1965,1966 computer
(Compare the example given above. I don’t think that you can do a search like this with Google today. This should also help with issues like newer content masking established, older one.)
Edit: As it is, search engines return a quite diffuse field of information, which may not what you want, when you're looking for specific information. Some of this is related to the confusion of search terms and categorial and/or contextual information, while its also due to the eagerness to return any information at all (and as current as possible). Sometimes, this may be even worse than returning nothing. Also, this may lead to an eventual erosion of information in the long term.
For centuries, human knowledge (at least, in Western cultures) has been organized in terms of content and categories. In a classic library, a research like the one mentioned above would have been easy. As it is, this kind of structured concept doesn't work with search engines anymore, or, at least, not too well.
I disagree. I use Bing for about half of my searching because Google simply struggles with things like exact phrase matching in many cases. Where Google falls off, Bing tends to succeed. Though Bing also sometimes lags behind, where Google tends to do better, hence my split usage.
Altavista was dominant internet search engine in the 1990s but search quality was bad compared to what could've be done.
Google figured out machine learning was the key to success e.g. PageRank.
Google got incorporated in 1998.
Good leadership(Larry, Sergey and Schmidt) and vivid vision propelled Google to 90% of market share in search.
World Wide Web was drastically smaller two decades and one decade ago and Google was still able to "handle" it with already established machine learning algorithms.
Fast forward to today World Wide Web is zettabytes big and not even Google can rank it "reasonably" and efficiently. Google doesn't care that much about search quality anymore because they have 90% market share in search, billions flowing in ad revenue plus 99 other projects they need to manage and develop.
DKB is a German bank. Shouldn't be unknown in expat circles because it used to be close to the only one accepting customers living abroad and actively marketing towards them. Not sure whether they still do so, but at least they haven't made any attempts to get rid of me (knocking on wood).
Offtopic background: International banking (for small customers) is a nightmare because of compliance issues even within the EU which is supposed to he a single market. Of course as soon as you have the US in the picture it gets completely hopeless (because of crazy rules for their own citizens and them acting world police, i.e. requesting others than their own citizens and businesses to comply).
No one needs filters, backround colors and all that useless fluff - that doesn't really hit the nail on the head. I just want to get a useful information quickly. Google is arguably unbeatable when it comes to dev related queries, but a complete trash with pretty much anything else. Now, the problem is with all of these appalling websites filled with ads, pop ups, hidden agendas of selling me something etc. That's on google because their business model depends on serving these results.
A new kind of search engine does not run on ad revenue but on value add for the user in exchange for their subscription fee.
People don’t want this. My first boss told me he would bet on regular TV if only streaming was left. He didn’t account for recommendation algorithms, so it’s easy to prove him wrong. But the takeaway here is that people prefer things working for them out of the box. That’s the secret to Apple’s or Google’s success: It just works.
Just works can mean that you’ll get “some” search results or a TV show that matches your preferences.
Almost nobody would like to put effort into customizing their experience.
Google provides “good” search results for a majority of users. From a business perspective it would be a wrong decision to enable customization for a neglectible fraction of users if the vast majority just want to “consume”.
Yeah, and that's why Kagi.com is bigger and bigger.
I would also predict search to be much more compute intensive and therefore more expensive. The ad model is likely not going to be sufficient to pay for compute resources.
> That Kagi went bankrupt in an unfortunate turn of events. We liked the name and acquired it when we got the chance.
SANCTION is an experimental search engine powered by the popular experimental search engine Million Short – an experiment within an experiment. (This is alpha and buggy!) With SANCTION you can choose to remove results from any Country you wish, for whatever reason.
Result removal is based on both the ccTLD and geographic location of where the page is hosted (a 1st degree connection). We have also provided the ability to remove results based on the ccTLD and location the scripts a particular page includes via script tags (a 2nd degree connection).
I think one strong contributing factor to Google's success is its simplicity. All the listed competitors add a lot of complexity imho. While all the customization buttons, knobs, lenses, meta crawling, code generation features etc might add some value for the advanced and technically skilled user, it provides rather little value for the average user who just wants to look up a cooking recipe.
So maybe when searching for "the next Google" the interesting question is not "what search features can be added", but "how can search even be simpler than using Google".
Google also has a few good things going for them:
- it’s the standard when it comes to search and is setup by default on just about every platform
- People use the brand name itself when talking about looking stuff up (no one says “just duck it”)
- They control the ad market, the standards for what a “optimized” site is, and the web standards themselves.
I work for Neeva, and this is a big part of why I left Google to join Neeva. There has to be a better experience, and it doesn't start from another business that works just like Google. It has to be a different kind of business. Neeva does not make money from showing you ads, so it can provide a different search experience... a simpler search experience, like the original google even, but it can go further...
With the Neeva app for example as you start typing in the URL bar, it will take your input as search suggestions (just as any other browser + search engine would) but instead of just showing you completed search suggestion, Neeva will show you the results from running those searches inline. The idea being that maybe those results will be helpful to you and make it so you don't even need to go to the search results page. You can just take the result right there from the URL bar suggestions drop down. Saves you time. Simpler.
Stuff like that. There's a swim lane of innovation and ideas on how searching and browsing can be better that is just really hard for Google to build, even though many of these ideas are thought of inside the walls of Google. They just can't ship them if they are stuck being beholden to their search ads model.
Another great example... ever wonder why Google isn't working to make it so Chrome doesn't have a million tabs at the top of your browser? It gets to the point where it is hard to get back to what you were doing. Me, I just end up closing the tabs, declaring tab bankruptcy. Google is okay with that because it means I have to search again. The Chrome team wants to fix this but it is hard to do so as it would result in people searching less often!
Again, just means there is opportunity for a simpler better experience to be had and Google won't be the ones creating it.
The "I'm feeling lucky" button would never be added to Google if it didn't already exist. It was grandfathered in.
Sure, you could still make a bot that auto-generates blogspam without the aforementioned things and rank as high as before, but what's the point if you removed the very things that get you paid?
There's plenty of things out there that are trivially spammable but outside of isolated cases (where the objective is to cause damage/annoyance rather than profit) nobody is spamming them because there's no profit to be made by doing so.
This sounds like it was written by the infamous Dropbox commenter.
No, most people have never given a single thought to this.
It might plenty to create a nice business, but a bit absurd to call it the next Google.
Could be Wikipedia-style cultivated if enough people know and care.
Probably because they're stuck in a local maximum with no idea how much better it can be. They may not even be aware that there are sites out there that have quality non-commercial content if all they've been given for the last decade is spam to the point where spam has been normalized.
I predict that in 10 years Google Search will still be dominant, and we'll still be complaining about it.
Edit: on the wait list
They don't want to.
https://kagi.com/faq#Google-killer
> We don't want to kill Google :) Google serves a purpose in the world and it did help enable the modern society to exist, with all its marvels and flaws. Heck, it even enables Kagi to exist! > > Think of Kagi as a small, premium brand, providing a very different, tailor-made, search experience.
I get less ads, more privacy, and pretty much the same or better results. Plus I can use my down arrows to traverse the results list!
This pitch voice makes me want to gouge my fucking eyes out.
I feel a lot of modern software is becoming very authoritarian. “We know better than you what you need”.
Do we really need to store everything ? Maybe a local "ai magic" could select the specific information we regularly search for and sync from an open public index.
In the case we need results it could sync and/or request those bits we are not indexing locally.
Just thinking out loud
Privacy no need to explain why this is important
Resilience if google/any other search engine goes down it usually mean chaos
You have total control of what is shown to you so no one can manipulate
what is shown to you(at least not that easily) assuming it is open sourced
of course
https://en.wikipedia.org/wiki/Wikipedia:Size_of_Wikipedia
20GB, which is tiny these days. Fits on a $10 USB key.
I think with some of that AI magic we'll soon be in a situation where most of the traditional (non-proprietary) human knowledge you ever need (text, images, 3d models of machines, anatomy, etc.) will be immediately accessible for free via an open-source knowledge engine.
With a commodity network connection, that knowledge engine can keep itself apprised of the post-traditional/proprietary knowledge by doing all the drudgerous work of reading through the world of Ads to get to today's news, etc..
Mate that with more AI magic to make it easy to conversationally interact with the engine and you get an agent who can represent you online and provide a distilled interface of online for you. Your digital twin or avatar.
I hope the open source community can come up with a solution for this. Maybe there are already projects with this goal, if not someone should start it.
It will improve our collective knowledge and at times like this were disinformation is really high we need technology/tools to reduce it
That's all I want from a search engine. I don't care about shopping or maps or any of the other 'features' all of these alternative are trying to implement. I want a gnu style search engine. I want something that "does one thing, and does it well," and I want that one thing to be 'search.'
If only we could get some sql-like powers in that simple input box. Maybe I'm not their target audience...who knows.
- People Want Customization - Searching for web content would be better if users did half the work.
Both are ridiculous. See here: https://chapra.blog/search-is-still-dead-demand-side-woes-72...
"[...]We do not log or associate searches with an account.[...]" why would I trust that and why would a create an account in the first place ?
I don't use it as my primary search engine as I often want images and news related to the search. But for any search that is going to get flooded with SEO type stuff it can get rid of a lot of the crud very easily.
Here's wishing for this in all search!
Google replaced other search engines that had just an input box...
I have a fundamental disagreement with this. The best search engine is still an input box that spits out links based on what was typed. This allows the user to express what they're looking for, their query, in most natural terms.
Everything else is secondary. At least for me..
"The next Google can’t just be an input box that spits out links. We need new thinking to create something much better than what came before."
Could not agree more.
Great roundup of some exciting search alternatives. Thanks for sharing.
Nobody seems to have The Next Big Idea for a better search engine yet.
I think the bigger reason why search isn’t getting much better is that incremental technical improvements are being offset by Google trading their dominance for greater shareholder value. A safe business can take away or buy-and-bury features that are better for the customer but worse for the business’s top line. Things like the Power Search API, or search that integrates with other platforms that hurt Alphabet owned businesses.
Traditionally big innovation breaks some socio-political constraint on market participation. I don’t see how AI will do that yet.
I don't know anything about AI or the relationship between market participation and innovation. All I know is that LLMs are a useful tool that is already displacing search in many parts of my life.
I will never ever ever ever spend time going into the settings of a search engine to customize my search experience. Not when Google does a good enough job for most of my tasks.
Privacy aside, I find the results much more helpful on average these days, and weeding past sponsored links is enough of a hassle within google search results to consider it a UX QOL downgrade
Search engines used to seem better because they were filtering from a small pond of reasonably clear water, not an ocean of sewage.
Google is ripe for disintermediation by a competitor that doesn't siphon cash off search to subsidize other things.
Now, all we need is a technology shift that erodes their moat.
No we don't. Google needs to go back to what made it successful: showing web pages that have the words you search for, untainted by SEO shit.
Ok, so how does the new generation pay for itself? I couldn't find that anywhere in the article.
On first pages, many clickbaits and generated spam sites.
I'll happily pay for it.
Will the search take 15 seconds or 1.5?
if your advertisement for your search engine is not easily searchable, then...
I also don't want to customize anything anymore. I stopped playing RPGs, and I don't wanna RPG my engines too.
I don't want any features. I want a search bar, a search button, and results that relate to my query based on more than simply matching a keyword to a query word, but based on less than whatever Google factors in these days and certainly not based on whether someone has paid to have their domain featured for a query. I have never clicked a paid promoted search link on Google, or any other service, and I never will. If you have to pay for your content to be displayed as matching a query, it probably shouldn't be. Content should speak for itself.
The one thing I 100% never want to see is a selection of results that are at all influenced by what lots of other people have clicked on. I want to see results that pertain to my query, regardless of whether they have attracted the interest of others who have searched for something the same/similar.
I also don't want to see results that are influenced by my previous searches, by my location, or by data gathered about my Google account unless I specifically want those to be factored into my search.
Features like ways to refine search like inurl: or intitle:? Yes, cool, have them available.
Also, I expect a search engine to load fast. I don't expect a ton of JavaScript to be required for an input box and a button, and I don't want doodles of the day and other junk. They're fun, sure, but at least let me turn that off because I'm a psychopath.
I don't think we'll see a true Google competitor. Would I pay for a search engine that was just as good but kept it simple and returned better results than the trash Google can come up with? Yeah, probably, but I think all that will happen is that Google itself will come up with Google Premium Search, run it for two years and then shut it down.
I don't think for one second think that search is easy, it most definitely isn't, and Google has a head start on its competitors -- but I do think that Google was more effective 10 years ago than it is now. I cannot however say whether that's down to Google overcomplicating their search algorithms or whether there's just x% more garbage out there, where x is some very large number.
Then it's definitely not what I want.
Not defending them and also dislike this (but can’t think of a way involving payments which doesn’t do it). I do find their privacy policy quite clear and transparent - https://kagi.com/privacy
And also let’s not kid ourselves - I would guess this is still likely more privacy-forward than a Private Session on Google. Would be surprised if Google isn’t using every potential signal to fingeprint you and link back that session back to some other login.
Good luck!
Hard disagree. I spent years of my life working on a "new way to search" (Graph Search at Facebook, ~2012) and the biggest takeaways for me from all of that time is:
1) people don't want to learn a new way to search, and they don't want to learn new tools, filters, lenses, whatever you want to call it. it must be a keyword-search based input because Google has trained billions to search (and to think) this way for 20+ years now. imo the next google shouldn't fight this behavior, but rather adopt it as part of the core strategy
2) ownership of the "search intent entrypoint" is key. people generally don't care what they are using to search, as long as it's easy and fast to use and actually works. google understands this and makes it almost impossible to bypass searching with google if you're on an Android phone. All of these alternative search engines have a huge barrier to entry not only on the technology and data side, but mainly on the entrypoint side. They can't reasonably expect hundreds of millions of the people to go through and manually update their default search engine on all of their devices. What about when I go to use Google Assistant? Google maps? All entrypoints that Google has used to lock up search intent. It's brilliant. I believe there are moves still available for a future Google assailant to overcome these barriers, but I haven't seen many that have directly touched on this point yet. new technical shifts may open up new entrypoints here (VR/AR?)
3) the search engine that has the lowest time between [query] -> [answer] for a user wins. any search engine that adds additional cruft through forcing more clicks, usage of advanced tooling (filters, lenses, etc), visualizations, etc will lose out because they are getting farther and farther from the core problem a user has - "find me x". this is the thing that is driving the hero/knowledge units on Google, further cementing their leadership
4) the one brightspot - verticalized search intent is a huge area of untapped potential in the search arena. Google is great at general search (and honestly also at a huge number of verticals that it's developed over time), but there are many verticals that google honestly sucks at. I do believe there is a potential entrypoint here for a "new google" if they can take on a vertical-by-vertical expansion strategy. Owning the intent entrypoint would still be as important as ever, but this play would look a lot like the early Yelp vertical expansion strategy from the early 2000s. It's anybody's guess what the actual right starting vertical and entrypoint match is though
tl;dr Google won't go down easy, and a Google killer may actually look incredibly similar to Google in it's end-state
So I love you.com - it has an app for everything!