Website search hurts my feelings
openmymind.net
openmymind.net
About 15 years ago or so, together with a friend, we started a company whose goal was to offer a working product search for E-commerce.
Faceting was just one aspect of the problem, a much more pressing issue for non-English speaking countries is that you need a way to deal with declension and various misspellings (especially from people that do not enter accented characters). So we did. Solve the problems, that is. With a pretty good SaaS offering, reasonably easy to integrate into any E-commerce site.
And then we discovered that you can have a great product, but selling it is an entirely different problem, and we were not able to solve that problem. It turned out we were competing with marketing budgets: a customer (E-commerce site) could either pay us and improve the customer experience significantly, or they could spend this money on customer acquisition and improve the bottom line right away. Guess which approach won.
Another realization was that customer experience might actually not matter much. People are used to crappy websites and will jump through all sorts of hoops if the price is low. So many of our potential customers preferred to pay for customer leads (price comparison sites) and offer low prices, rather than improve customer experience, because customer loyalty isn't necessarily a thing these days.
Also, if your main acquisition channel is price comparison sites (which is the case in many countries), you don't care about search: customers will land directly on product pages (where the price is right) and will endure the crappy search to add some stuff to their carts.
We ended up with a bunch of happy customers who stayed with the product for years, but we stopped trying to sell it and moved to other things, it was just too hard to sell.
In other words, addressing the OP's (excellent) article: your hurt feelings don't matter much, and nobody really cares, as long as the customer acquisition funnel works and there is a steady increase in the bottom line numbers.
While I've never been as deep in the details, my perpetual answer here is:
"You're so vain, I bet you think this search is about you?"
I understand it because people are human, but I can't help but feel annoyance when people are like "the search engine isn't doing what I want!"
It's not designed to DO THAT, it's designed to maximize it's own sales and we're pretty much at the point where we should try to stop pretending otherwise.
Users should expect better. People with your attitude are not helping.
I guess I'm baffled as to how search has stayed so garbage for so long and why this hasn't been more disrupted yet. We know (or should know) the problem and we have things that look a lot like the solution? Wikipedia has lessons, always adding "reddit" to your searches has lessons, now ChatGPT has lessons.
I guess, to me, this type of complaining is just kind of, yeah duh. Now what?
Actually…
Consider this: the goal for the E-commerce site is to make more money. If showing unrelated products in search results brings in more revenue than showing the products the customer actually searched for — then that's what will be done.
This is not theoretical, I actually know of this being done on purpose.
As long as the market share is that high, there's no reason to improve anything.
Gliding past that with "Google [doesn't] care [about] search quality" doesn't add much.
Amazon avoided that for a long time, but not anymore (imho). These days there's lots Amazon doesn't have, and their search seems designed to get you buying faked results. And this is the case for all commercial searches, but still less so for Google.
You can acquire customers with lower prices, but can you keep them with a bad UX?
It’s a feature but not the only feature. Ppl complain about Google search (arguably their only feature) on HN everyday, but they’re still essentially a monopoly.
Users are deciding that chatGPT offers a better search UX in some cases and abandoning Google.
So the answer is "No, unless you are a monopoly, and even then don't push your luck."
Probably because so many companies have burned that loyalty for short term gains. It isn't worth being loyal to a company if at any time they can decide to throw those loyal customers under a bus to make a quick buck.
If you want to get started fixing website search relevance I recommend the books “Relevant Search” by Turnbull and Berryman, and “AI Powered Search” by Grainger, Turnbull, and yours truly. Both published by Manning.
Website search is... hard. A lot of the faceting still needs to be done by hand. I think there's probably some opportunity for LLMs to make some sort of autotagging/categorization easier, but there will likely still need to be a human in the loop to verify.
Search is hard because you need to anticipate and model the language of all potential searchers and the content.
There’s also lots of ambiguity because you’ll only get one or two keywords from the searcher without any other context, and you need to take into account trends and content quality and metadata.
Also, in lots of cases search is bad because the product team either doesn’t know or doesn’t care.
That's the definition of a table stakes feature. It's not as if you can not implement it and people won't notice!
I had waited so long for rating filters, my kids became adults.
However, I was wondering what Netflix got after 17 years of development at $215k-$700k per dev - and now I know.
That might make sense in an abstract way, but no, that's not what people expect. If I'm searching for something at Netflix, I don't want the answer to "does it exist?", I want the answer to "do you have available for viewing?".
In the more general sense of the search topic, I don't want a search to return things they don't have. If it's out of stock, say so prominently. Don't make the user click through to the item itself only to see "out of stock".
> I've been impressed by Netflix search. I would search a name and they do not have that movie, but would suggest a mix of similar movies, movies with the same actors or director.
An excellent, early pirate feature. Good on Netflix for adopting it.
My main issue with the input search is the TV keyboard. There was a Netflix hack day that created a circular keyboard like a dial w/ the selector a joystick in the middle, so pressing up to 12 o'clock was A, 12:05 was B, etc. And the joystick would return to center per selection! It was everything I ever wanted in a thing, and now I can't even find the article on it.
On the other end of the spectrum, try Digikey:
https://www.digikey.com/en/products/filter/embedded/microcon...
or try McMaster Carr:
https://www.mcmaster.com/products/screws/socket-head-screws~...
Functional, well-tagged, data-driven parametric search. Rockauto.com sells auto parts, which requires a whole different type of search, but they deliver a pretty impressive product as well.
I suspect that much of this discrepancy is because there's enough of a chance that someone grocery shopping and looking for lentils will think "Oh, I need pancake mix too" and click on the ad. Meanwhile, someone looking for a specific grade of M8x25 socket-head cap screws or a low-power Cortex-M0 microcontroller with a particular set of peripherals is unlikely to abandon their work to add a hose clamp or electrolytic capacitor to their cart instead.
Market forces seem incapable of delivering a product with a better user experience if that hurts the bottom line even a little bit, they're doing gradient descent to local minima and powerless to say "no" to finance and climb out of the awful product that results.
You can't look at one narrow usage path to judge an interface. You can only customize the interface so much for specific use cases before it becomes confusing as hell to use because it lacks consistency between tasks.
The problem is obviously that the data isn't tagged. But Costco is huge, profitable and doesn't have a huge inventory, so why can't they tag it? And if the tags are so incomplete, maybe it's better not to include them?
Well, no, if I ask for rice, the odds are overwhelming that I'm looking for rice, in which case chicken and rice canned soup is a totally inappropriate response. It would make as much sense as returning alcohol-free wine when asked for "alcohol".
You should be able to unearth the soup with a partial match. What makes the alcohol/alcohol-free problem is that alcohol free is one token that can't be subdivided.
Moving the argument into the realm of things that matter, chicken and rice soup is not a product that can be called "rice", and therefore isn't an appropriate response to a search for "rice". The title is irrelevant; if you're stocking rice with a fancy name and its title doesn't include the word "rice", that product still should show up in a search for "rice".
Yes: chicken and rice, rice cakes, rice paper, cauliflower rice, rice cooker, rice krispies, rice university jersey, mochi, risotto.
No: licorice, ricetta tiramisu.
As I said, that is inconsistent with your simultaneous position that alcohol-free wine is not a match for "alcohol". Either they both are matches to their respective queries or they both aren't; there isn't a difference between the two examples. This is true from the perspective of dumb text searches, but it's equally true from the perspective of correct linguistic parsing.
That rice is part of the listed title for chicken and rice soup is a coincidence; the argument for returning it also tells you that you should return the same soup when asked for "tomatoes", if tomatoes are an ingredient in the soup.
Your "yes" and "no" examples don't seem to work. What rule excludes "licorice" that would include anything containing the raw textual string "rice" without regard to whether any rice is involved? Rice paper, rice cooker, cauliflower rice are all purely textual matches. Mochi is a purely semantic match!
Oh! Chicken and rice soup, I could go for that. Cart.
It's not even a problem... it's an unfounded assumption that alcohol-free wine wouldn't ever be relevant to the user at all. If the only place open on a holiday was Walmart that I didn't realize didn't sell booze, and I needed some kind of booze to make a pan sauce or a braise, I wouldn't necessarily think to search for non-alcoholic booze-- but it would be a very useful substitution for me, and not by accident... they're two versions of the same product. I'd much rather see that they had non-alcoholic beer and wine in my search than completely empty search results.
This is why competent companies hire designers to work on interactions like this rather than leaving it up to the implementing developer's gut instinct about user needs. If only the tagging could keep up.
However it shows restaurants that paid to be featured at the top of the results.
So the search results will often contain restaurants that are multiple hours away.
I also noticed the same in several other website where "relevance" trumps my sort request (e.g., price, distance).
Search along route sometimes works better and sometimes also zooms out.
Especially to such an extent as the umbrella example.
What's the failure mode here? Microservices that "own their own data"?
Fuzzy matching and boosting often trip people up or lead to folks shooting themselves in the foot relevancy-wise.
If you want really great results, you need to spend the time crafting your query, dealing with synonyms ("pop" vs. "soda"), stemming, typos, negative boosts ("non-alcoholic", in the example) etc.
It's not necessarily hard, just often forgotten or not included in the initial scope.
Someone complained that for public instances, Google worked better (this was back before Google started to suck).
My response was that if I could write a better search engine than Google in my spare time, I'd be living a lifestyle considerably more luxurious than the one I actually had. :-)
The site search picks up unrelated items, misses others, and so on. I would complain about the inventory management/substitutions too - but since they also have in store shoppers using the same inventory I guess they can’t always be 100% about what’s on the shelf.
You’d think a chain that size could do better, but I doubt they have much incentive. The competition is minimal and not any better.
At the end of the day, gathering and aggregating all of that crap into a stable API is a chore unto itself. Multiple people's jobs. Updating products, associating taxonomies, in-store locations, duplicate locations, optimizing pickup routes, etc etc etc. There is a lot of grunt work here that can't be entirely automated away when dealing such a dynamic business.
Now you need to build a website that takes all of this data and somehow displays it in a format that is comprehensible and accurate for consumers. Showing relevant recommended products, search keywords, cache and image management (for potentially resizing dynamic supplier photos). Then think about the cart and checkout, two areas and Shopify and Amazon have devoted literal billions of dollars to optimize. Your volume is big enough that using an off the shelf payment provider is off the table; not to mention that you don't have the margins anyway. Have you thought about mobile apps and the development effort required there as well?
Grocery stores are hard enough as is but doing them online is a totally different beast. There's a reason that all of the "Kroger" brands use the same exact website with a different logo in the corner. The only players in this space doing it reasonably well are huge conglomerates with money to burn on building the required infrastructure to build the audience. Actually, I just looked up Harris Teeter and they appear to be a Kroger brand as well. TIL. I would say that "Kroger" is one of the better ones. Whole Foods is also doing decently well if you can afford the price premium, but I realize for most people that isn't a luxury they can afford.
The best way to get them to improve is to keep using the service and demanding better (or use a competitor if available). These departments have KPIs. If you get the wrong product, file a complaint at the end of your order process and mark any NPS surveys appropriately. If fulfillment has a lot of bad data from elsewhere in the organization driving their performance scores down, then they are incentivized to get it fixed. I know it sucks, but the only other alternative is to not use it at all and risk is going away forever.
n -> suggests nothing
ne -> suggests news.ycombinator.com
new -> suggests news.ycombinator.com
news -> suggests nothing
news. -> suggests nothing
news.y -> suggests nothing
news.yc -> suggests nothing
news.yco -> suggests nothing
news.ycom -> suggests nothing
news.ycomb -> suggests nothing
news.ycombi -> suggests nothing
news.ycombin -> suggests nothing
news.ycombina -> suggests nothing
news.ycombinat -> suggests nothing
news.ycombinato -> suggests nothing
news.ycombinator -> suggests nothing
news.ycombinator. -> suggests nothing
news.ycombinator.c -> suggests nothing
news.ycombinator.co -> suggests news.ycombinator.com
news.ycombinator.com -> suggests news.ycombinator.com
new -> suggests news.ycombinator.com
news -> suggests nothing
You're right though, this is incredibly dumb design.
In practice people type what they think will turn up the results "news", say, and don't look at the results until they're finished typing.
Another annoying aspect to it is that things enter your history as you browse, so the number of characters to hit that point where it shows what you want might shift around. Which totally destroys muscle memory. This isn’t the browser’s fault of course, but it would be nice if they could design around it.
a, ad, adv -> first result is adventofcode.com/2023
adve -> first result is adventofcode.com/2023/day/1 [honestly, this one baffles me, but I guess I went back to day 1 at some point after typing `adve`]
adven, advent, advento -> first result is reddit.com/r/adventofcode
adventof -> back to adventofcode.com/2023
I actually *appreciate* this behavior; it means I can better muscle-memory a certain number of letters and go down to the first result when 2 sites have similar names (very annoying for twitter & twitch btw).
So I don't think the problem is that weights can change as you type more letters; but that an entry can disappear altogether when there are no other suggestions.
And if they included the full URL, sheesh… you could effortlessly query against search params, path segments, fragments, and origins all in one intuitive interface.
‘It’ suggests It, It’s, and Its.
‘Its’ suggests Its and Itself.
Given that the apostrophe requires accessing a different keyboard overlay, and that only two suggestions are presented (leaving one suggestion slot empty), why not just include the contraction “It’s” in the suggestions?
Like I’ll search the full word “passwords”, pausing after each letter, and not get the entry titled “passwords” in my results the whole time. Back out, do it again, this time it’s there by the time I type “pas”.
This used to work perfectly, but has been inconsistently broken for years now. Most recent time I saw it was a week or two ago, so it’s still happening. What’s most baffling to me is that a search over such a relatively static dataset (app settings are in there, but those don’t get added or modified often) can be non-deterministic.
But for example, if you start a sentence with “We’re,” it will often take out the apostrophe… until you reach the end of the sentence, and then when you add the period I guess it’ll analyze the sentence and figure out that you actually meant what you wrote, and put it back. It does highlight the corrections, which is really nice. At this point, I just ignore it until I’m done with the sentence and then go back and check.
Just try to get to display settings on windows these days through the start menu.
start -> run -> "ms-settings:display" -> enter
I lost hope it and other issues would be fixed and moved to Chromium on Android.
#1 a lot of teams are understaffed, overworked an not aligned, improving search is simply not a priority (at the same times, a lot of people are bored out of their mind, because of inefficient allocation of capacities). And of course, usable analytics data on search does not exist as it is hard to collect...
#2 search might be actually ok, but the quality of the catalog data is a nightmare - and the catalog is maintenance by a tiny team that overworked and underpaid. To add injury to insult, there is also no clear accountability established and 5 departments and a couple of vendors play the "not my problem" game. Sometimes 10-30% of the catalog is renewed every year and people gave up a long time ago to report problems.
I got used to good search tools in grad school, and then came Google. And ever since I've not gotten better than a result salad. Sometimes a tasty salad, but still a salad.
And, to be complete, firms like Nerac offer good search tools, but you need to work with someone who performs the search for you (last I used them).
The only other enterprise application I am aware of is applying the homework cheating app to customer facing roles that require email/copy generation.
No, with LLMs you're getting both, because a. LLMs need the human-curated data to ingest and b. after training the model is going to get RLHF.
Also, a square kilometer of office space still uses much less electricity than a square kilometer of GPUs.
My gf (who used to work in the sector) says a lot of people like to sit in front of the TV and “watch what’s on” — one of the reasons broadcast TV still exists.
You see this with sites like Netflix becoming decreasingly useful, urging you to simply click on something rather than consult your curated watch list. Some of them now hide your watch list!
She suggests the same applies to shopping: some people just want to push the buy button, which is well known in brick and mortar retail, especially low margin ones like grocery shops: end caps, mid shelves and all sorts of other strategies are used to get you to either buy on impulse or buy the version they’d rather sell of what you have in mind.
So for these sites the same applies: show a few options and the customer will simply pick one. Spam the tags, DWIM the customer’s ill-formed, uninformed, or simply unsatisfiable query, and see if the punter buys.
For nerds like us this is a pessimization. But perhaps for the majority this works better.
Mentally when I'm at a store and I have 3 options to pick from I seemingly enjoy the process better then if I had 6 or 10. Eventually the picking itself becomes work. If it's a topic I enjoy like motherboards or RAM then all the choices are fine, I can spend hours on that, but if I'm picking tomato sauce for a quick and sloppy dish I don't want to spend the time selecting.
Maybe if I'm on Netflix to watch what's on, that's fine. But e.g. on Amazon, I find "search" so adversarial that I often close the tab and go somewhere else, or give up.
The one and only reason I am considering office depot as a place to spend my money is because they're local and I can go get whatever cable or thing I need, right now.
A. [explicit claim; humorous] Analytics confirm that, if you show people whether products are in stock locally, they won't buy those products on your website.
B. [by implication] This is bad, because it "lowers" your sales. So you can't display this information.
C. [by implication] The retailers are making a mistake - the reason people don't buy products on the website when availability information is displayed is that they go to the store and buy the product there.
D. [summing up] If you measure something you don't care about, you will end up doing things that make you look like an idiot.
This isn't just hypothetical: for a while I was using QFC (Kroger) for grocery pickup, but after one too many instances of adding an item to my cart and then only finding out at checkout that a third of my items weren't actually available, I said to hell with it and switched to doing all that at Whole Foods instead, since their inventory counts are actually correct.
Try searching on Walmart to find local stock. You have to click "In-Store", then change the Fulfillment Speed to today and tomorrow to find things that are in stock. Except now they also show results from other stores, like Advance Auto Parts, and there's no way to filter that garbage out.
I tried searching Academy Sports for local stock. The list of items says it's in stock but when you go to the individual item it says it's OOS.
Lowe's and Home Depot local stock does seem to work reasonably well, for now.
Inventory will never be well-tagged. The process change to ensure it would slow down procurement.
Ideally, they'd expose an inventory XML list and we'd just use our LLM on it.
The hard part is how to incentivize by permitting them control over advertising. An alternative is the browser LLM just searching across the page (Arc Browser can do this).
And did anyone else notice the omission of that useless copyright line at the bottom of websites as if we know anything about copyright law?
> Of course the list of switch games isn't under Games > Nintendo Switch Games that would be silly.
Well.. There is a list of games when you open Games > Nintento Switch Games. What is he complaining about?
That the list doesn’t contain all Nintendo Switch games that the site offers and there doesn’t seem to be any clear reason why those particular 120 games were chosen.
This isn't just bad search, it's objectively wrong (and a lot more basic).
You can get better results with a 5minute tsvector in Postgres, At least it will show the words you typed in in the results!
And yes I understand it may be a different "scale" of products, but do people not test these things?
The only search tools that actually give me great value are history shell with fzf on the terminal, and grep (and similar like ag or ripgrep) in combination with a fuzzy search frontend (similar to piping grep to fzf).
I don't know where I'm going with this. I guess searching just sucks now.
If you want to get academic with it, you could say something about revealed preferences vs stated preferences (they asked for Coke but they really wanted Pepsi), but I really think they’re just throwing as many suggestions at the user as possible in the hopes they stay engaged on the site and eventually click buy on something.
Good website search is hard. Funnily in some respects it's harder than web search, since there are no popularity signals a la pagerank. (There could be such signals, but you have to work hard to create them.)
eBay too, it's OK (compared to Amazon) but most of the items on a search are always "mousepad [or whatever] suitable for" the laptop you searched for. Why eBay don't clean up their categories and ban sellers I don't know, it makes the site so much worse to use.
I wonder if anyone is _ever_ hoodwinked by the vast majority of sites that sort by Most Expensive by default?
It's all part of enshittification presumably, these sites monitor so many metrics it must surely be a profit generator to make search frustratingly bad.
There should be a movement for solutions that put power in the hands of the user.
And yes, the results are between amusing, annoying and infuriating.
The “alcohol” example finding mostly comes from alcohol free products advertising this feature in their name - no need to name your regular beer or wine “with alcohol”, at least not in the product name.
I'm wondering how well something like GPT-4 generates tags with a product description/product image. Be interesting to see its accuracy and comprehensiveness.
They don't do big box retailer stuff well (behind glass), they don't do local grocery well, and they don't even do discounted items well (5 below, Temu).
Their stores are garbage and they are asleep at the wheel on their website.
It is only a matter of time.
* PO: We need search!
* PM to architect: Google how to implement search!
* Architect: googles "best search library 2023 java" Aha! Let's use Elasticsearch!
* PM to Team: Alright, I need a search feature next sprint. Architect googled for 5 minutes and says we should use elasticsearch!
* Team: Alright, integrating elasticsearch in a sprint should work. Bob, you work on loading the data, Eve, you work on rendering the search results.
* Team (2-3 weeks later): Another successful value delivery! Alright, what's next PM?
* PM: PO says we need a chat bot...
If it's not obvious why this is a problem, getting search to work well requires a lot of tweaking and tuning. You can't just slap a search engine into a product and expect it to work well. It's hilariously incompatible with the assembly line style of work that many software outfits employ these days. It also requires deep understanding of what a search engine does and how its algorithms work.
It helps we have two internal departments whose primary jobs are data management and analysis. (Real statistics, not “AI”.)
- Typesense (https://github.com/typesense/typesense)
- Algolia
- Google Programmable Search Engine (https://programmablesearchengine.google.com/about/)
We have gotten used to almost flawless experiences from amazon shopping. Google search finds (or used to) results sometimes almost like magic, etc. The thing is, that's the core business. These companies invent new technologies and have huge teams because doing this is so hard.
Now take a random supermarket chain. Their knowledge is about physical stores. Their core business has taught them where to open a store and how to arrange things in it so that they maximize their sales in that environment. It's very hard and it takes a very long time to shift to an online model. You have to find people with the right competencies and the right leadership to convince the company to do this and this is actually very hard to do.
Look at the categories example. That, to me, screams backend database. The company has invested a lot of money into building business intelligence on top of their physical stores. That organization screams "perfectly curated data warehouse" and I imagine suggesting something like "we need to reorganize the way we store data" is going to be met with blank stares if not full on outrage.
It's pretty good, but there are some longtime flaws that blow my mind, such as the order of results when sorting by price. It never makes sense to me.
So, if someone is selling a used item in poor condition with maximum shipping, that product is going to sort as a low priced item.
At least, that's my reverse engineered understanding.
In my opinion its too fuzzy, which is usually the opposite problem I have with website searches (e.g. searching a news site for keywords of an article I know exits, but it's taking me too literally)
If I search for "iPhone 14 Pro case", I don't want to see cases for iPhone 15 __, or non-pro models. I've (to my own fault) bought way too many of the wrong product because I search for a specific model and don't read the title before ordering, only to realize that Amazon didn't give me exactly what I typed in.
what kind of stiff are you buying you call this "flawless"? In my experience the Amazon search is worst there is. Search for "AAA batteries" and it will offer you AA and even N ones. Why on earth would anyone want that?
They even got the basics wrong. The other day I was searching for power bank under $5. Instead, many listings was $10+. How hard is it to get this right?
Because of horrible search quality I actively avoid Amazon when I can.
But for a search engine, showing incremental value should be easy if you have a real product reason to do it. If you expect search to help people find content, then you look at statistics on engagement --- in aggregate did people use search, does it look like they found what they wanted, did they return and use search again. In specific, which queries seem popular and provide good results, which queries seem popular and don't provide good results. You provide incremental value by moving queries from the second to the first category, where possible. If you can kind of classify some of the less popular queries, there's value in improving results for those too, but classification is also hard.
Sometimes the numbers are easy... I participated in an application of search where a support request hit the FAQ before submission --- if search works, there should be fewer tickets and especially fewer tickets where the user can easily solve their own problem. Search in a shopping context should lead to more sales. Etc.
Quite often the company won't make any more in those long tail sales, but instead better placement of the products they buy in mass.
The amount of high level product people that can't demo their own product and cannot find features they hype up in slide decks is depressing.
The profit margins on selling products are abysmal for e-commerce. The profit margins on ads are amazing.
I'll be the first admit I don't have a ton of respect for the "product owner" role which has morphed largely into Tom Smykowski from Office Space[0] but the places I've seen them be actually effective and net positives for an organization are where they have the reach and authority to demand things like this be fixed, regardless of what department is "in charge." IMO a product owner who doesn't do this is refusing to do one of the few useful product owner roles.
[0] https://www.imdb.com/title/tt0151804/characters/nm0726223
If I use my product and hate it, I make it better. Knowing they some other department sucks for years is no good.
I think all this boils down to companies not really giving a flip.
The work of product person shouldn't stop at the idea phase.
We just told our customers to google the part number. I don’t know how these places stay in business.