Show HN: LazyWeb – Conversational search engine that's ad-free and private | Hacker News Reader

Show HN: LazyWeb – Conversational search engine that's ad-free and private

lazyweb.ai

24 pointsjedwhite14 comments

Hi HN,

We're Jed and Angie, and we're building LazyWeb (https://lazyweb.ai), a search engine with a simple chat interface that's free from ad-tech and tracking. It's like messaging a smart friend that answers questions and sends you useful links.

I'm a reluctant expert in how much damage ad-tech has done to media. I'm a programmer, but I also trained and worked as a journalist and editor, and built a SaaS startup serving media companies. So I've watched first-hand how the media industry has not-so-slowly starved, including seeing friends and customers lose their jobs and businesses. In its place, ad-tech, SEO spam and clickbait have taken over the Internet and dominate search. HN has posts every day about Google search getting worse, and ad-tech more intrusive.

We're not under any illusions how hard this problem is to fix. But we think we can make a dent in it by building a search application based on ethics and trust, and focused religiously on users, not advertisers. Google is an ad-tech company, not a search company. More people work on ads than search there. We want to deliver results without commercial influence, filter bubbles or censorship.

We're using APIs rather than trying to duplicate Google. We use NLP models based on transfer learning (GPT-2 for conversation models + some Amazon Lex for routing) to detect utterance intent and extract entities from queries (eg programming, knowledge graph or restaurant searches). We use a combination of routing logic and deep learning models (trained on public search query data) to predict the best sources for that intent / entity type / topic. Then we query the APIs for those directly (ideally for a direct match), or spider their search-result pages. We have a database of around 15K sites.

We rank the results for topic match, content quality etc using models trained on example content. We use a BERT-style approach to extract key information or a direct answer within the top-ranked results. Examples: Computation queries => Wolfram Alpha or direct calculation. Knowledge graph => Wikipedia or DuckDuckGo Instant Answers. Programming => github, stackoverflow etc. Web searches from Bing or other web search APIs are 50%+ of results. We are building some specialized indexes (eg direct navigation). We are "broad but shallow" to start. We're in the OpenAI GPT-3 beta and excited to experiment more.

We will make money three ways. 1. If a user buys something after searching, we may make a small commission from anonymous referral links. We will share revenue 50/50 with content producers used in the search. 2. We will have some paid Pro and Business plans in future. 3. We will provide a chat plugin that businesses can offer for search and navigation on public or internal sites.

Try it out and see how the results work for you!

Some fun features to try include recent events (try "is elon musk taking dogecoin to the moon"), direct navigation (try "go hn bitcoin" or "go youtube aws lambda"), or question answering ("how many baseballs fit in the moon"). You can also change how results display. Try visual results with feed or card grid views. If (like me) you love HN's minimalism, or miss Google circa 1999, try the "Hacker" or "Goggles" view on desktop (top right). Also try Reader Mode for a proxied ad-free experience.

LazyWeb is in early alpha testing. It's free and open for anyone to use, and improving fast. We'd love your feedback, suggestions, bugs, improvements and anything else, and we're here to answer any questions you have!

How do you train newer models then? From what I read you use public datasets to train your models but what about in future? You would need some kind of data collection mechanism?

Gpt-2 and gpt-3 are great but the datasets they are trained would soon get old.

Hey thanks. We will not log searches or collect personal data.

There are public sources of search data that we can use with transfer learning against large scale language models like GPT-3, and that are updated regularly. Transfer learning works well without needing massive data sets with this sort of data (phrases mapped to intents).

Having said that, the app tracks the intents and topic profiles of searches (not the search itself, just for example FoodPlaceSearchIntent) and whether the execution was likely a good result or not based on signals (like whether the search was likely repeated or rephrased - again without recording the actual search), and the models learn from that. We're adding signals including anonymized upvote/downvote as well.

Approaches like differential privacy are something we want to pursue more in future. We are still very early days!

Makes sense though I wonder what would be the original source of that data (someone like google/microsoft must be logging user data and then making some parts of it anonymized and public).

Maybe also look into on-device learning, it can be efficiently hooked up with differential privacy and give more specific results.

Yes, ironically, SEO industry resources can be helpful, and we used them in putting together training data. If you're interested there are some good simple free ones to get started also, like these from Mondovo:

https://www.mondovo.com/keywords/

Brave browser uses aggregated search history data that's been anonymized, but we're not trying to personalize results (we're looking for objectively true, rather than "true for you"), so we're not trying to replicate ad-industry style personalization. A good set of labelled data matching intents to phrases helped us build some models simply that are surprisngly good at picking intents :)

Today I was watching a documentary in CNBC - how 70+ local news outlets/ community news outlets shut down in 2020 itself. Reason being the source of revenue for these companies are chucked up by big3. It is a noble cause Good Luck Angela and Jed.

As media companies lose revenue (both circulation and ads) they get more desperate, and so they start doing things that damage their own trust and relationship with readers. The invasive and polluting ad-tech all over newspaper websites is a clear example of that. A lot of news websites are almost unusable with pop-up, pop-under, overlay and dark pattern ads, or video rolls that start without warning.

The end result is that they drive readers away faster than ever chasing after fewer and fewer crumbs.

[EDIT for typo - and thanks for well wishes too! :) ]

Seems similar to what we built -- https://insideropinion.com/

Happy to chat offline, curious how you implemented it if you have any explanation.

Cool, so it's like a HR insights tool? What sort of sources are you using?

We run mostly on AWS - essentially Lambda/API Gateway/SageMaker etc, with a bunch of supporting proxies and other services on AWS EKS K8S that we migrated from Heroku using Porter (https://www.getporter.dev/) which is a YC Alum.

We have some supporting services on MS Azure. And consume a large number of APIs.

Did you write your own chatbot infrastructure and is it live to play with somewhere?

All the best!

When I search something using lazyweb, I'd get results to the point. has been useful to me, Congrats to Lazyweb.ai team.

Thanks! We're aiming to reduce cognitive overload by providing direct answers where there is a result with high enough confidence on the inference. But then the full results are available as well.

Originally on desktop we weren't going to show the full results by default, or maybe only if the main result was below a certain confidence threshold.

But people are so used to getting overwhelmed that going back to a single result displayed introduces cognitive dissonance :)

Exactly!, no matter how good a product or service is, at the end the social behavior is the driver of success, and has to be taking into account 90%

Dear LazyWeb

Yes! That was the spark for the name. It's who you call out to when you know Google will be too muck work or hard to find the answer.

Down the track, the @lazywebai twitter account will have a bot that will answer "Dear LazyWeb" questions or when people tweet to it. That should be a really interesting NLP experiment :)

Reply on news.ycombinator.com