Hacker News Reader

Comment by Havoc | Hacker News Reader

I have a searxng instances but stopped using it after doing a comparison vs actual search provider APIs.

searxng gets rate limited fast and the results are inconsistent depending on what providers it can reach at any given time / isn't captcha'd out.

Meanwhile there are multiple providers with generous free tiers so no point in feeding LLMs inferior data

Also did a couple of side by side tests asking LLMs directly which feed results it prefer for some sample technical searches. Exa and tavily #1, brave #2, searx last

View on HN

> searxng gets rate limited fast and the results are inconsistent depending on what providers it can reach at any given time / isn't captcha'd out.

Exactly. I really wonder what the people going on about using SearXNG are actually doing, because in practice I've seen just what you see. Even from a residential IP, you might get a handful of Google searches before getting backed off. If you ever have multiple searches going on at once, even Bing and Brave are going to back you off pretty quick. My home IP is apparently permanently banned from using Brave Search through SearXNG.

And when you aren't getting results from Google/Bing/Brave, it gets bad really quick for your agent. As much as I wanted to go fully self-hosted for agent search, it just doesn't work well.

Yeah, I had the same issues. I just ended up paying for Brave and Exa. It's literally like, at my volumes, a couple bucks here and there. Nothing to really think about at all. I am a heavy user of AI, but it self-hosted AI, but still the volumes have nothing to worry about. But I'm not trying to scrape Web or anything like that.

I have been using SearXNG as my main search engine for a while (not for AI stuff), and it's been great up until 2-3 months ago when it started being rate limite/captcha'd frequently.

My main reason for liking it is that it just gives me results without any ads or special summary/snippet/QA/etc... boxes.

I really wouldn't mind paying API pricing for search, I think it would end up at less than $10/year, but I haven't seen a good self-hosted search interface that allows plugging in api keys.

I ran into the rate limiting very fast. Then i realized i had a SerpAPI key. I checked whether SearXNG supports it and it… doesnt, which is bizarre.

Qwant as search engine is pretty good.

Sadly, they're really difficult to integrate with SearXNG due to the aggressive captchas.

Reply on news.ycombinator.com