HNHacker News
TopNewBestAskShowJobs

hubraumhugo

9,168 karma · joined June 23, 2019

Building web datasets for humans & agents @ kadoa.com

Also made HN Wrapped (hn-wrapped.kadoa.com)

Happy to chat about data, AI, startups, or anything else: adrian@kadoa.com

submissionscomments
hubraumhugo··on LLM-hacker-news: LLM plugin for pulling content from Hacker News
I recently built this because I spent too much time here and had FOMO about relevant topics :) https://www.kadoa.com/hacksnack

Let me know what I should improve.

hubraumhugo··on Trump's Trade War Escalates as China Retaliates with 34% Tariffs
Pharma products were previously tariff-free under a WTO agreement that is based on the principle that life-saving items shouldn't face any tariffs. Looks like this exemption is now also being eliminated and will probably cause the prices of drugs and treatments in the USA to skyrocket.
hubraumhugo··on Ask HN: Who is hiring? (April 2025)
Kadoa | Frontend/UX | Remote | Full-Time | https://kadoa.com

Kadoa is an AI platform that automates unstructured data ETL.

80% of data is unstructured and traditional methods such as web scraping, PDF parsing, and manual data preparation require expensive engineering resources and constant maintenance.

We’re solving this at scale, but we can't do it alone. We are looking for talented people who share our passion for crafting exceptional software products.

There is one official role, but we welcome conversations with anyone interested in joining the team.

- Frontend Developer/UX Designer: https://www.kadoa.com/careers/frontend-dev-ux-designer

If you want to iterate to greatness and work at the intersection of data and AI, please email us at (adrian at kadoa dot com) and mention HN in the subject line.

hubraumhugo··on I tried making artificial sunlight at home
That's the most HN headline I've read in a while
hubraumhugo··on I fear for the unauthenticated web
We should try separating good bots from bad bots:

Good bots: search engine crawlers that help users find relevant information. These bots have been around since the early days of the internet and generally follow established best practices like robots.txt and rate limits. AI agents like OpenAI's Operator or Anthopic's Computer Use probably also fit into that bucket as they are offering useful automation without negative side effects.

Bad bots: bots that have a negative affect website owners by causing higher costs, spam, or downtime (automated account creation, ad fraud, or DDoS). AI crawlers fit into that bucket as they disregard robots.txt and spoof user agent. They are creating a lot of headaches for developers responsible for maintaining heavily crawled sites. AI companies don't seem to care about any crawling best practices that the industry has developed over the past two decades.

So the actual question is how good bots and humans can coexist on the web while we protect websites against abusive AI crawlers. It currently feels like an arms race without a winner.

hubraumhugo··on Mistral OCR
It will be interesting to see how all the companies in the document processing space adapt as OCR becomes a commodity.

The best products will be defined by everything "non-AI", like UX, performance and reliability at scale, and human-in-the loop feedback for domain experts.

hubraumhugo··on Claude 3.7 Sonnet and Claude Code
You can get your HN profile analyzed by it and it's pretty funny :)

https://hn-wrapped.kadoa.com/

I'm using this to test the humor of new models.

hubraumhugo··on Benchmarking Vision-Language Models on OCR in Dynamic Video Environments
As someone building in this space, we've found that raw OCR accuracy is just one piece (and it's becomming a commodity).

The real challenge is building reliable and accurate ETL pipelines (document ingestion from web, OCR, classification, validation, etc.) that work at scale in production.

The best products will be defined by everything "non-AI", like UX, performance, and human-in-the loop feedback loop for non-techies.

Avoiding over-reliance on specific models also helps. With good internal eval data and benchmarks, you can easily switch or fine-tune models.

hubraumhugo··on Hacker News for Gamedev
I've seen many "HN for X" projects for various niches now, and they all suffer from the chicken/egg problem of getting a critical mass of participants.

Isn't game development already discussed here?

I actually built a side project that categorizes front page articles so I can filter for topics. Here's an example for recent gamedev content: https://www.kadoa.com/hacksnack/d57360e8-1eb1-4800-a711-f0d5...

hubraumhugo··on Show HN: DeepSeek Your HN Profile
Should work again.
hubraumhugo··on Show HN: DeepSeek Your HN Profile
It takes the last 75 posts and comments for performance reasons. Going to improve this.
hubraumhugo··on Show HN: DeepSeek Your HN Profile
fixing, look like we're hitting token limits :(
hubraumhugo··on I am (not) a failure: Lessons learned from six failed startup attempts
To everybody reading this and having doubts about starting your own venture: DO IT!

Yes it's a lot of risk, yes most startups fail, and yes you could make more money at FANG. However, this is one of the most exciting times to build a startup, thanks to AI.

The big winners of the all the AI advancements (especially OS) are devs and AI startups:

- No more vendor lock-in

- Instead of just wrapping proprietary API endpoints, developers can now integrate AI deeply into their products in a very cost-effective and performant way

- Price race to the bottom with near-instant LLM responses at very low prices are on the horizon

It's as if your product automatically becomes better, cheaper, and more scalable with every major AI advancement. This leads to a powerful flywheel effect: AI improves product -> better product attracts more users -> users generate more data

Besides the product perspective, AI also allows you to operate your startup a lot more efficiently.

Will we see a single-person unicorn soon? Very likely not.

Will we see very successful small and bootstrapped startups? Yes, already the case.

hubraumhugo··on Nepenthes is a tarpit to catch AI web crawlers
The arms race between AI bots and bot-protection is only going to get worse, leading to increasing infra costs while negatively impacting the UX and performance (captchas, rate limiting, etc.).

What's a reasonable way forward to deal with more bots than humans on the internet?

hubraumhugo··on Aaron Swartz's Weblog
The /archive page is a great overview of all his posts: http://www.aaronsw.com/weblog/archive
hubraumhugo··on Boeing engineer shares what 30 years at the beleaguered plane company was like
https://archive.is/6xOMu#selection-1727.0-1727.86
hubraumhugo··on Show HN: I created a directory of the most durable products in the world
I'm having a bit of a deja-vu with this since I tried to build the same a few years ago, see my Show HN from back then: https://news.ycombinator.com/item?id=24707407

Some challenges I faced:

- Warranties have many nuances depending on the country and the fine print

- Monetizing this is hard. I see your using affiliate links, but this will hurt the trust of your users.

- Only very few users are willing to write valuable in-depth reviews. I tried to collect data on how products perform over their lifetime (https://www.looria.com/reviews), but never reached a critical mass.

- Astroturfing is a major issue. I aggregated many reviews from Reddit, but even there it's getting worse.

Happy to chat or help out with data if you're interested :)

hubraumhugo··on Wildcard: Customize HN using a spreadsheet view
I love the idea of having a plug-in spreadsheet on every website, so I can work with the data without having to do any scraping or programming.

Unfortunately, it looks like the project was abandoned after the initial talk and paper in 2020.

I assume that we'll get there soon with small LLMs running directly in the browser (like Gemini nano) that will make it easy to query/structure website data.

hubraumhugo··on Show HN: HN Wrapped 2024
It's pulling the 30 most recent comments + submissions. Any suggestions for a more representative way?
hubraumhugo··on Ask HN: Who is hiring? (January 2025)
Kadoa | Senior Software Engineer | Remote (Europe) | Full-Time | https://kadoa.com

We are building the foundational infrastructure for unstructured data ETL.

Our platform enables any organization to instantly extract, transform, and integrate unstructured data (websites, documents, emails, etc.) - without code or maintenance overhead.

You’ll work at the intersection of AI, data engineering, and software development, shipping code with real-world impact at industry-leading customers.

There are two official roles, but we welcome conversations with anyone interested in joining the team.

- Senior Software Engineer: https://www.kadoa.com/careers/senior-software-engineer

- DevOps Engineer: https://www.kadoa.com/careers/devops-engineer

Please email us at (adrian at kadoa dot com) and mention HN in the subject line.

hubraumhugo··on Fogus: Things and Stuff of 2024
Any good alternatives for "adopt: Blank Spaces app" on Android?
hubraumhugo··on Moon
Ciechanowski is likely the best content producer of our time, absolutely fascinating reads. Imagine having such a person as a teacher - he could probably excite students about any scientific topic.

I'd love to spend my time working on such articles when I'm retired :)

hubraumhugo··on Llama-3.3-70B-Instruct
Meta continues to overdeliver. Their goal from the start was to target and disrupt OpenAI/Anthropic with a scorched earth approach by releasing powerful open models.

The big winners: we developers.

hubraumhugo··on Kagi Teams
I can't think of any other product that makes it to the HN frontpage so consistently. I guess Kagi really hits the bullseye in terms of HN as a target audience.
hubraumhugo··on Understanding the BM25 full text search algorithm
Given the recent advances in vector-based semantic search, what's the SOTA search stack that people are using for hybrid keyword + semantic search these days?
hubraumhugo··on Launch HN: Midship (YC S24) – Turn PDFs, docs, and images into usable data
Congrats on the launch! A quick search in the YC startup directory brought up 5-10 companies doing pretty much the same thing:

- https://www.ycombinator.com/companies/tableflow

- https://www.ycombinator.com/companies/reducto

- https://www.ycombinator.com/companies/mindee

- https://www.ycombinator.com/companies/omniai

- https://www.ycombinator.com/companies/trellis

At the same time, accurate document extraction is becoming a commodity with powerful VLMs. Are you planning to focus on a specific industry, or how do you plan to differentiate?

hubraumhugo··on Show HN: Aide, an open-source AI native IDE
I'm curious - what does the AI coding setup of the HN community look like, and how has your experience been so far?

I want to get some broader feedback before completely switching my workflow to Aide or Cursor.

hubraumhugo··on Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
I've seen quite a few YC startups working on AI-powered RPA, and now it looks like a foundational model player is directly competing in their space. It will be interesting to see whether Anthropic will double down on this or leave it to third-party developers to build commercial applications around it.
hubraumhugo··on Swarm, a new agent framework by OpenAI
Has anyone seen AI agents working in production at scale? It doesn't matter if you're using Swarm, langchain, or any other orchestration framework if the underlying issue is that AI agents too slow, too expensive, and too unreliable. I wrote about AI agent hype vs. reality[0] a while ago, and I don't think it has changed yet.

[0] https://www.kadoa.com/blog/ai-agents-hype-vs-reality

hubraumhugo··on Web scraping with GPT-4o: powerful but expensive
We've been working on AI-automated web scraping at Kadoa[0] and our early experiments were similar to the those in the article. We started when only the expensive and slow GPT-3 was available, which pushed us to develop a cost-effective solution at scale.

Here is what we ended up with:

- Extraction: We use codegen to generate CSS selectors or XPath extraction code. This is more efficient than using LLMs for every data extraction. Using an LLM for every data extraction, would be expensive and slow, but using LLMs to generate the scraper code and subsequently adapt it to website modifications is highly efficient.

- Cleansing & transformation: We use small fine-tuned LLMs to clean and map data into the desired format.

- Validation: Unstructured data is a pain to validate. Among traditional data validation methods like reverse search, we use LLM-as-a-judge to evaluate the data quality.

We quickly realized that doing this for a few data sources with low complexity is one thing, doing it for thousands of websites in a reliable, scalable, and cost-efficient way is a whole different beast.

Combining traditional ETL engineering methods with small, well-evaluated LLM steps was the way to go for us

[0] https://kadoa.com

← PreviousPage 3 of 14Next →