Hacker News Reader

Comment by floro | Hacker News Reader

This is just the AI grift and uneducated hypebros. There is zero reason to use an LLM as a classifier. But AIbros with zero technical knowledge will push for it anyways even if it is 1000x slower and 1000x more expensive to run.

View on HN

You can can't build an SVM-classifier for a ticketing system without domain data. You can ask an LLM to do it, and it will do it very well.

With the LLM you don't even need a data scientist to train the model, you just need someone who can connect to APIs. And the LLM is fine with adding and removing categories. No retraining.

It makes the initial faces incredibly fast. Then it is slow and expensive to run, so once the system settles, it becomes the target for model optimization, e.g, a dedicated, or semi-dedicated model.

I mean just have a coding agent do the whole process from start to finish. Tell it you want to seed an SVM from LLM runs on ticket data. I can bet that it'll do it all for you without much input from you at all.

I think AI is making people forget that code can be used for automation.

I see coworkers using AI for tasks like "visit an API endpoint, download a file, put it in S3". This is so stupid. Okay if you want to use AI, but use it to write python code to do that, why pay in tokens every time you want to run that task, it makes no sense

How do they use it like that? Like a once off action by the agent or is it embedded in a process?

No, they literally just ask. "Can you pull this file using this API endpoint and put it in S3"

Probably under the hood Claude creates some temporary python code or something.

But it bothers me, I don't know why. It's like paying a subscription service for the most stupid tasks

If you don't know what "python code" is, you'll just ask the AI to do the task.

But there are teams in your org that could write something in 10 minutes

Yes, after the biz unit spends a month talking with IT about it, it might take the developer 10 minutes.

I see this a lot among non-tech folks, who have a pretty thin understanding of the differences between "code" and "an agent". But do you see this among tech folks as well?

Your S3 case tho I think is a bit misplaced. You're right that if all your keys are configured correctly (API key, S3 key, etc) that a script to do that is like 5 lines in Python (once you get your venv going or use uv to setup the project lol), but tell an agent to just do the thing, and it will make sure that your API keys are in place and that you have access to the S3 bucket you're trying to write to and that the whole thing is in a venv, and then just report back that it's done.

Is it wasteful? Sure. For years I used to feel that way when I saw someone using a messaging app to message a friend across the hall in a loud room or when I saw someone taking a phone picture of their computer monitor and sending it. In the former case your packets are literally going from your phone to a central server and then onto someone else's phone in the same room. But, well, convenience wins.

Yeah but here's the thing: Ease of setting up and how low that bar is, this is not the most important factor for most companies. Also, yes, you need data to do this with an LLM. In fact the LLM was trained on that data for you beforehand. This is not an argument because you can also buy a pretrained classifier.

You empathize the set up cost but ignore that the model is in the magnitude of a million x in size and compute cost of a classifier model. That's like comparing a CPU with an IC

I can spin up a classification solution with an LLM in ca. 5 minutes that is likely to do an okay job. To build a custom classifier, I need lots of training data first.

That is the reason.

Once you then have a system that is used enough, it starts making sense to look to replace that with a leaner classifier, but LLMs have a place in vast number of places where we'd previously never use a classifier because it was too much effort.

> but LLMs have a place in vast number of places where we'd previously never use a classifier because it was too much effort.

Absolutely! But that doesn't rule out that there are AIbros that will tell you to use an LLM, even if the task could be done with a static analyzer with a few rules. Use an LLM because it's AI!

You seem to be quite fixated on a personality type rather than the underlying technology.

Not really, although when a certain personality type dominates a scene, people tend to conflate the two. Especially when they are louder than the rest.

All of the replies talk about how easy and fast it is to set up. That's literally the ONLY reason you would do it with LLMs. If you consider any other factor, it makes zero sense.

There are absolutely good reasons to use smarter things for classification. Svm with tdf idf for example is super fast but will make very bad classifications at times in ways entirely unfixable with that system (thinking back to bats being an acronym for a theology degree classification or similar)

This is a bit too radical and misses some nuance. In order to start building classifiers you at least need some ML domain knowledge, like what to google at least. Meanwhile, point anyone at the jev/open-jev-clones API and have them start classifying in <10 minutes. There's some value there. (same used to be valid for LLMs before jev became a thing. You can use llms as zero-shot classifiers with the right API wrapper, w/ no domain data).

Even if you have ml knowledge, it's still nice to be able to experiment with various use cases with zero upfront data/training time.

1000x more expensive run can be overall cheaper. You avoid upfront training/tuning cost. It all depends on your scale.

Reply on news.ycombinator.com