Hacker News Reader

Comment by supportengineer | Hacker News Reader

There are a couple types of roles that sounds really interesting to me. One would be taking some proprietary data and training LLM in a format it could use. One company I know has a database of cars. They want to train their LLM with some inventory facts like "We have a Ford Mustang on the lot whose vin is ABC123 and it has the following features...". And then another role would be writing the prompts for API calls to the LLM. "Write a report on all cars currently in the lot which are available for sale and have the following features...."

View on HN

This doesn’t seem like a thing any business should need AI for.

Filtering a known list for a report takes a few seconds at most, and is much more reliable than LLMs.

> They want to train their LLM with some inventory facts like...

So are they actually intending to retrain their LLM every time the inventory changes? Because, otherwise, how is it going to "know" the current state of the inventory? This is useless after a single sale or a single new delivery without retraining. (And it's likely useless before that anyways.)

And if they already have a database of inventory data with all this then they could just generate a report the "old fashioned" way that's worked for decades.

I would expect the solution is to take the NL question and get GPT to transform it into a SQL (or similar) statement to extract the data. Then another call (or set of calls) to generate "reports" summarizing the data returned by the DB query.

At which point, maybe a "GUI-interface" will be cheaper to build and maintain in the long run.

Now if AI could automagically update inventory data with what's actually physically happening on the lot, that would be cool.

That's a very generous take. But that application would be far more useful than just car inventories (the limited application described) and not trained in the manner described (on inventory data). It would be trained on transforming natural language to SQL (or other) query languages, and the application of that is exactly what we're seeing with code generation applications of LLMs (to the extent they're presently useful).

Existing LLMs are already pretty good at this, no? The tricky part is mapping however the NL question refers to the various types of data to the actual column names, which is where I'd imagine some prompt engineering (or pretraining) would be necessary.

BTW I tried it with ChatGPT 3.5 - with a prompt that roughly described the database schema and a question "I need to know the manufacturer for the vehicle with VIN X7820-A and to confirm whether it has the feature 'rear camera' installed", it came back with

    SELECT TVehicles.Make, 
       CASE 
         WHEN TVehFeatures.FName = 'rear camera' THEN 'Installed'
         ELSE 'Not Installed'
       END AS RearCameraStatus
    FROM TVehicles
    JOIN TVehFeatures ON TVehicles.TV_ID = TVehFeatures.TV_ID
    WHERE TVehicles.VIN = 'X7820-A';

One interesting thing to note - I didn't tell it that "Make" and "Manufacturer" are the same thing.

I even went the next level and asked it to write me code to execute the query and generate appropriate HTML output from the results. It didn't quite manage it to handle any possible SQL query (remembering that the query itself has been dynamically generated), but wasn't far off. My description of how the output should look was simply "sleek and modern", and it came up with CSS that could be reasonably said to fill that brief.

What I have done for a similar use case is to have ChatGPT via the OpenAI API generate SQL (or KQL) based on the user request and then run that query and display the results (with some prose if appropriate). Works fairly well even with GPT-3.5-turbo. With GPT-4 can handle more complex requests (slower). It could even create a custom Chart.js chart on the fly if requested.

To me this demonstrates that there is a specific job here, even if you don't want to call it "engineering". Which I would argue is the correct category of job at least.

The above project was presented as "let's put a table of data in a vector database and then search it using the embedding of the user query". Here you were suggesting fine-tuning an LLM with the structured data. Again, it makes more sense to just generate the SQL and leave it in the relational database.

So there are a few basic things about how this stuff works that are not obvious and require some specialization. Even for programmers.

Right now I think it's fair enough to put it in its own job category since there are plenty of software engineers that just don't have any experience with generative AI. But within a few years, I think knowing how to integrate generative AI into a product will be considered core knowledge for a software engineer. So using LLMs or Stable Diffusion will become bullet points on a job requirements list.

Reply on news.ycombinator.com