My 2024 AI Predictions
axflow.dev
axflow.dev
If I had to invest in any one area of LLM usage, it would be this. There is so much unstructured data in the world, and converting things like legal contracts or chatlogs into structured, queryable data is absurdly powerful. Nobody wants to talk about this usage for LLMs because they're too busy making TikToks about how GPT4 actually has a soul or whatever, but this will be the lasting legacy of LLMs after the hype around generative AI dies out.
> A decent engineer will likely be able to write a slack-like application, definitely good enough to cancel the 500k/year contract, in a couple of months.
And this is why generative AI is massively overhyped: the people hyping it don't understand the true value of the products they allegedly replace. Very similar to the crypto/blockchain hype where people who understood nothing about banking or logistics insisted that blockchain would solve all the problems there. If you think a corp is paying Slack $500k/year because it's hard to write a piece of software that can send messages between people in an organization, you're completely off base. (IRC exists, can do this and is free by the way.)
Can you give a couple of specific examples? There are already sites like pdf.ai that let users chat with documents, are you thinking of something different?
[0] and very hard to answer with an LLM, as they are notoriously awful at doing any sort of math.
If possible, could you talk a bit about your process for training your model? It looks like it is specific to legal documents, how easy/hard would it be to do the same for other types of documents?
Also, what level of accuracy is good enough, for your use case?
I don't have a really specific accuracy target but since my main interest is in the aggregated results it's not a huge deal if the aggregations are slightly off. (The manual human approach is not 100% accurate either, after all; it's extremely common for humans to make data entry mistakes.)
Healthcare's another. Exchanging information between benefits brokers is madness because of inconsistent data formats.
A more consumer-facing example: exporting/joining data from one or more apps to migrate into another app. This is a hellish process most of the time that an LLM could, theoretically, reduce to seconds (not counting execution time).
I tested poorly OCR'd text from a late 1800s magazine and was pretty impressed with the results from both GPT-4 and even Bard. In addition to making inferences based on the letters, it was able to infer from context the historically accurate terms. However, one prompt asking it to correct words didn't work as well as two prompts with the first one listing candidates for bad OCR'd words.
While that might not seem like what you're talking about, it has the benefit of adding to the overall corpus that the future models are trained on. Also, with the NYT lawsuit, I'd presume fixing OCRs of old magazines and articles would be a pretty good way to fill the gap left behind.
> A decent engineer will likely be able to write a slack-like application, definitely good enough to cancel the 500k/year contract, in a couple of months.
A decent engineer can already crank out a working Slack prototype within a couple of months, and there are mature Slack alternatives today. There's a reason companies are paying $500k/year, and I doubt it's the code: maybe it's the enterprise support, the external integrations, or even just the name recognition.
Companies getting leaner may be true (it seems like this has already been happening the past couple years regardless of AI, and companies used to be lean in the 2010s).
There definitely is room for such applications, but a chat application probably won't set the business apart.
This is a pretty hard problem. And I haven't found anyone that's too good at this, but here are some interesting players:
- https://www.phind.com/ is a custom model fine-tuned on code, and pretty damn good - https://codestory.ai is a VSCode fork with an assistant built in. One of the things it does for you is write code, but imo that's not its biggest strength yet. - https://sweep.dev have a bot where you create a GitHub comment and it writes the PR to fix it. They have between 30% and 70% success rate. This is pretty bad but they're one of the best today - https://sourcegraph.com is pivoting and building a copilot application (named Cody). This is pretty good, since sourcegraph is great at understanding your code
(disclaimer: I work at Sourcegraph)
People are rightfully calling out this bit. It still wouldn't make sense for a Slack customer to make their own version of Slack in-house, but it does lower the bar for a lot of Slack competitors to get to feature parity much faster.
I go back and forth on this. While I see this being the case for data collection wearables like humane or tab, it makes sense to have a personal AI computer like bedrock [0], tinybox [1], or a mac studio for running background tasks on personal data. If you're running agents that do more than chat, you need something that's going to be able to handle doing inference for extended periods of time without worrying about heat or battery life. You likely also want something capable of doing fine-tune level training on your personal inputs. A lot of the more interesting use-cases are on data you probably don't want to expose to a cloud provider. That said, probably Apple is eventually going to crush here as well, but maybe there's room for a challenger to develop as this niche opens up.
[0]: https://www.bedrock.computer/gal [1]: https://tinygrad.org
I don't think it will take much more than that to get the rest of the way.
I don't think that just because the author has a pseudo-conversation with ChatGPT using voice as the interface means we've passed the Turing test.
They don't seem to be actively interrogating ChatGPT to determine whether it's a human or not - something that I'd expect would still be quite easy to do. And, as I understand it, the Turing test could be administered over text.
And millions of people are fooled into thinking GPT is a real person every day, with spam and robocalls and social media bots. Maybe it won't fool everyone all the time, but it can fool some people a lot of the time. And it's only going to get more sophisticated. The only ones concerned about the Turing test are 70 year old GOFAI professors -- everyone else is dealing with the practical realities of computers suddenly having language capabilities.