cmcollier

This is a lot more work than basic zero or few shot prompting of a large general LLM. It's also a bit contrary to the challenge of an autonomous agent in an open ended environment.

It will take time for people to do that kind of work and apply these models in their domais. Similarly it will take time for someone to figure out how to potentially automate the decomposition process so they could be leveraged in more autonomous agents.

Here's my data and search engine with all the HN data:

* https://orangewords.com/

Orange Words. My hobby project, a hacker news search system. It was initially created by hand and now I use AI augmented development. It's a good low risk environment for experimenting.

- https://orangewords.com

Hey all, site owner here. Thanks for the visits and all the fun stories! I really miss this era of computing. Feel free to let me know if you have something that should be added to the site.

Here's some site meta-history too:

https://telnet.org/history/

* https://hamel.dev/blog/posts/evals/#level-1-unit-tests

And more broadly:

* https://applied-llms.org/

* https://www.together.ai/

Here are all the models:

* https://docs.together.ai/docs/chat-models

* https://docs.together.ai/docs/language-and-code-models

* Any solid search engine (bm25 + embeddings and hnsw)

* Any api to a model (gpt3.5, gpt4, claude, etc)

* Some middleware to call search then build the prompt

Then the remaining:

* Create an eval dataset, then tune the search and the prompt as needed

Complex systems can be created through the composition of simple processes which are easily explained or modeled. Sometimes there are mysterious emergent properties in the overall system, even when we can explain the components. Other times, through investigation / science / engineering, we are finally able to explain the entire system. It might lose a little of the magic or mystery as a result, but the system itself didn't change. Instead our perspective and understanding changed.

On that note, until we can fully explain some of the workings of our own minds, I'm reluctant to write off "just predicting the next token" as an unimportant process. It's one way to explain LLM inference simply, but it doesn't eliminate the importance. It also doesn't account for as-yet unexplained things which may be happening as a part of training.

More technically, here's one of the key papers discussing the topic (from google):

* https://arxiv.org/abs/2206.07682

Emergent Abilities of Large Language Models

Scaling up language models has been shown to predictably improve performance and sample efficiency on a wide range of downstream tasks. This paper instead discusses an unpredictable phenomenon that we refer to as emergent abilities of large language models. We consider an ability to be emergent if it is not present in smaller models but is present in larger models. Thus, emergent abilities cannot be predicted simply by extrapolating the performance of smaller models. The existence of such emergence implies that additional scaling could further expand the range of capabilities of language models.

Version history (for relevant dates):

   [v1] Wed, 15 Jun 2022 17:32:01 UTC (59 KB)
   [v2] Wed, 26 Oct 2022 05:06:24 UTC (88 KB)

* The people or industry have low tolerance or fear around risk of false positives

* The industry is centered around billable hours and has no incentive for automation

* The engineers or people perceive ML as this obscure/difficult thing

I'd say the incentives and risks have hindered lots of legal adoption (this is what I observed while working in legaltech for instance). Insurance sounds similar, but I'm less familiar and assume they are coming along more quickly.

[0] I agree with minimaxir's point, that it's a bad assumption to think few teams use basic ML functionality. This will become even more true as emergent tech such as zero shot classification with LLMs becomes more commoditized.

* https://en.wikipedia.org/wiki/Low-background_steel

In terms of overhead, with lower doc counts there's not much overhead with embeddings and knn/ann. Imagine 384 floats per doc or whatever embedding size. At scale it becomes more problematic and less comparable.

With all that said, messing around with vector ops and WASM sounds more fun :)

Vespa really is an impressive platform. I've been working with search platforms for a while, starting with FAST Instream, MarkLogic, Elastic, Weaviate, and now Vespa. As others have noted, Vespa has a deep technical lineage. You can see a lot of the hard earned lessons in the design, as well as improvements in the flaws of the spiritual predecessors (FAST et. al).

The history alone makes the project interesting. But then, in the last couple of years (or maybe earlier), they started introducing all the fundamentals for supporting deep integration of tensors and a custom HNSW implementation. Whether accident or planning, this was a nice strategic move. They were leagues ahead, just in time for the popularization of learned embeddings.

Now they have the best combination of traditional lexical search, semantic search with embeddings, and the combination of the two in hybrid search. That's without even getting to all the functionality in the multi-phase ranking, hosted ML models, and document processing engine.

Obviously I'm biased (we use Vespa at work for enterprise search products), but I can't recommend this engine/platform enough.

Congrats again to the team!