TmpstsTrrctta

They mention tool calling in the link for the smaller models, and compare to 8B levels of function calling in benchmarks here:

https://news.ycombinator.com/item?id=41651126

I’ve utilized this in Lambda@Edge for use case half feature flag-y, half HTTP routing/proxy serving rules as mentioned in a sibling comment. Lambdas pick up ~50-200MB of data on first boot, and keep their copy through their lifecycle.

As requests come in, gather their features, convert to effectively an int vector, filter for row where match and serve request

+1 for rails like these. I’ve used 10 series aluminum extrusions in a roof rack, roof top tent, awning and solar panel setup. I purchased all mine and accessories from here

https://www.tnutz.com/product-category/10-series-extrusions/

I have experience in traditional search as well and I think this is doing some limiting of my imagination when it comes to vector search. In the post, I did like the introduction of the Contextual BM25 compared to other hybrid approaches then doing rrf.

For question answering, vector/semantic search is clearly a better fit in my mind, and I can see how the contextual models can enable and bolster that. However, because I’ve implemented and used so many keyword based systems, that just doesn’t seem to be how my brain works.

An example I’m thinking of is finding a sushi restaurant near me with availability this weekend around dinner time. I’d love to be able to search for this as I’ve written it. How I would search for it would be search for sushi restaurant, sort by distance and hope the application does a proper job of surfacing time filtering.

Conversely, this is mostly how I would build this system. Perhaps with a layer to determine user intention to pull out restaurant type, location sorting, and time filtering.

I could see using semantic search for filtering down the restaurants to related to sushi, but do we then drop back into traditional search for filtering and sorting? Utilize function calling to have the LLM parameterize our search query?

As stated, perhaps I’m not thinking of these the right way because of my experiences with existing systems, which I find seem to give me better results when well built

I had success in a project using Sapper, a precursor to SvelteKit, with using an early version of this called server routes. On server routes, you could define a JSON api which could also have its data statically exported.

On the other hand, I’ve seen server actions abused in large production apps leading to lots of sprawl when handling similar data patterns. Everything becomes an action and it becomes difficult to update the system cohesively or get data externally.

I do feel there could possibly be a bit more magic with Server Actions to generate actual API endpoints. I’m a fan of FastAPI’s handling of pulling path and body variables into an endpoint, typing the params and then accessing them in Next.js on the other hand feels more burdensome. I could very well just be not using it correctly.