Hacker News Reader

The current trend for productionizing LLM-based applications is to write your own (really thin) wrappers around the actual LLM call. The majority of your code should be business logic independent of the LLM, anyway: information retrieval, user interface, response logging, and so on.

In my experience with cloud-based and local models, LLM chaining compounds errors; I would urge you to look at few-shot, single-query interaction models for business applications using LLMs as a new unit of compute.

It’s great for prototyping and seeing what is possible but for running in production you’ll likely need to write it yourself, and it will just take a few minutes.

Check out https://www.featurebase.com/blog/doctorgpt-harnessing-the-po... and the DoctorGPT repo for doing some things with documents and LLMs, without a framework. I use simple templating and vectors to assemble prompts.

Could be we're in a (short?) interregnum analogous to pre-Rails Ruby: there are lots of nascent frameworks, but the dominant one hasn't been born yet. FWIW - DIY is working well for me.