seany62

I hear this a lot and I think it is good advice because the only person who should actually start a startup is the one who sees this but still does it.

Should we not mimic our biology as closely as possible rather than trying to model how we __think__ it works (i.e. chain of thought, etc.). This is how neural networks got started, right? Recreate something nature has taken millions of years developing and see what happens. This stuff is so interesting.

Give examples of how the LLM should respond. Always give it a default response as well (e.g. "If the user response does not fall into any of these categories, say x").

> I can manually add validation on the response but then it breaks streaming and hence is visibly slower in response.

I've had this exact issue (streaming + JSON). Here's how I approached it: 1. Instruct the LLM to return the key "test" in its response. 2. Make the streaming call. 3. Build your JSON response as a string as you get chunks from the stream. 4. Once you detect "key" in that string, start sending all subsequent chunks wherever you need. 5. Once you get the end quotation, end the stream.

There's very strict regulation around unlicensed individuals performing activities requiring a license (negotiating, showing property, etc). Thus I would be surprised if they have AI doing this for a client!

AWS only brings in 80B+ a year, must be from everyone self-hosting. I'm curious why people think this is different?

I've gone down the same rabbit hole and wasn't really able to find anything. Have been thinking about building an open source alternative myself