HNHacker News
TopNewBestAskShowJobs

b2v

5 karma · joined June 2, 2024

Verifying ownership of this account: I am @b2v_ on b2v.xyz | https://b2v.xyz/b2v_/mRrk4eufB3PBmf?s=hn:b2v

contact@b2v.xyz

submissionscomments
b2v··on Every Way to Get Structured Output from LLMs
I might share it with you later on your discord server.

> I can send over my best guess of a good prompt!

Now if you could automate the above process by "fitting" a first draft prompt to a wanted schema, ie where your library makes a few adjustments if some assertions do not pass by have having a chat of its own with the LLM, that would be super useful! Heck i might just implement it myself.

b2v··on Every Way to Get Structured Output from LLMs
Neat, thanks! I'm still pondering wether I should be using this since most of the retries I have to do are because of the LLM itself not understanding the schema asked for (eg output with missing fields / using a value not present in `Literal[]`) — certain models being especially bad with deeply nested schemas and output gibberish. Anything on your end that can help with that?
b2v··on Every Way to Get Structured Output from LLMs
Thanks for the clarification. How do you handle dynamic types, ie types determined at runtime?
b2v··on Every Way to Get Structured Output from LLMs
I'm not sure I understand, in the docs for the python client it says that BAML types get converted to Pydantic models, doesn't that step include the extra latency you mentioned?