I was playing around with OpenAI's API, and found I could get the same output from the same input if I turned the temperature down to 0. Does anyone know if this is a feasible way of getting deterministic output from an LLM?
Yeah, you have to force the results into strict JSON, enforce it, and handle a lot of error cases, and be prepared to give up or reroll on failure. It basically takes a 2nd LLM pass to standardize the JSON keys of the initial result. But it's worth it to be able to turn messy stuff into structured data.