This looks great; very useful for (example) ranking outputs by confidence so you can do human reviews of the not-confident ones.
Any chance we can get Pydantic support?
Any chance we can get Pydantic support?
If you run "bananas,fishbowl,phonebook," and get {"sponge": 0.76}
It doesn't mean that "placemat" was the 76% correct answer. Just that the word "sponge" was the next most likely word for the model to generate.
The library is compatible with that but does not use Pydantic further than that.
```
class Classification(BaseModel):
color: Literal['red', 'blue', 'green']
then the output type would be:
```
class ClassificationWithLogProbs(BaseModel):
color: Dict[Literal['red', 'blue', 'green'], float]
Don't take this too literally; I'm not convinced that this is the right way to do it. But it would provide structure and scores without dealing with a mess of complex JSON.
One question I always had was what about the descriptions you can attach to the class and attributes? ( = Field(description=...) in pydantic) is the model made aware of those descriptions?