In addition to paid subscriptions, you might want to consider running an LLM locally. One of a number of projects enabling this approach is "continue" - https://github.com/continuedev/continue
The issue with local IMO is that you'll going to have to compromise on quality. And I really don't want to waste my time talking with inferior models when better models are widely available.
Yeah, at the time of writing this comment it feels like if you want to easily convince me that your local LLM is good enough, just do a side by side comparison of prompts with your LLM vs GPT4. It doesn’t even have to be better. It just has to be kind of in the same ballpark
I'd like to try this, do you have some prompts you like to use for evaluation?
people who think that their local LLM's are in the same ballpark as GPT or Claude are delusional
Haven't found any good local model yet that rivals or come close to gpt-4
Have you tried miqu q5
I have dabbled with the quant version (exl), still not the same. Which quant/version are you using? There are so many that the output shifts dramatically!
I'm using gguf q5 and I find the output degrades going to even q4. Mistral has said they still intend to open newer bigger models, so I'm really looking forward to that.