Is anyone using open source models to actually get work done or solving problems in their software architecture? So far I haven't found anything near the quality of GPT-4.
https://github.com/nlpxucan/WizardLM/tree/main/WizardCoder
https://huggingface.co/WizardLM/WizardCoder-Python-34B-V1.0
Top of the line consumer machines can run this at a good clip, though most machines will need to use a quantized model (ExLlamaV2 is quite fast). I found a model for that as well, though I haven't used it myself:
https://huggingface.co/oobabooga/CodeBooga-34B-v0.1-EXL2-4.2...
The decent chat ones are based on gpt data and they’re basically shitty distilled models.
The best use case is a narrow one that you decide and can create adequate fine-tuning data around. Plenty of real production ability here.
GPT-4 has an estimated 1.8 trillion parameters. Orders of magnitude beyond open source models and ~10x GPT-3.5 which has 175 billion parameters.
https://the-decoder.com/gpt-4-architecture-datasets-costs-an...