Mistral 7B,The complete Guide of the Best 7B model | Hacker News Reader

Mistral 7B,The complete Guide of the Best 7B model

mistral-7b.com

32 pointsgstarwd15 comments

IIRC there was a recent paper that showed Mistral may have data contamination, leading to high scores on the benchmarks. In my experiences with Zephyr, OpenOrca, and Dolphin finetunes of Mistral, the model did demonstrate ChatGPT-esque format in responses, but it didn't demonstrate any more intelligence than vanilla Llama 2 7B.

Another thing to worry about is that Mistral (the company) has publicly said that future larger models will not necessarily be open like the 7b model. After a certain model lock-in has taken place, the company will guard the model behind API (as they mentioned) and it won't be open anymore.

I don't know whether Mistral had benchmark data contamination or not, but it's definitely a stronger model than Llama 2 7b or 13b even on non-benchmark tasks.

We proactively tested Mistral across all of our customer-deployed models on our fine-tuning platform (openpipe.ai) and found that with the exact same training data it outperformed both of those models consistently. At this point we're only keeping Llama 2 7b/13b around for our legacy customers and recommend that everyone training new models use Mistral because it's just so much better.

I've tested it and many fine tunes and while it seems to work ok for short chat it's really inattentive in summarisation and information extraction tasks. Continuation of long context stories/chat is also severely lacking, making up lot of contradictory details as it goes on.

It might or might not be better in benchmarks, but I just don't see it in practice for many tasks where you'd pick a LLM and go

There is one thing tho where I've seen it perform really well, which is single task fine tuning. It's small enough you can do full tuning and it's steerable enough you don't need more than few thousand samples.

But it's definitely not a pick and go model like a Vicuna-13b would be.

Eh, I've found llama 7b finetunes to be underwhelming too. They both seem to fall apart at detail retrieval and such, even with Mistral's longer base context.

For what its worth, Mistral finetunes excel in this ranking, and Mistral AI almost certainly did not contaminate it with that test: https://rentry.co/ayumi_erp_rating

I wonder if it's possible to do something like folding at home but train an llm and have it always be open

https://github.com/bigscience-workshop/petals

Inference only: https://lite.koboldai.net/

wow didnt know that, thats really cool, ill have to play with it

If you want to host for the AI Horde, Aphrodite has the most GPU throughput, and Koboldcpp is the easiest and quite flexible:

https://github.com/PygmalionAI/aphrodite-engine https://github.com/LostRuins/koboldcpp

Picking a model to host is quite a rabbit hole, but I am very interested in SciPhi and Zephyr at the moment.

I havent messed with Petals yet either, but swarm finetuning is very appealing... If thats even implemented atm.

No offense, but at first glance I thought this was a GPT generated SEO ad farm... with titles like "Best AI chat bot Zephyr 7B inside finetuning Mistral 7b," "How to Finetune mistral 7B for free and easily with AutoTrain" and the general format.

But its ad free, with no promotional links, and the information is good. For instance, I like the VRAM calculator, and the list of the top finetunes is excellent (like Zephyr Beta and SciPhi, both of which are very new and quite impressive).

Mistral is indeed amazing. I hope they keep updating it, and maybe even publish a 13B version.

> No offense, but at first glance I thought this was a GPT generated SEO ad farm...

You're not far off. The majority of the content is ripped from the real site: https://mistral.ai/news/announcing-mistral-7b/ and https://docs.mistral.ai/usage/how-to-use/

Unfortunately the real site doesn't have gems like "Top rated Vedio about Mistral 7B " in 30pt italics.

Its weird... it has AI generated images for the finetunes listings as well, complete with some text, with descriptions taken from HF of course. But some of it is... off. Seemingly not human curated.

And of course the Mistral company page does not list any finetunes.

Is this a click farm? Bait and switch for future ads? What the heck is this?

We are using Mistral at https://lemonfox.ai/. With Lemonfox you can easily switch from the OpenAI API to open source AI models. Mistral has been working well as a gpt-3.5 replacement but we are currently planning to switch to Zephyr - a fine-tuned Mistral model. We've seen some nice improvements with Zephyr especially with non-English prompts/responses.

Are y'all doing anything in particular to stretch the context out? Just plain old rope scaling?

While it seems to be a base model, I'm pretty excited about coming finetunes on this one: https://huggingface.co/SciPhi/SciPhi-Mistral-7B-32k

Reply on news.ycombinator.com