Hacker News Reader

In my opinion the problem with these is engineering a good prompt. I read of lots of people only getting nonsense or repetitions, and learned a bit from what they shared. These models are not chat bots.

Vicuna is more friendly in that regard.

But I’m well aware of their limitations also, and I can see how one can be underwhelmed. They are not jacks of all trades

View on HN

Alpaca & Alpaca-LoRA are literally trained to be chat bots. Their goal is to be instruct-tuned.

Alpaca-LoRA, and all LoRas, are garbage. Alpaca is horrible compared to newer finetunes. Even cleaning the Alpaca dataset and retraining a cleaned Alpaca improves its performance greatly.

But newer finetunes like Vicuna go well beyond that, including hundreds of thousands of real human conversations with GPT-4 ChatGPT in the dataset (unlike Alpaca's fully synthetic dataset).

Vicuna-13B in 16bit is easily comparable to ChatGPT-3.5 in capability. Newer finetunes coming out nearly every day are going beyond chatGPT-3.5 and getting closer and closer to GPT-4 performance.

You don't even have to install anything to validate this for yourself. There's a live web demo of Vicuna-13B right here: https://chat.lmsys.org/ (disable ad blocker if it does not load)

Also, InstructGPT was released in January 2021 and Vicuna-13B blows it out of the water.

Reply on news.ycombinator.com