We're Gonna Need a Bigger Moat
steve-yegge.medium.com
steve-yegge.medium.com
Ok, so. Where to start?
Hmm… ok. Well, how abort we ignore all the rest of the stuff and cut to heart of this article:
Assertion 1) Vicuna 13B is as good as bard and chat gpt.
Assertion 2) LoRA can stack infinitely and every LoRA you stack on top is additive only, increasing capability knowledge with no downside.
I think you can make your own mind up if you choose to believe these assertions, and judge for yourself if the author is on the right track or not.
I will link however, to the actual comments by the authors of vicuna, specifically the limitations and evaluation section: https://lmsys.org/blog/2023-03-30-vicuna/#how-to-evaluate-a-...
Maybe read it yourself and make an informed opinion…
(Unlike, it appears, the author of this article)
And sadly it just ends up being a sales pitch for his latest project.
That said, as a user, the moat between ChatGPT and say, Bard seems as wide as the Pacific.
Put another way, in my tests giving both coding questions, I'd probably hire ChatGPT as a junior or intermediate developer. I'd probably tell Bard to choose a different profession, if not recommend classes for the mentally challenged.
If all of this were so fast moving and cheap, why is Bard so far behind?
Do you think it’s a coincidence Microsoft bought GitHub and all of a sudden they have a coding focused bot ? Weird right ? Think of all that sweet training material.
Just answering questions is a glorified Ask Jeeves.
But yeah, I’m not quite sure what the end goal of l this research is supposed to be either. Seems like we’re not acting very intelligently on many fronts.
If you can't properly answer about your own API's what good are you ?
From my observation bard is better at sql than chatgpt
Knowing what those little components look like, for me, would help me evaluate which model is actually better. Right now it feels to me that the difference between the two comes down to more copyrighted materials in ChatGPT’s training set… and more sophisticated ways to hide that, but that’s just my hunch.
“That fundamental formula has not really changed much for years,” says Jakub Pachocki, one of GPT-4’s developers. “But it’s still like building a spaceship, where you need to get all these little components right and make sure none of it breaks.”
https://www.technologyreview.com/2023/03/14/1069823/gpt-4-is...
Like the “invention” of fire?