The Llama Ecosystem: Past, Present, and Future
ai.meta.com
ai.meta.com
I have mixed feelings, llama is great but it's perpetuated it's shitty license. They could have done so much more good if they'd used gpl style licensing, instead they basically subverted open source, using an objectively good model as leverage.
Absolutely not. There's a corner of the overall community that hovers it and overperceives it as everyone else only uses it too.
Its great if you have an Apple ARM machine and want to see an M2 Pro do 10 tokens/sec (and what could make an Apple ARM have 30 minute battery life).
I also doubt it's a slight, the only callouts are large commercial collaborations, ex. nVidia, AMD, Google are representative of each of the 3 groups we could assign it
There's a certain type of myopia that leads to overindexing on llama.cpp that makes it easy to classify. to wit:
> not aware of a competing format for quantized models
ONNX, that's how its done in prod and on other models besides (and including) LLaMa. Quantization is a general technique. 100 small variants of llama2 GGML weights feels like spam from that perspective. (sort of civitai vs. huggingface, hugginface smartly stopped that with AI art).
llm.mlc.ai for a more academic / less ad-hoc approach.
> [stars on github]
It's great for a very narrow & simple case that matches a large demographic on Github, and the demographics of people talking LLMs casually on HN: MacBook, wanna run locally and dream of a future free of having to ship your data to servers to get personalization. 5% of overall usage can be #2 in usage, if that makes sense.
Right, looks like you made fun of / were condescendingly dismissive of my comment in another thread, I wouldn't have replied here if I'd realized you were the same person.
I apologize for making you feel condescended to, but also would like to point out the _mean_ comment is +7, much less this one: there's a pretty significant gap in your knowledge and reality is going to keep intruding. Engaging in public is a wonderful way to learn, but you're coming across as glib and assertive and uninformed. You thought llama.cpp invented quantization and there's no other real format? :X
Most human people doing LLM at home aren't interested in cargo culting the for-profit corporate and instituational stuff since their resources and incentives are so different from human being's incentives. As there are more humans than corporations or institutions and they tend to talk more, what they use tends to be more known than the stuff optimized for making a profit and serving business needs with business culture.
AWQ support is spreading more, which is nice.
Of course quantization was invented well before LLMs. However, LLMs have dramatically accelerated development on quantization and resulted in an explosion in use.
The license for Llama 2 is pretty intense, but mirrors that intent by limiting interactions with individuals at scale, as well as limiting anything learned from the model through inference in being used to train another model. I suspect this is because the dataset on which it was trained is the company's IP, which again is for the shareholder's benefit.
The code is open though, I think out of necessity. AI poses a significant challenge for our survival, and making it open is an indication of transparency. They still need to make money at what they do and charge people for using their IP, within reason.
I guess my question would be that, if I used Llama (not the code, but the model itself) to code up a new model, would that be a derivative work?
Aka, my own comments being sublicensed back to me, after I licenced them to Facebook.
Let’s say I want to find the latest or most recent projects on this, is it possible to find them on GitHub based on that criteria?
Llama is not Open Source but until we get a court case ruling one way or the other we don't know if it's actually locked-down in the way Facebook intends; and I want to strike a balance between (correctly) pointing out that Facebook is misusing the Open Source label while not ceding to Facebook's claims about how much it can legally constrain people who have never signed a single Llama TOS.
ok seriously though I had fun over the weekend chatting with Samantha on a long car ride on my MacBook. We were mostly asking about history.
[0]: https://huggingface.co/TheBloke/Samantha-1.11-CodeLlama-34B-...
It’d just be better if it was around RWKV or something that doesn’t prevent you from improving any models outside of the llama ecosystem.
It’s a great embrace, extend, extinguish play by meta.
> It’s a great embrace, extend, extinguish play by meta.
Meta released Pytorch, Pytext and even built ONNX with Microsoft to avoid an EEE situation. What more could you possibly want?
No. A good parallel would be if Microsoft (say) wrote their own linux clone that was compatible but had some proprietary enhancements that made it desirable over open source distros. The only catch being, it wasn't gpl licensed (they wrote it from scratch) it had a proprietary MS license that says you can only use it for things MS approved of, and are using it at their pleasure, to be revoked at any time.
People don't care about the license, they call it open source and move away from gnu/linux to the proprietary MS version, and now we're only doing what they allow us to.
That's exactly what's happening in the ML model world right now, but people are happy with the shiny models Facebook lets them use so they say "what's the big deal".