Comment by aloknnikhil | Hacker News Reader

Comment by aloknnikhil | Hacker News Reader

Parent Full thread

I completely disagree with you. The fundamental problem with your concept of open source is it goes against what open source really is. The ability for you to completely change what a piece of software can do. IMO, even with LLMs, models are "executables" and weights are "configuration". Yes, of course you can tune the weights by changing the values, but that's the most I can do. Can I actually add "features" to the model? Perhaps you "open-sourced" an LLM model trained on the United States Constitution. Can I change the model to then be a specialist in real estate law? Not with weights. I need it to learn case histories to extend its "feature-set". Without data and the mechanism to reproduce the model, how is this "open-source"?

> Can I actually add "features" to the model?

Yes. You can use a number of libraries to add, mix, merge, etc. layers [1]

> Not with weights. I need it to learn case histories to extend its "feature-set".

Again, yes. You can add attention heads, other features, heck you can even add classification if you want [2]. Because you are working with an open architecture! What you think of weights are not binary blobs. That is a common missconception.

[1] - https://github.com/arcee-ai/mergekit

[2] - https://github.com/center-for-humans-and-machines/transforme...

At first glance, that just seems like a bunch of libraries linked together to form a binary. That is not open-source. I completely agree with you that there is just not enough clarity out there. For my education, following up with my earlier example, can I remove the layers that have references to all chapters / laws in the constitution except for the ones meant for real-estate? How would I do that with the approaches you mentioned here?

Fundamentally, if I have to "reverse-engineer" something, then it's not open-source.

You would have to do the same fine-tuning as if you had the training data.

Reply on news.ycombinator.com