It is possible to merge two fine-tunes of models from the same family by... wait for it... averaging or combining their weights[0].

I am still amazed that we can do that.

[0]: https://arxiv.org/abs/2212.09849

Yes. Using fine tuning.

Yes, there is the concept of a "frakenmerge" and folks have also bolted on vision and audio models to LLMs.