The reason your question is "stupid" is because you think your phone is just one computer.
In fact, you can do just what you describe - and it wouldn't be that difficult - but it would only be for the application processor and you would end up with a nice PDA.
In order to make phone calls you would also need to control the other two, fully functional, general purpose computers inside of your phone: the baseband processor and the SIM card.
Currently, there is no way for you to control either of those in any meaningful way.[1]
There are deeply entrenched economic and technical forces standing in the way of controlling your own baseband processor and we are nowhere near to doing it.
[1] Osmocom on 2G phones is not meaningful, no matter how impressive and interesting it is.