Unfortunately modular phones haven't become a thing yet. Google tried that with Ara, but it's in indefinite hiatus.
Of course, the next problem with making phones modular like that is a lack of standards. One developer makes the modules and keeps others from making them, thus entering the walled garden model.