Should is a funny word. Just because our perception of fairness dictates that we should control our phones, its just not going to happen in the foreseeable future.
It is in our long term interest to have as much control of our "digital assistants" (which includes both smartphones and personal computers) in order to ensure that they assist us, and not some other party.