I have a dumb phone and mainly leave that at home. Anybody who wants me can leave a message and wait for a callback. Seems to work well for past 20 years.
I don't have a fancy one for the very reasons of tracking, loss of privacy/data collection and above all for this type of use, and what the parent didn't mention, I do not trust tech even slightly, being a tech guy myself.
If you rely on power and intelligence, other than manpower and brainpower, to unlock a door, then you're a bloody fool.
(NB cost of a spare key ~£5).