There's a balance to be struck here.

Coming from HomeKit, I've tried (_really_ tried) to move to Home Assistant. The gap in usability was too enormous for me to cross, and I'd brand myself a hacker.

I won't trust any kind of a "smart" device to operate the front door lock - ever - but smart lightbulbs are still stupid lightbulbs. I can just flick the switch.

Privacy concerns are valid - I can be profiled based on usage. But it's not like Apple doesn't know my precise location already.

With all that in mind, I'd say usability comes first.