It's standard practice for cleaning/home-maid services and a number of food delivery services (e.g. Abel and Cole) already do this. It's just a matter of trust and reliability.
I'd like to see how Amazon manages the access. Can access be revoked manually? Can you generate your own access permissions to give to a friend? Is this access code a one-time code?