The idea might be weird to approach at first, but you'd be surprised how many people run out and make a copy of their key for you to hold onto too.
Maybe works better for business. I did this with the insurance agent next door when I was running an Engineering office.
Plus, if they have a sealed envelope with the key, they can get enough information about the key (the key bitting) to make a copy. They don't need to open the envelope, they just need to press the envelope down enough to see the shape of the key.
Although, I'm inclined to say if you're giving your house key to someone you don't completely trust, you should maybe seek an alternate candidate.
Then again, "trust, but verify" is indeed a real thing. Your system provides the verification part.