What if you and your wife enter on different flights? Wouldn't it makes sense to carry each other's devices and not share the passwords with each other? You aren't refusing; you have a plausible defense to not knowing the passwords.
It's less compelling, but i guess you could have a trusted friend change your passwords for you and require them to later unlock them. (after you get out of detainment 5 days later and fight in court -- instead of the gov't winning, you both lose)