Wouldn't it be a violation of the 5th amendment to be compelled to provide that passphrase, because it is an admission against interest and therefore would be admissible if you disclosed it? Wouldn't it also serve to waive 5th amendment privilege, and possibly put you at risk of being forced to take the stand?
If it were an admission of a different crime, a court could grant you immunity on those unrelated charges, but if it is relevant to the crime the government is investigating by asking you to reveal your passphrase... how can anyone, luddite judge or not, separate "key" from "testimony" in that circumstance?