Similar questions for civil trials, divorce proceedings, child custody....
Your phone is already more or less an extension of your brain, and whether or not you can be forced to unlock and surrender it for inspection is already a contentious topic.
IANAL, but phone privacy would probably set the precedent for AI assistant privacy.
Always keep your phone encrypted and be aware what your local laws are. Some places will force you to provide biometric authentication, but not provide a PIN or password. Check if your phone has a duress lockdown mode: some phones lock and/or wipe if you press the power button five times or something like that.
Humans can be required to testify too if they're immunized.
If a cell phone is recognized as having a higher expectation of privacy than a mere passive document, then it stands to reason courts will also recognize a personalized machine that is even more earnest to help as my infernal "by the way..." Alexa home device.
The fifth amendment has very different criteria - with a proper warrant, your most private things are admissible evidence. You can't be compelled to testify but all your most private notes in a safe can be "interrogated"; in many situations you can't be compelled to testify against your spouse but any writings or recordings of what you said about him/her are valid evidence. There is no debate that all the contents of your computer or phone can be used, they definitely can, the cases you quote are disputed only because they are "worthy of the protection for which the Founders fought" which is the requirement for a warrant.
I'd say that fifth amendment is not a privacy law (which is the 4th, stating that you have all the privacy without a warrant and no privacy with one), the 5th is essentially an "anti-torture" law to prevent coerced confessions, and there is no reason why it would apply to some physical evidence like the data for a trained model on your device.
You can read a letter. You can play an audio recording. You can look at a photograph. You can print a word-processing document. Those feel like static records. Each tells a single story, no matter how many times it's read.
An ML model, on the other hand, is just a long sequence of floating-point numbers. There is no meaningful model "viewer" that spits out a text file or JPEG. The only thing a non-engineer human can do with an ML model is interact with it when it's running. Thus, any output is the product of both the human and the model. It doesn't feel like a record. It feels more like a performance -- and a performance by the interrogator, at that.
If an ML-model "record" is a special kind of record that needs manipulation to produce human-readable output, then it feels like we're back in the 5th amendment department. No, chatting with a chatbot is not torture. But it produces a kind of evidence that is an unwelcome collaboration between the interrogator and the record (unwelcome by the owner of the record). It's one thing to let a jury look at a screen full of numbers. It's another thing to say that an expert witness used those numbers in a chat session that printed "Yes, it was my owner, in the living room with the candlestick."
If you lock up a person and ask them the same question over and over again, eventually you'll get the answer you want. The same will probably turn out to be true for that person's chatbot. Should society allow that second case?