If you think that it's not true (that silence is not complicit), then shouldn't you attempt to explain that?
You aren't entitled to a conversation.
Work isn't your entire life, you can be loud outside of work and yet still be silent at work. Think of the phrase, "Don't shit where you eat".