According to the recent Salinas v. Texas decision it's not even enough to remain silent. You have to assert your 5th amendment privilege immediately.

As described in a number of articles, 'you don't have the right to remain silent.'