You are required to show your card to police when they suspect there is a crime. You can refuse by asking if there is a reason to believe there was a crime and if this is a "professional inquiry"

If yes, they must provide the reason. [1] At that point, you may have to show them your card depending on the circumstances. They may leave you alone, however.

My experience: I was once taking photos in a dark alley for some cyberpunk type shots around 9 p.m. I was approached by two officers who asked, "What are you doing here?" Kind of a trick question, because then they said, "Oh, you speak Japanese. Do you have your resident's card?"

I complied here because of the optics. After complying to show my card, my bag and me were fully searched which I was not okay with. After that I experience, I became well acquainted with my rights because, at that time, I did know them well.

1 - https://www.japantimes.co.jp/community/2014/04/02/issues/rig...

[1]'s source / reference: http://www.debito.org/shokumushitsumon.html