On a dumb phone yes contacts are stored on your SIM maybe using some kind of weak crypto, yes. On an iPhone X, it’s completely unused for crypto or security, or for any other purpose other than carrier authentication.
sims are used for more than just phones. the chip on your credit card is a Secure Element. the chip in your passport is too. these are some of the most physically and technically secure pieces of hardware ever made