You are taking PIN security too seriously, yes. Your bank doesn't store them in the clear, they store them in an encrypted form, but they are able to decrypt them. There is a huge legacy of old hardware and software in banks. Much of the PIN issue has to do with maintaining backwards compatibility with cards without chips (magnetic strips), and readers that can only read those cards. I've encountered cash machines which didn't prompt for my pin code until after my card had been returned.

Don't sweat it. This is the least of your concerns.