There are two parallel systems for payment cards, only one of which is "protected" by the PIN and which one is probably instructive for you.
Authorisation is the system which verifies that you, the card's holder, authorised this payment. This is the part that EMV ("Chip and PIN") and lots of anti-fraud technologies are focused on, and yes its main purpose is to prevent you (though also the merchants) from defrauding the bank or other card issuer.
Settlement is the system which transfers money from your account to a merchant's account.
Authorisation is optional. If there is zero authorisation a settlement will still work, though there is some higher chance either the card holder later disputes it or the issuer decides it was fraudulent by their own methods.
Big merchants often just don't use Authorisation at all. Whereas if they didn't do Settlement they wouldn't get the money, not doing Authorisation just means it's harder for them to "prove" you agreed to pay, but they may find it just isn't worth fighting you anyway.
Knowing this you should read your card statements every time they're sent, in proportion to how careful you need to be about money (if you're Elon Musk maybe don't sweat any line item under $1000) because that's the first time you may find out that, for example a hotel in a country you've never visited has submitted Settlement data taking $500 for a hotel room you never booked. Mistake? Fraud? Who cares, if you don't spot it then you're out $500.
Yes banks may tell a jury in a fraud trial (against you, their customer who has been stolen from, convicting you of fraud means they aren't on the hook for the missing money) that only you should have known the PIN and therefore the jury should assume that your inability to explain how "someone else" had the PIN means you're lying. You will need a good lawyer and experts to explain that actually the bank's insiders can know the PIN (the bank will try to dodge this awkward fact, insider fraud at banks is common and banks would prefer you never think about that) and there are various means by which it could have been stolen that the bank doesn't do enough to prevent.
When EMV ("Chip and PIN") was introduced the main increase in fraud went like this: You hide a skimmer inside the PIN terminal, and it harvests card details (it could also harvest PINs). The terminals are supposed to have "anti-tamper" systems to defeat that, but they're trivially bypassed. Now, you can't use these to make EMV cards, but you can use it to make a convincing fake non-EMV card, and then use it in countries that haven't rolled out EMV.
This fraud mostly went away as more countries finished deploying EMV but other types replaced it. Some of these things are only possible because EMV is a typical "Hand rolled crypto" solution where nobody remembered to hire actual expert cryptanalysts to critique and revise the design before it shipped. Others aren't technical at all.