You aren't liable for credit card fraud, but that money comes from somewhere. Today it is a small percentage charged to the vendor; do they pass it on?
And tomorrow, when the problem gets worse and the fees start to climb, will you still not care?
Why be content with a system that may indirectly charge you for other people's lack of security?
Why not look for ways to focus the cost on the vendors who lack security?