It's true to some extent that crypto users are more savvy than most, but I think cryptocurrency also has obviously superior security properties to the conventional banking system, technically speaking. In the conventional banking system, there are no "savvy" users because everyone is equally insecure no matter what. In the conventional banking systems it's all based on trust. Trust that the bank obeys the law, trust that law enforcement is not corrupt, etc. The security mechanisms of cryptocurrency are at most a superset of what you can do with the conventional banking systems. If you want to have a third party supervising transactions, you use 2-of-3 multisig for example (https://en.bitcoinwiki.org/wiki/Multisignature). If don't trust your family member to authorize payments without you, you use 2-of-2 multisig. If you don't trust yourself to not lose your keys, you back them up. If you want to limit your risk, you keep a small amount of cryptocurrency in a "hot" wallet.
Secondly, I don't think that the idea of a keypair is beyond the understanding of an average person. They effectively already know how to manage secrets in the current system: passwords, bank routing numbers, etc. It's just that the keypair is superior to these systems of authentication which often require you to reveal the secret itself to authenticate (credit card number), do not have enough entropy (4-digit-pin), are open source (E.g. security questions like "what's your mother's maiden name?"), or rely on other centralized systems (SMS-based 2FA). Even if you implemented some sort of "custodial keypair" that allowed you to transparently sign transactions without revealing your secret, that would be a major improvement over the current system which is based on (typically bad) secrets.
In many ways, the conventional banking system is more complicated than cryptocurrency, because the failures of cryptocurrencies are "solid" and well-defined (e.g. 51% attacks, MITM attacks, etc.) while the failures of the conventional banking system are "soft and fuzzy". For example, I was reading about a scam the other day wherein the attacker sends the victim a fake check, and asks them to cash out the money- this scam works because banks generally accept checks before validating them, allowing you to spend money that hasn't been validated yet and then charging you later. You might think this is obvious as a boomer, but as a zoomer who has never cashed a check before, this is not obvious at all.
And I'm not necessarily saying that cryptocurrency is the end-all-be-all of payment systems. There are superior systems like chaumian cash (https://taler.net/en/) but they require the permission of the existing banking system (which generally profits off providing services that surveil users and """fix""" the existing insecurity), so they haven't taken off.
I get the impression that regulation will never fix this because the nuances at hand will go over the head of any lawmaker who has merely accepted the insecurity of the status quo. I think that even if you get some libertarian or pro-cryptocurrency person in office which doesn't accept the current system, I highly doubt that they would make the right decision needed- it's more likely that any pro-cryptocurrency candidate is just going act in a way that benefits cryptocurrency owners.
Compare this to a topic like net neutrality. Even though I am a libertarian, I am more aligned with the democrats' views on net neutrality because of the obama administration's actions. Why? Not because the democrats are especially aligned to solve this problem, but merely they happened to have a good cabinet member or something that happened to understand the issue that election cycle and advise obama on that issue. It seems just as likely to me that the opposite might happen, albeit the democrats tend to be more pro-consumer in general. My point being that elected officials will not campaign on this because it is too nuanced, so solving this through politics is futile. It is better to just to improve cryptocurrency (or some other non-permissive technical solution) until it is competitive and forces the government/banking system to adapt (e.g. Project Hamilton).
P.S. I don't understand what you mean by a "direct wallet" here. A hardware wallet?