For key pairs that can be transferred, it's no different than the threat of password managers stealing your keys to the castle. You just have to trust the cloud and your entire hardware and software stack your password and passkey manager run on, and/or that nothing in that stack gets swapped out without you noticing.
Similarly, I've yet to see a hardware security key that you can verify doesn't have backdoors compiled or soldered in. All the cryptography in the world doesn't matter if a government/whoever can just pop your Yubikey in a machine that does whatever magic incantation is needed to dump your keys. That magic incantation can even be secured with strong cryptography to ensure only the US government/whoever can use it.
The Precursor is also open hardware and software.
If you trust any smartcard at all running a Javacard-compatible operating system, there's also https://github.com/BryanJacobs/FIDO2Applet .
And of course if you're truly paranoid you can get a FPGA and implement a hardware security key on that. The overall security posture would likely be weaker, but you could be confident, hopefully, that nobody has put some kind of backdoor into the hardware you designed yourself to run atop a generic array of logic gates.
Looks like the Solo HE lets you load your own firmware on to it, but it doesn't let you load your own signing or encryption key to ensure firmware updates are trusted. Apparently the Solo HE can be flashed once permanently by overwriting the bootloader, though.
The non-HE versions of the Solo 1 and 2 will load new firmware signed by Solokey.
Earlier this year I looked into this and remember finding out that it was either the Solo 1/2, Somu or one of the Nitrokey products that, while shipping with a secure element, didn't actually use the secure element.
If you think some company in the future will have the ability to somehow "steal" the contents of the device's flash, you'd still have to climb the mountain of explaining how they could then break the encryption the open-source software already - before they got hold of the key - applied to the flash contents.
Just to make sure this is clear, the security key at rest is not storing your credentials. It is storing AES256(key=<PIN>, value=<credential>). It is not storing the PIN.
You only need to trust the hardware to implement encryption correctly, which you can - of course - verify yourself. It's not realistic to say that the pre-encrypted values might be secretly stored somewhere else: there just isn't enough space on the device to do that.
Spy agencies, law enforcement, and criminals would all much prefer people use easily guessable and/or unsafely stored passwords. Those are both easier to discover, easier to brute force, and easier to intercept (specifically all you need to do is intercept a password once as it's definitionally not based on a single time exchange).