Yubikey webauthn over RDP works just fine for me.
I've spent way too long reconfiguring yubikeys to act as an OTP device so that they work over RDP.
(edit: reconfiguration is needed to capture the device ID/secret and get them registered to our MFA provider as an OTP token, I realize they do OTP OOB :) )