Not true, you can, but you have to approve them from the system preferences panel.
I ended up having to pay $100/yr for a developer account to be able to sign and notarize the binaries. They can now open them like usual, with just a warning that it was downloaded from the internet. But without the signature they get an uncircumventable gatekeeper error.
ETA: See https://wiki.freepascal.org/Code_Signing_for_macOS#Big_Sur_a...
We use cross-compiling infrastructure on linux/docker for the app I maintain, which don't generate these signatures. This causes the resulting binaries to be rejected on recent apple devices (although they used to work just fine with user override on intel-based macs).