This really sucks. This is supporting a cipher with questionable design decisions and the authors have been combative when confronted with technical questions.
I don't know if it's backdoored but I certainly don't want it supported by the mainline kernel as it would aid acceptance even though the ISO rejectes it.
You can disable this in the kernel by unsetting CONFIG_CRYPTO_SPECK.
Is there some particular reason why you'd want to? "Controversial" it may be, backdoored even, but just having it isn't going to compromise your machine.