Has any botnet ever been written by a good cryptographer? I remember Vern Paxson getting a whole paper out of supposedly-skillful botnet authors not even being able to generate random numbers securely.
I don't know about its cryptography specifically, but Stuxnet, for example, is a botnet regarded as a game-changer precisely because it was such a professional hit.
BTW, I'm trying to see if there's any interest in a collaboration on the cryptanalysis of this custom encryption system. Anyone with an interest please ping me. It may turn out to be nothing.
(2) Do you have the source of it? Do you have a non-obfuscated version?
(3) Even if the answer to both questions in (2) is "yes", I still doubt any serious cryptographers would take a look at it. They (mostly) do things to write papers, and you don't get a paper out of "we broke a really weak encryption algorithm in a botnet". The chances that the paper is instead "this botnet has a reasonable encryption algorithm" are so slim, they won't consider it.
(4) That said, there are a number of non-serious cryptographers who would find it an interesting challenge, but if they can't break it, it still doesn't mean it's any good.
(2) No, I haven't heard of anyone getting the source. There are 4.5M PCs with an obfuscated binary, so that should be obtainable. We can de-obfuscate it ourselves (it obviously has to load into memory at some point), or we can also ask other researchers who have already done this. The latter is likely to be successful according to 'who' and 'how serious' we have interested.
(3) You never know and it never hurts to ask. Don't forget, they also often have students looking for projects. Anti-botnet ops is an active research topic in data security.
(4) Yep. I doubt anyone would consider using it for anything else.
There's nothing preventing a scenario where someone with nefarious intent and a large pile of money convinces a skillful cryptographer into building a cipher robust enough be resistant to cryptanalysis.
(I'm not your downvoter by the way)
Still you could easily end up with something much less secure than any of them used independently. For example, read about the "meet-in-the-middle" attack.
If you somehow convince yourself that the NSA can break many well-known encryption algorithms and that it's willing to risk exposing this ability to shut down your botnet (which is the counterfactual I'm working off in this thread), this may give you some peace of mind: Blowfish and AES are well-tested designs by separate teams and SALSA is a very simple, very different design that you can easily implement yourself and that's pretty resistant to side-channel attacks. It seems likely that any agency that can break all three can break anything you can come up with.
We obviously use three independent keys. The security of the composition is then at least as good as the security of the best of these algorithms.
That's not obvious at all to an implementer. For example, there are NIST standards for 1-, 2-, and 3-key variations on triple-DES. http://en.wikipedia.org/wiki/Triple_DES#Keying_options