More challenging, yes. Riskier, no.
I don't like that Javascript doesn't have native support for big integers (like Python does), or that it stores numbers as floating points in a 52-bit mantissa, but I fail to see why this makes developing crypto code riskier.
> Also, it seems to me like you are at least forgetting timing and possibly other side channels.
Well, I ain't. When you don't control the instructions being executed by the CPU you may have the risks of security-sensitive information leaks. This applies not only to Javascript, but also to all scripting languages. I could say that it also applies to Java, if the methods in its BigInteger class aren't fixed-timing.
In other words using Javascript doesn't make the problem any worse. If you disagree, you're invited to take a look at End-To-End, find a side-channel leak and write an exploit for it. You could earn serious cold cash with that finding.
> I just can't see any actual scenario where that helps, mostly because it seems to me that the cipher text usually would be a plain-text equivalent, so it doesn't really matter to the attacker whether they have the plain text or the cipher text.
I described where it helps in my article.
Re your last point: if doing SSH in a browser isn't crypto I don't know what could be. Is the only thing you consider Javascript crypto encrypted webmail? That's your problem then. You know one wrong use case, and you refuse to admit that there are other legitimate ones.
Edit: remove a few unnecessary sentences.
Edit: some people don't like Javascript crypto so much that they downvote me without saying anything.