> But they admit that if you can secure the transmission of the entire contents of the page (HTML, JS, everything), then your JS crypto should be OK
Nope. The server operator can still serve you (perfectly secured over an SSL channel) backdoored javascript crypto code.