The definition of a supercomputer has changed over the years, but currently what defines a supercomputer is its interconnect. Most of them have InfiniBand or similar with <1us node-to-node latency. It's necessary for the type of workloads typically run on supercomputers, e.g. scientific applications. The fastest you can get on EC2 right now is ~15us, and that requires specialized hardware and software.

Back in the day, Geoffrey Fox defined a Supercomputer as a compute system that costs $20M or more. I think that definition still holds to some extent.