I tried to look up who first said something along the lines of "A supercomuter is a tool for turning a compute-bound problem into an I/O-bound problem" and all I could find was an ArsTechnica article referencing it as an old joke. I thought it was an old joke by someone of the stature of Cray or Amdahl, but maybe that's apocryphal.
In any case, our network simulations attempted to perform a min-cut (using number of packets as the cost metric) to partition simulation nodes among threads, run one thread per core, and pin a thread to each core. With a network simulation, though, as you can imagine, the problem was usually bound by memory bandwidth.
Also, the client was a defense contractor, and the 400k+ USD box was an artefact of "use it or lose it" budgeting. If they didn't spend all of their budget by December 31st, the group's budget would be reduced the following year. In their case, the high cost of the box was a feature.