I imagine they have a basic design in verilog with various tunable parameters (memory size, clock speed, how many instructions to issue at once).

They also have a way to run that hardware in a simulator and see how quickly it could train some network.

The ML optimization problem is to come up with a bunch of constants which performs well, but also compiles into a manufacturable chip. Clearly setting the clock speed to 9999Ghz isn't that...