I'm not sure if this is what they're doing with this project, but in the past they've used separate chips for training and running the NN https://en.wikichip.org/wiki/nervana/nnp

I'd imagine that the actual architecture consists of a large number of neuron-like components that can be interconnected in various ways, allowing different application to run, or maybe even learn. This is a lot like how an FPGA works!