Hacker News Reader

The reason 2D grids work well is because you can split tensor axes along whatever dim you want across PEs (batch, sequence, hidden, etc.). And your per PE code is relatively simple. It's of the type : get data from neighbor, compute my subtensor, send to neighbor. And you can implement broadcasts and reductions along any axis.

View on HN

You have to route around disabled subchips though.

My guess is it is more like the opposite. They distribute spares evenly along every array in every dimension, so that there are always the same number of selected and unselected cores in each core-set.

With flexible addressing and bus-linking, along any set-line. So configuration happens the same way, with the same efficiency, regardless of whether any of the unselected elements are failed or not. As long as failures don't exceed the unselected-elements per element-array-set number.

And that number of unselected cores per element-array-set, is chosen to put wafer yields where they want them.

Trading a generous fixed wafer under-utilization for high yields, uniform configuration and processing efficiency, despite variable element failures.

Reply on news.ycombinator.com