In theory "it is complicated" because an instruction might have N inputs and M outputs and the latency matrix might have different values for every element in that N x M matrix, but in practice instructions have 1 output and the inputs are usually symmetric with regards to latency so one figure is enough for most. It's worth calling out the exceptions though, and uops.info is supposed to do that I think (an example would be great).