1. You have lots of spy-data samples that reveal which physical key is pressed (perhaps they sound different) and the precise timing of those strikes, but you don't know what scrambled numbers were actually being shown. (And it's always the same code.)
2. The trick is that users take longer to press a number when it's displayed far away from its "normal" position, because they had to seek longer to find it.
3. This means you can infer the true numbers based on how quickly or slowly presses happen versus which physical key is struck.
For a simple example, assume a two-digit code where there are nine keys. If the fastest first press is always the top left corner, and the fastest second press is always the middle, we can guess the code is either 15 or a 75, depending on if the user is accustomed to phones or keyboard numpads.
One mitigation might be to get the user to enter digits at a consistent pace, by forcing a delay between showing the random layout versus accepting a button press. There would need to be some penalty for early presses, to keep lazy users from just tapping the desired button repeatedly until it became active.