Because mining is dependent on the rate at which information can propagate across the network. If a mining pool is able to hear about the next block 1s sooner than another pool, it has a significant advantage when blocks only take 12s to mine. If blocks take 10min to mine, that 1s is less of an advantage.