An increase in blocksize means an increase in propagation delay. Too much delay and miners who are closest to the most recent block get a head start mining the next block. Mining is suppose to be a lottery, not a race.

Research done on the matter shows that for each kb above the current 1mb block, 80ms of delay propagation is added.