That's true in the general case, but it only happens after a difficulty drop. Until then, miners dropping out will raise the block time (again, just a tiny bit). For cryptocoins with more dynamic difficulty adjustment, this process happens a lot more quickly.