The only downside I can see are restaurants/shops that work with those office employees. Some might not survive. In fact, even if offices are converted 100% into housing, the turnover of people is gonna be smaller. Anyway, rent should also decrease for those after a bit.