That's pretty much the only advantage, other than liquidity for early employees, which most companies don't really seem to care about, since many of the early employees have left by the IPO, and the ones that are sticking around for their payout are probably just coasting anyway.
But the disadvantages are numerous. You have to tell your competitors about your profit and loss. You have to hire a bunch of auditors to make sure you follow all the SEC regulations. You can't do any long term bets because the market will skewer you and make it hard to hire. Your every action is up for public scrutiny. Your valuation is no longer flexible when you want to make acquisitions with stock.