The downside is that you're normally a small fish in a big pond as opposed to a big fish in a small pond somewhere else. The perks seem like the pay, the scale at which you work generally, and the quality of your coworkers is higher than normal.

If you like hacking on features, not having to do robust A/B testing or walk through tons of leadership / management hoops, and having a wider role in the development of something, then a smaller company or startup would be better.

TLDR: as with any thing in life, it depends on the person, and what makes you happy at work.