I find that the only 'drawback' is that it is not sexy. When people have talks about what they are going to buy or how much they have made buying and selling bitcoins last week you can't really take part of the conversation.
Also, even with my passive strategy, I still have fun talks of my investments with friends. It's more talks about general trends and what companies I'm looking to hold for a long time.
Globally, the unsexyness of passive index investing is the best thing about it. A huge percentage of the angel / seed / FFF capital in the world comes from bored people who don't want to talk about their Vanguard Target Retirement 2050 investment and so put 10% of their portfolio into crazy startups that will sound impressive to their friends.
Consequently, for the posters on this site, the boringness of passive investing is a huge creator of jobs and opportunities for us!