In particular, there were anti-Muslim riots in Gujarat while he was in charge that he didn't do enough to prevent and/or condemn.
For the record, all my information has come from The Economist. I'm not Indian or an India expert.
But to be fair, he's not carried out anything stupid as Prime minister. His election was based on the promise of development, and he seems to be intent on that.
One cannot say the same for BJP unfortunately, Hindu nationalism has risen and is trying to make it's voice felt. I hope Modi is wise enough to shut that down.
Hindu Nationalism sounds more like a reaction to me than anything else.