The gangs there cared about drugs and making money. As long as you weren't there interfering with them making money, they didn't really care about you as a tourist. They might get a phone or pocket change from you but why waste time and money on just that? Bigger fish to fry.
In all the years my ex hungout and worked in the favela, not one bad thing happened to her. Which surprised the hell out of me. (She was a foreigner but spoke the language)