But all other things are never equal. So this near-consensus is useless in practical terms. The arguments always start over which other things that are not equal should affect one's decisions, and which shouldn't.
For example, take Singer's classic analogy between saving a drowning child at the cost of ruining your clothes, and donating to charity to save a child in a faraway country at the cost of some luxuries. Obviously a lot of things are not "equal" between these two scenarios, but Singer is claiming that none of those things matter: if you would save a drowning child at the cost of ruining your clothes, you should forgo luxuries to donate to charity.
But Singer ignores one glaring thing that is not equal, and which, IMO, does affect the choice you should make: drowning children are rare. That's why we have a moral consensus that we should save them even if it costs us a ruined set of clothes. If there were drowning children all over the place, and everyone was always having to drop what they were doing and save them and ruin their clothes and have to buy new ones, all the time, we would not have the same consensus. Instead, we would be asking why there are so many drowning children around, and would be taking different actions to prevent them from being at risk of drowning in the first place.
But that's exactly the position we are in with respect to world poverty. Poverty is not rare; historically, it has been the normal condition of almost all human beings who have ever lived. Those of us who are fortunate enough not to be in poverty cannot afford to treat it as if it were a rare emergency like a drowning child, because it's not rare, it's common. (It's hard for people who live in the developed world to see this because what defines the "developed" world is precisely that poverty, real poverty, is not common. In the US, for example, a "poor" person typically has enough food--in fact, too much, since a major health issue for the poor in the US is obesity--some form of shelter, access to public transportation, a cell phone, and an internet connection. That is not "poverty" by any reasonable historical definition--it certainly isn't compared to the condition of people in, say, sub-Saharan Africa. There are people in the US who are in real poverty by this definition, but they are very rare, so it's hard for people in the US to realize how common real poverty is elsewhere.)
So if we are going to fix poverty, we can't do it by helping individual poor people, one at a time. We have to fix it by asking why it is so common in the first place.