The problem is that not everyone has the same neighbors. If everyone gave what they could to their neighbors, then people in poorer neighborhoods would still be worse off.
Maybe you are speaking more in the abstract, but I think it is important to also donate to groups who _aren’t_ in your local topology, or else you will still have wealth/benefit disparity.