We, as Americans, are responsible for poor working conditions in large parts of the world. People recycling metals from computer parts (a topic HN users probably think about) develop neurological disorders, based on what we do.
Am I harming someone directly by not thinking about recycling more often? No, but I'm still part of a system that perpetuates these effects.
Your conflict and resentment stem from what some people before us have done. I have conflict over the families whose breadwinners die due to my electronic excess. But the answer isn't to deny the system I live in but to be more thoughtful.