This might not be the one you're looking for?
And if you want to use the bible as a justification for not feeding those who don't work... Matthew 6:36? Matthew 25:35? Isaiah 58:10?
Or let's go with the canoncial James 2:14-18, What good is it, my brothers, if someone says he has faith but does not have works? Can that faith save him? If a brother or sister is poorly clothed and lacking in daily food, and one of you says to them, “Go in peace, be warmed and filled,” without giving them the things needed for the body, what good is that? So also faith by itself, if it does not have works, is dead. But someone will say, “You have faith and I have works.” Show me your faith apart from your works, and I will show you my faith by my works.
There is context for it, and I don't consider it an indictment of Christianity, but pretending it's not in there doesn't help your credibility.
If you know of an interpretation of this including the context you refer to, I'd appreciate a pointer very much.
Verse 15 hints at how you might want to interpret these verses as compared to the references you cited: "Do not regard them as enemies, but warn them as believers." You see, a Christian has rather different conduct when he is dealing with a believer or a non-believer. If someone is your brother in Christ, it's ok to be a little hard on him. Same way you have the right to be a little harder on your own family members than I would. It's all supposed to be done in love, of course.
Hope that helps.