Just say you're busy / in the middle of something. Decline their offer enough and they'll eventually stop asking you. No hard feelings (I say this as the guy who often invites others out).
This is what I recommend too. If someone declines me politely with a reason like "Sorry I'm busy today, maybe next time." I'll probably wait a week or so and invite them again if I want to eat lunch with them. If they reply with something shorter like "I got something to do, sorry," with a less kind intonation I probably wouldn't invite them again.