I agree. I've gotten this often and it's never been a problem. Why? Because I always point out something that's both true and not an issue. Just stay focused on your strengths.
For example, most of my career involves unix systems internals, databases and networking. Stuff generally involving valuable state. I've done hardly any professional graphics or gui work. So when I get this question, I generally just point that out. Ephemeral stuff like video games isn't my line of work, and I'm not very experienced at UI stuff.
Nobody who's ever care enough to interview me has been unhappy with that answer.