Yes, "Hey you're from India, what's it like it Japan" is almost hilariously insensitive. (Actually just tragic).
It's along the same lines as:
[Hears X is from country Y] "Oh do you know bob?"
It's along the same lines as:
[Hears X is from country Y] "Oh do you know bob?"
whether he is indian or he only knows about Indian culture, it is a stretch to say "so you must know about Japan!"
Which is an entirely reasonable question, because if X from country Y is at the same party as you, then there's a good chance he might know your friend Bob. It's a small world, they say.