Moving from one German country to another is not immigration. It's probably less of a change than moving from Nebraska to Texas.
The xenophobic quip just confirms why it would have been difficult for you to live in more diverse places.
As for their grandparents, there is nothing easy about growing up in the epicenter of massive global conflict, particularly when you are on the losing side. I'm thankful I did not have to experience war like my grandparents did, and they were on the "winning" side.