Maybe that's not the right solution? Switching to qwerty and back everytime you play the game is a pain, you should only have to configure your controls once.
And switching to a layout you don't normally use absolutely sucks for e.g. in-game chat.
It's not the right solution, it's a sad workaround.
Things get harder still if you need to rebind keys because of the physical arrangement of the keys on your keyboard. In this case, you might have to create a custom layout for each game.. screw that.
Windows does this. Its horrible. It also doesn't remember what window has what layout between sessions, so you have to manually set the layout each time. If the game is a fullscreen game its even more effort because if you alt-tab out, it is no longer the focused window (and therefore changing the layout does NOT apply to the game.. frustrating).