They've been hosting the Formula 1 race for a few years now. An expensive, but probably effective way to show the world that you exist. I'd never heard of the country before F1.
Also their name as pronounced by our commentators kinda sounds like a joke country that doesn't actually exist, but everyone is in on the joke and letting them compete.
That's probably why Azerbaijan is going to stick in my memory forever. Even if I have no idea wherw it is on the map.