Good driver support is still pretty important... it's definitely one of the worst parts of Linux right now. I have an Nvidia card with Optimus in my laptop, and it's a total headache... I can't do stuff like connect a TV to my laptop and extend my display like I can with windows. The performance is not as good (which obviously matters for demanding games). The battery life is not as good. I just broke my X server when I tried installing some package (with apt) which wiped out some of my graphics libraries somehow... definitely not something your average joe can handle.

Overall, it is a huge pain in the ass. I oh-so-wish that graphics cards would "just work" like wireless cards now do.

But if you're on a desktop and get a well-supported card, you're right. It's not a big deal.