As stated elsewhere, this can actually be a very frustrating process. I lost a good chunk of my long weekend trying to build TF from source for CUDA 8.0 / cuDNN 5.1. Generally speaking the culprit is that the CUDA installers for Linux are highly dependent on your kernel and gcc versions. This is a huge headache for people who want to stay up-to-date on their distro packages. CentOS has no problem because hardly anything changes, but you're essentially handcuffed to whatever version s of Ubuntu or Fedora were out when NVIDIA decided to start packaging up the next release. Bumping gcc to 5.4 in Ubuntu 16.04.1 broke the 16.04 installer, which relied on gcc 5.3.

Because GPU-accelerated learning is exciting, and most of the directions you find for setting it up don't work. (Judging from other replies, this post may be no different.)

This probably has something to do with the fact that GPUs are flaky and idiosyncratic, and all the software that uses them depends on black-box libraries handed down by Nvidia, who is completely shit at maintaining software.