opencl appears to be supported by TF now. Regarding FPGA, some startups such as teradeep are on it, whether it is using Caffe or TF.