For those looking to do it on the jvm, I have a prepacked scientific computing framework that might be interesting:
This is a generic wrapper with ndarrays for cuda and normal blas operations. Deeplearning4j (my deeplearning project) also has support for GPUs (it's built on nd4j)
Stable version coming soon =D
For those of you in python land, I would look in to theano