The problem with delegating tasks to GPGPU's is, that GPGPU cores can do only a very limited subset of what a CPU can do.
First of all, each GPGPU core has only a small amount of fast local memory available. In most applications, tasks need to be able to access databases, large in memory datastructures and what not. The GPGPU architecture is not designed for that. When data outside the core's local memory is needed, data transfer to and from individual cores is costly and when too much of that has to happen, the benefit of the many GPGPU cores rapidly disappears.
Second, a GPGPU core does not have access to other components, e.g. IO controllers. So it can't directly read from disk or write to the network. This again limits the types of work that a GPGPU core can do by a lot. For all network and disk access it needs to communicate with the CPU, and again, this communication is slow.
And finally there's a difficulty of another nature. OpenCL or CUDA does not integrate nicely with higher level languages such as Java. The article mentions Aparapi, which is one solution that tries to deal with this by compiling Java code to OpenCL. The trouble with this is that Java code is living in another world as OpenCL; code that works well for Java might perform horribly on the GPGPU, because the GPGPU's architecture with its memory locality is so different. So to write Java code that will be compiled to well performing OpenCL, you have to code it with OpenCL in mind, and you have to be aware of the manner in which Aparapi will convert your code to OpenCL. This defeats the purpose of Aparapi; it might very well be easier to just write your code in OpenCL so at least you have a clear view what is going on.