"Eh, I don't think I want a common programming model across CPUs and GPUs. They are fundamentally different machines, and I don't think it makes sense to try to lump them together."
Well, except for APU's, which I think is an unstated target for this discussion. Or on-board graphics chips.
In these cases, there is almost certainly extra work being done, regardless of your programming philosophy.