Would you be interested in trying to adapt some of your approaches into a C++ GPGPU library (https://github.com/kylelutz/compute)?
https://github.com/vectorlang/vector/blob/master/rtlib/vecto...
Basically, the VectorArray class contains both the host array pointer and the device array pointer. There are also two boolean flags, h_dirty and d_dirty. When you modify array elements on the host, h_dirty is set to one. Then, when you run a kernel, the data is copied to the device if h_dirty is set, h_dirty is cleared, and d_dirty is set. When you try to read an array element again on the CPU, the data is copied from device to host if d_dirty is set, and d_dirty is then cleared.