Would it not be possible to do the image processing on the original image to create several pixel masks which then draw in the high resolution image data?

For instance, you take the source image at resolution n*n and do all processing at n/2 or n/4. Then at the moment you are going to composite the finished image you instead draw "pixels" from a source image which contains X,Y not RGB. Then you upscale the output image x2 (or x4) and replace each X,Y index with the relevant pixel from the source image (adding X%2,Y%2 or X%4 to the indexes to return to source resolution).