One of the things we discussed to address this is to have the ability to:
a) turn off detail hallucination completely given the same bitstream;
and
b) store the median/maximum absolute error across the image
(b) should allow the user to determine whether the image is suitable for their use-case.
You could use an OCR encoder and cat that to the feature vector. Great work by the way!
Agreed! Alternatively, some semantic segmentation network could be used and a masked MSE loss. In this paper, we focussed on showing the crazy potential of GANs for compression - let's see what future work brings.
Is it possible to put the decoder into a feedback loop and search among multiple possible encodings that minimize the residual errors? Similar to trellis optimization in video codecs. http://akuvian.org/src/x264/trellis.txt
It would be possible - but by minimizing residual errors you end up in a similar regime as when minimizing MSE again, likely making reconstructions blurry!