The first thought I had, after looking at some of the examples, is that it should be fairly trivial to train or finetune the compression network to hallucinate missing detail with "BPG-style" or "JPG-style" smoothing artifacts by feeding compressed images as "real" samples to the adversarial network during training/finetuning.
I wonder if doing that would make it possible to achieve even greater compression with loss of detail that is indistinguishable from traditional compression methods and therefore still acceptable to the human eye.
It definitely is a valid approach but the limitation is that if the network needs some texture-specific information that cannot be extracted from the decoded pixels, it can't really do much.
There were approaches where such information was also sent on the side, which yielded better results, of course.
The field is wide-open and each approach has its own challenges (e.g., you may need to train one network for quantization level for example if you're going to do restoration).
Forgive me, is that in encoding or decoding? And size of library required to decode it? ( Edit: Looks like answered below )
It was only an hour ago I was looking at some pictures encoded with VVC / H.266 reference encoder, it was better than BPG / HEVC, but this HiFiC still beats it by quite a bit.
This whole thing blows up my mind on limitation I thought we had on image compression.
May be I should take back my words on JPEG XL is the future. This seems revolutionary.
Thanks for your kind comment!
Of course, we'd be happy to add any additional images from other codecs if they're available.
Admittedly, you're not likely to learn much from this that is useful for your research, but most of the interest from people clicking on this is probably wanting to see the latest developments in image compression.
The adversarial loss "learns" what is a compressed image and tries to make the decoder go away from such outputs.
The perceptual (LPIPS) is not so sensitive to pure noise and allows for it, but is sensitive to texture details.
MSE tries to get the rough shape right.
We also asked users in a study to tell us which images they preferred when having access to the original. Most prefer the added details even if they're not exactly right.
Could you expand on this point
On the other hand, in our paper we show that some generated detail can help even at higher bitrates.
EDIT: You mention CLIC2020 in your paper so I'm a bit more confused.
Another question would be,what are your thoughts on the future of learned image compression and ideas for future work?
One silly question, the red door example image... why is the red so saturated in all the compressed versions vs the original?
edit: Ah seems to be some kind of display issue in Firefox. When saving the images and comparing the saturation level is roughly equal.
Our training set consists of a large set of high-resolution images collected from theInternet, which we downscale to a random size between 500 and 1000 pixels, and then randomlycrop to256×256. We evaluate on three diverse benchmark datasets collected independently ofour training set to demonstrate that our method generalizes beyond the training distribution: thewidely usedKodak[22] dataset (24 images), as well as theCLIC2020[43] testset (428 images), andDIV2K[2] validation set (100 images).