The concept makes sense but I'd never thought of that before.
Arithmetic coding allows fractional probability distributions to be represented. It's usually at least 10% more efficient than Huffman coding.
It's other parts of JPEG compression which lead to quality loss, and those parts aren't recompressed.
What they're saying is that you can take a JPEG compressed image, decompress to raw pixels, and then recompress with JPEG more efficiently (if you're careful, no specifics on how this is done), and you save space.
That's why they mention that they're doing it very carefully, because you've got to make sure that when you decompress the new optimized image that it pixel for pixel matches the original decompressed image.
As far as I can tell, 'raw pixels' is not quite accurate. Both PackJPG and Lepton seem to decompress as far as frequency-domain coefficients, but no further. That means they don't repeat the lossy stages in JPEG encoding - the transformation of pixel data to the frequency domain, and the discarding of high frequency infomation. It looks like PackJPG and Lepton do some interesting tricks with the coefficients to essentially make them more amenable to the following lossless compression. Both PackJPG and Lepton have their own file format, so they're definitely not outputting standard JPEG images.
Here's the Lepton page: https://blogs.dropbox.com/tech/2016/07/lepton-image-compress...
And for more detail, here's the paper that PackJPG is based on:
http://www.elektronik.htw-aalen.de/packjpg/_notes/PCS2007_PJ...
Besides removing information from the file that doesn't affect the rendered image (like EXIF data), lossless recompressors typically replace the huffman coding of DCT coefficients with a more efficient arithmetic coder. So you don't start over from raw pixels, but you replace the type of compression used with a more modern and efficient algorithm. That means ordinary software can't read the JPEG (since you've essentially created a new format) but you can just decompress into standard JPEG whenever someone wants to look at the image.
You can do this if the goal is pixel perfect accuracy, but Flickr can’t do this since they have “a long-standing commitment to keeping uploaded images byte-for-byte intact”…
Lepton (one of the examples they mentioned) losslessly compresses a JPEG to Lepton format, then losslessly decompresses it back to JPEG. The pixels are never decompressed and in fact a JPEG decompressed from Lepton is bit-exact the same as the original. I've tested and verified this on several million images.