1,103 karma · joined March 22, 2009
Is this 7 nm node really older than raspberry pi 5's 16 nm node?
EDIT: Hm, or maybe ts_zip uses just the token probabilities directly. I thought it was slightly more efficient about it.
"The language model predicts the probabilities of the next token. An arithmetic coder then encodes the next token according to the probabilities."
It's typically not done at the pixel level, but at the "latent space" level of e.g. a VAE. The image generation is done in this space, which has fewer outputs than the pixels of the final image, and then converted to the pixels using the VAE.
Ah, yes, _preferably_ some random string that is hard to block rather than the descriptive ones.
Not to say a 0-th order entropy coder could not be useful for certain applications if it was e.g. faster than, say, a range coder while maintaining similar ratios. But I doubt that is possible with this method.
It would have to assume there's _exactly_ N ones and N zeros in the bit string. Start with those counts:
remaining ones = N;
remaining zeros = N;
It would encode bit for bit. After every zero, it decrements the remaining number of zeros. After every one, it decrements the remaining number of ones. The probability of the next bit being zero or one is derived from the known remaining number of each, P(one) = (remaining ones / (remaining ones + remaining zeros)). When one of the counts reaches zero, the probability of the other becomes 100% and no more information need to be encoded.
Does the linked code do that? No.
I don't have a ready-made example because this is not an assumption about the input that is useful in the majority of cases. The linked example can encode _any sequence_, not just ones that have an equal number of ones and zeros.
If you spent time implementing this scheme you would learn a lot about this field.
Not at all. The best compressors learn to predict the frequencies as they go, they do not compute them up front and store them in the output.