Language Modeling Is Compression
huggingface.co
huggingface.co
PAC learnability is directly related to compressibility, with only ergotic systems being fully learnable as would be required to happen for lossless compression.
Perhaps review of Ergotic theory and PAC learnability would help convince reviewed more.
Comparing gzip, which was designed with time and space complexity constraints as StoA compared to a training transformers was called out on the author's previous work.
https://openreview.net/forum?id=Hygi7xStvS
If someone can educate me on what I am missing it would be appreciated