- Obtain a large dataset of observations related to a phenomenon of interest (eg English text, bioimaging data, economic reports, car-mounted camera recordings, etc)
- Develop a theory of the phenomenon, or revise a previous theory
- Build the theory into a lossless data compression program
- Score the theory by invoking the compressor on the dataset, taking into account the size of the compressor itself; lower codelength is better.
- Adopt or discard the theory as appropriate and return to step #2.
I believe that this is a valid variant of the traditional scientific method, in which data compression takes the place of experimental prediction. In Graham's terms, this idea is a small delta of novelty, but since it refers to a predecessor idea of tremendous significance and generality, it could be very important. The book is available here:
https://arxiv.org/abs/1104.5466
I've spent a few years pursuing this methodology in the domain of English text. The result is a sentence parser, which works quite well, but was built completely without the use of hand-labelled training data (eg the Penn Treebank). You can check out the parser here:
http://ozoraresearch.com/crm/public/parseview/UserParseView....
I'm always happy to talk to people about these ideas, ping me at daniel dot burfoot at gmail if you're interested.