I think you misunderstand what quantiletransformer does; it just transforms the distribution of the data be be more normal, it's not a binning technique. To many quantiles will just result in excess noise, not loss of information.
If this can be turned into mechanistic theory, then implement a version of it that can solve CartPole (as an easy first test). Otherwise, it's just hand waving and conjecture.
I think using the expected logarithm of the number of solutions (entropy) makes more sense from an information theory perspective. I wonder if it works better in practice.
I think the statistical aspect is an important part of the algorithm; it's trying to match the statistics of the input data. I guess you can think of it as an approximate sampler for Markov Random Fields.
The naive dynamic programming solution doesn't work because it's possible to 'backtrack'. But I used Bellman-Ford which is almost as simple and very fast.