Is anyone aware of a formalization of the idea that to get “symbols” out of fuzzy probability distributions one needs distributions whose value goes exactly to zero over some regions of the domain? I.e. Gaussian mixtures won’t cut it. And they will need very high Fourier frequencies.
I have the gut feeling that until a model allows for a small probability that 2x3 is 7, there will always be hallucinations. Probabilities need to be clamped to zero to emulate symbolic behaviour.