Intuitively I've always been a bit skeptical of quantization. Wouldn't there be a tiny loss in precision by doing this type of quantization? I could imagine the error function increasing by utilizing these types of techniques.
TBH I think they won't get anywhere. Doing good game engine work... why that would translate to AGI?
It discusses whether it is possible to evolve AGi using... computer game engine! And that is John's bread and butter.
> It is interesting that things still train even when various parts are pretty wrong — as long as the sign is right most of the time, progress is often made.
https://forums.fast.ai/t/how-to-do-reproducible-models-and-u...