I think you are hiding a key assumption - that an AGI must be capable of performing optimally on a task with Archimedean measure. This is a very strong assumption - performing eps-close to optimal may certainly fall in the definition of AGI, and this might be achieved by a non-Archimedean approximation. Defining AGI as performing optimally on any set of tasks is problematic from a computational theory perspective in general - even with real reward signals.

Furthermore, infinite rewards are not compatible with human behavior. Humans never optimize for a single event at infinite expense w.r.t. other goals.