Yep its clear that the NFLT only apply if we consider all possible environments equally.

In practice, we are indeed not interested in every imaginable environments, only in "realistic" ones.

It was not clear for me if the paper addressed such concerns for AGI, e.g. when writing:

To achieve good rewards across the universe of all environments, such an AI would need to have (or appear to have) creativity (for those environments intended to reward creativity), pattern-matching skills (for those environments intended to reward pattern-matching), ability to adapt and learn (for those environments which do not explicitly advertise what things they are intended to reward, or whose goals change over time), etc.

But like I said, I only skimmed it.

In general (not talking about the paper there), I have the impression that this is something that may be missed (sometimes even by researchers working in the domain), and I agree very much to your point!

This is why I think the NFLT gives us an interesting theoretical insight here:

Making a "General" AI is not actually about creating an approach that is able to learn efficiently about any type of environment.