Under this perfect assumption you end up having a problem with the red shift caused by the expansion of the universe (which I didn't account for above) that lengthens the wavelength of the light you want to see, requiring an even larger lens.
On the lens side of things, as far as I'm aware (not a physicist, math PhD student who is just generally into this sort of thing) there isn't anything fundamental preventing you from collecting this light and building the lens, but from our current understanding of materials science I'm fairly confident it's currently impossible to construct a structure that will stay together that large.
That being said there may be ways around this problem, like I said not a physicist or engineer, but you are right that from an information theoretical perspective if you ignore dust and other things in the way then yes all the information is still there.