So am I getting this wrong? Isn't this exactly how a float described in bits? If you had infinite memory then this would possibly be implemented with floats.
No, floats have explicit exponent and fraction parts. For the "finite-ish" parts of the knuth construction, the maximum size depends on the "generation" of the construct. A "knuth"-string float would have a variable length, depending on the amount of precision necessary to express the value you seek.