Its worth thinking about the idea that we are actually made of highly sophisticated self-replicating nanotech machines, and although they've been pretty successful for the last few billion years, they're still nowhere near good enough to achieve exponential growth that consumes all matter.

Maybe someone somewhere has already explored the theory that nanotech that could re-shape all matter around it is probably impossible due to the energy challenges inherent in re-shaping arbitrary atoms.