Allow me to attempt to clarify.
Let's say a civilization creates Von Neumann probes. Ignore whether such devices are even technically feasible. It sends these Von Neumann probes out into the universe in all directions.
The chances of any one of them ever encountering anything larger than a grain of dust before the heat death of the universe is practically zero. In almost all possible universes, all of the probes are swallowed up by the void. In almost all of the rest, one probe finds enough material to be able to replicate, and all of those are swallowed up into the void.
The scenario whereby a stable chain of exponential reproduction and growth via interstellar transfer occurs lasting long enough for a significant amount of stellar matter to be consumed and for an obvious trace to be left which just happens to be visible to us, is not the most likely scenario in a universe where such devices exist, it is still so unlikely that its feasibility borders on requiring sorcery. Because the universe is just that empty.
It's the sort of idea that only works on graph paper assuming a "perfectly spherical cow in a frictionless vacuum" kind of universe. The lack of a universe awash in Von Neumann probes is not a strong argument against the existence of life or advanced civilization in the universe, it is, itself, a strong argument against the feasibility of Von Neumann probes.