Except that you need to ultimately compute the backwards light cone of every possible event that is part of a causal chain. I.e. this caused that, so we have to compute why this caused that, and what caused those things. Sure, if an event is never observed, i.e. it never influences any event which a consciousness perceives, either directly or indirectly, then you can leave it out. Problem is, we keep coming up with new ways of observing phenomena. We keep doing particle experiments and building telescopes, so the load on the computation of the universe is getting pretty big!