What happens if you just brute-force and re-render everything when there's a change? I'm all for optimizing but is this really required these days? Even considering laptops and mobile power requirements, rendering off-screen and flipping is lightning fast.
Layout flows are a different matter and can be expensive to compute.