Rugby has video replays, and these are only used when there is a potential try. It's easy to limit the video ref to perhaps situations where there is a goal scored, and maybe when there is a penalty is at stake.