I just got an Apple TV and got crazy with the unintentional swipes and touches. Then I discovered you can turn that off, to click-only. It‘s much better now, maybe that helps you too.