So 100 ms is on about human eye-hand reaction time. Not that humans think in discrete chunks, but 100-400ms is like 1-4 extra responses, which makes game play quite different.
Also, I know people will say "Game Mode!" but obviously, as a dev you can't just assume your end users will do that.