Mouse interactions have (at least) 5 different events: hover, down, up, click and double click. The interactions you describe all happen “onclick”, which requires a down and up event to happen consecutively.
I get your point, that small delays add up, but mouse events aren’t a great example, IMO. Each of the events have a purpose and you want to make sure that a button is only activated with an onclick, not just an ondown.