I don’t think that’s it, no one told me about that when I complained about that delay on the Mac Power Users forums, and I can’t find anything about it (such as removing that delay) online right now.
Which Mac(s) are you experiencing that on, and since which OS version?