Yup, you can change it in about:config under security.dialog_enable_delay. But here's a blog post from 2004 showing how easy it is to exploit a user if you turn it off. http://www.squarefree.com/2004/07/01/race-conditions-in-secu...
I question the utility of having this delay behaviour by default, worst that can happen is you accidentally start a download AFAIK, which you can just quickly delete/cancel afterwards.
And, there's browser.download.manager.alertOnEXEOpen too.
I much prefer Chrome's behaviour of starting the download and you instantly can see the filename and size in the bottom as soon as it starts and can then choose to either open it or show it in the file manager.
There's a potential DLL hijacking attack:
1. Accidental download of kernel32.dll
2. Ignore, who cares, do something else
3. Download actually_wanted_installer.exe, which finds malicious kernel32.dll in current directory instead of SYSTEM32
There are a few DLLs whitelisted for protection for this, but drive-by downloads are a real attack vector.
Also in the past, exploitable bugs in the thumbnailer if you browse to your Downloads directory (e.g. CVE-2017-11421 and others on Linux).
That said, starting the download and immediately getting it out of the user's face, requiring interaction to open them is a very good solution. It leads to some problems on Windows, but then, what doesn't?