Browsers (or site owners) could probably do a better job of differentiating links with download attributes. Maybe on mouse over show a little download cursor or something.
Yet, this required two mouse clicks in the "worst case" that the default selection didn't fit.
I think the best solution is: if no download attribute set and Content-Disposition is not "attachment", then render in the browser. If either condition is met, open a popup with large buttons immediately executing the action: download to default folder, download to temp (which will be deleted upon next browser start), download to specific folder, render in browser (if there's a registered handler plugin for this mimetype). Maybe add a checkbox "Open after download has finished", and maybe also add one "Try multi-stream download" (because many sites still limit single-stream bandwidth). Even more maybe, for video files add a "Preview with VLC" and fetch the required parts (header and final part of video) first, so I can stream and keep the file for later usage.
Just go to Options->General->Downloads, tick Always ask me where to download files
Internet Explorer also works like this practically eliminating step 1,2 and 4 from your scenario above.
However, it seems like a good candidate for a browser plugin for those who have fast connections and don't mind.
a[download]:after {
/* add an icon or something */
}