Huh. Whenever I've used it, things have all been on the same domain so this doesn't come up - good to know.
(Correction: per spec, the domain does enter the picture in that cross-domain <a download> is not supposed to work at all without Content-Disposition response header; that’s slightly different matter to what’s discussed.)
Something to watch out for.