If your user clicks the link with download attribute and your server (or reverse proxy, or CDN, etc.) issues an error page, user agent downloads the HTML of that page and saves it under the filename you provided. Domain does not enter the picture.
(Correction: per spec, the domain does enter the picture in that cross-domain <a download> is not supposed to work at all without Content-Disposition response header; that’s slightly different matter to what’s discussed.)