I’m OK with “Save as” (“Download linked file”, etc.) behaving as it does: it is more or less intuitively clear that with it we sort of “cheat” and use browser’s semi-hidden feature to interact with a website in ways not necessarily intended by its engineers. With the download attribute, however, any confusing outcome falls squarely on me, the developer, so I’d rather stick with plain links and response headers as a mechanism I have more control over.
There may be some workarounds to make download behave adequately in case of an error, such as having your server behave in some way that causes browser’s fetch sequence to fail with a low-level error. Unless I’m missing something, it seems like proper network error is the only condition under which download can actually fail, and I don’t think a typical web framework setup is equipped to intentionally cause that to happen. I doubt it’s worth the bother—if you have control over the server then why not drop the attribute and just return the appropriate headers.