Fun fact: did you know you can inline an image without base64 encoding it? Use svg you can inline the markup using a data url without base64 encoding. This is how I do my favicons at simpatico[0]:

  <link id="favicon" rel="icon" type="image/svg+xml" href="data:image/svg+xml,
    <svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 1 1'>
        <rect width='1' height='1' fill='DodgerBlue' />
    </svg>"
  >

What's nice is that this opens up the possibility to generate dynamic favicons really easily using only string interpolation.

0 - https://simpatico.io/svg.md

The error page just says 404, the image isn't critical.

And if the server has broken so badly it can't send out that image, then I have much bigger worries than an ugly 404 page.