I don't get how an app like this makes a difference. Isn't an image viewer just reading from disk, using whatever image decoding library that's standard, and copying to screen?

Of course you can get any of the above arbitrarily wrong, but supposing you don't, you have performance baseline. You can get cute and prefetch a few images or maybe downsample them and store the thumbnails somewhere, but that's something users might be unhappy about.

I don't know, can you handroll a jpeg implementation that can beat libjpeg?

Personally the image viewer I'm happiest with has been Quick Look in macOS, largely down to performance.