Does xv fit the bill?
Really minimalistic but at the same time quite powerful under the hood. All normal operations have quick vim-inspired shortcuts (and scroll to zoom) and if you want to get advanced you can trigger scripts to do some heavy-lifting.
Not for everyone but when it fits it is brilliant.
If sxiv isn't your thing and you want something minimalistic, maybe have a look at feh (don't know if it has scroll to zoom).
I also tried feh. It simply hangs when I invoke it with a png file over a remote X11 connection.
If you are starting to wonder whether there is something wrong with my setup: I can run other applications like xterm, xeyes, gimp, just fine over a remote X11 connection.