> you can't write an assertion like ("does this image look like a downscale of this other image"

You can though. Maybe not exactly what you are thinking but there are some pretty good image similarity algorithms like dhash out there that do stuff like this. Mostly they get used to check for duplicate uploads, copyright materials, or "does this look like porn" style filters

This isn't something that needs image models to accomplish really.