I'm pretty sure if you asked a group of humans to colorize the lighthouse you would probably get the same result.

There is no way, from the greyscale, to know that the sky should be orange.