Yes, this is an established task known as image outpainting. What you’re describing is actually video outpainting, because you would want to use surrounding frames for context.
Also, im sure I saw a very nice video on this, with a car driving through the landscape, using previous frames as well. Can't find the link though :-\