Try that while pointing two cameras at the scene.
My understanding it the rendering is done at higher quality when the background is the cameras field of view to save resources.. the rest is there just for ambient light.
Of course this takes a lot of pre planning, but those virtual set ups kinda always do.