> The math will work out.

When you're dealing with GPUs you're never dealing with just the math. Various vendors make shortcuts in the name of performance so there's a good chance your raycast code doesn't match 100%.

Seen this a lot with "golden image" tests, one vendor's rendered scene has bunch of inconsistencies with another vendor's(or different driver version) causing sporadic test failures.

That's assuming you have raycasting rendering(most game engines don't) and/or hoping that the way you draw your ray cast scene matches up pixel perfect with the GPU(it won't).

Generally rendering Geo + Fill is cheap, it's the complex shaders and post that tend to drag down your GPU. Heck, if you're doing GPU occlusion culling you're basically doing the same idea but for rendering whole objects.

You can also do this every 1/10 frames or so to keep overhead low. UE3 and a bunch of game editors use this technique pretty extensively.

Many game render pipelines will draw the scene a dozen times; here's an article about the pipeline from GTA V, for example: http://www.adriancourreges.com/blog/2015/11/02/gta-v-graphic...

Adding one more draw call for mouse interaction, especially one that requires no shading at all, is a pretty minimal additional cost.

It's "a lot of work" for the machine, but very little work for the programmer. It is assumed that rendering the scene is already fast enough to be happily interactive for the UI. Mouse picking is a human timeline interaction. And, it's likely that the scene is static during the pick (do you want to pick a tiny, moving atom?). So, if the pick is as fast or faster than a single frame of spinning the model, it should be unnoticeably fast for the user.

you only need to draw one pixel, the one under the mouse. cull to only objects under that pixel, set your frustum, viewport, and scissor for that one pixel and I'm guessing drawing is faster than walking triangles in the gpu