I don't understand the connection you're making between SfM (Structure from Motion) and surface shading.
I might be misunderstanding what you're trying to say. Could you elaborate?
I might be misunderstanding what you're trying to say. Could you elaborate?
However, non-Lambertian points move non linearly in viewing space (eg a specular point depends on the viewer pose).
So, automatically, their positions in space will be false, and you'll have floating points.
Gaussian 'splats' may have the potential to render non-Lambertian stuff using for example the spherical harmonics (even if I don't think the viewer use them if I'm not mistaken). But, capturing non-Lambertian points is very difficult and an open research problem.