Here's a Twitter thread that explains what this paper is about: https://twitter.com/yongyuanxi/status/1354478763065528320?
As far as I understand as a complete beginner in 3D rendering:
Regular 3d models/meshes are just a collection of points/triangles in a 3d plane, along with some additional data like normal orientations. For rendering, the mesh is constructed and then rendered with some perspective/distance/occlusion settings etc. for each pixel painted on screen.
This format isn't a discrete collection of points but some data set where every point queried during 2d painting returns a distance (if it exists) based on some function or ML algorithm, basically returning a point cloud?
float distance_to_surface_of_model(x, y, z);
For every point in 3d space you can get the distance to the surface. So, if you have a camera you can shoot a ray into the scene and pick some points on it to ask if it is inside the model. if so you paint that pixel with the surface material of the model. Then it's just a matter of picking a good way of sampling points along the camera ray. usually this distance function is some basic math (distance to sphere or so). But here they take a 3d mesh and create a neural network that can answer the question of distance to it (since meshes are very slow at that).
By contrast, SDFs require you to do lots of expensive lookups as you slowly march along a ray until you happen to bump into a surface.
The distance to the surface is independent of direction, it is only a function of position (as all SDFs are).
The depth map is the distance from ray origin to intersection only, not how close that ray came to the surface at any point along it's segment.
Are there applications you have in mind where the distance to the nearest surface is actually useful, regardless of the surface representation? As in, when would knowing the distance to the nearest surface be helpful when you're working with meshes? It seems to me you only need to know the closest distance to a surface when you have an SDF and are trying to compute an intersection along a ray.
The nearest sort of application of this kind that I can think of is Walk on Spheres (cool recent paper https://www.cs.cmu.edu/~kmcrane/Projects/MonteCarloGeometryP...), but in that case the authors do a closest point query, not a distance to the nearest surface. There's a subtle difference here because you actually need the closest point (not just the distance) so that you can look up a boundary condition. It's not clear to me how you could make use of an SDF in this setting, although maybe there's an interesting research direction there.
Distance fields are a subset of implicit functions (see wikipedia). The use of level curves (which can be interpreted as "distance" for SDFs) is broad. You can dynamically and trivially thicken or shrink objects by adding or subtracting to the distance. Functions defined on the level curves output of an SDF can be used to map a volume to another implicit function (see nTopology). With implicit functions, the non-zero level sets are really just as important, but most focus on the zero level set in rendering.
>There's a subtle difference here because you actually need the closest point (not just the distance) so that you can look up a boundary condition.
Vector to Closest Point (VCP) is subtle but different in important ways. It is unsigned, and cannot indicate whether a position is inside or outside the object. Implicit modeling is not generally concerned with parameterizing the surface, since we can use volumetric, 3D procedural textures.
To show why it is wrong: Change the mesh every frame.