Hacker News Reader

Comment by chaboud | Hacker News Reader

I'm disappointed that the illumination is not uniform in the stitch. Artifacts from the sampling and stitching leave subtle bands in the output that I'm fairly confident are not in the original. This is likely a biproduct of using some non-uniform scope-attached illuminator and not having sufficient overlap in their sampling.

Nonetheless, when zoomed in, the detail is impressive.

View on HN

I worked on this project. The images were indeed captured by a 3D microscope in about 9000 separate captures with non-uniform illumination, so this isn't so much a stitching artifact as it is an illumination artifact. The specular component of the varnish and the different illumination angles of the light sources make it almost impossible to capture a uniformly-colored field without the use of polarization filters, and the best stitching algorithms don't do well with different opinions from multiple images about the color of a pixel (they do fine with different opinions about brightness).

have you tried to extract the vignetting pattern form the captures and using it to normalize them? My first try would be to calculate the median grayscale image of all 9000 captures and then using this to normalize the intensity.

Yes, and the vignetting pattern isn't so much the problem. It's a color distortion problem (spatially varying chromaticity rather than brightness that is not just a function of the position within the field of view but also of the underlying material, unfortunately). So there isn't a nice way to correct each capture in a predictable way to ensure that overlapping pixels have the same colors consistently.

i see. interesting problem

Anyone have a nice primer to read to help nonspecialists understand what you are talking about in this thread here?

They are talking about a bidirectional reflectance distribution function, a now common modeling technique for practical representation of complex surface properties when illuminated.

That said, a full BRDF is not the only way to approach this problem, especially at reduced resolution where the artifacts are more apparent.

If you're an ACM member, there is a wealth of information in SIGGRAPH publications. Having been in graphics since the 90s, I started with Foley and Van Dam (Computer Graphics: Principles and Practice) and Watt and Watt (Animation and Rendering Techniques) and then stayed on top of SIGRAPH (attending regularly, though not annually) since then.

For a more whimsical survey from the pen of a straight up genius, Jim Blinn's books (e.g. Dirty Pixels) are fantastic reads.

I've worked with similar problems for agricultural mapping from drone images. You would need to build a BRDF model for the different colour channels for various types of materials, then assign the material based on a combination of best representative models. Then you can re-render with uniform normal lighting.

Would it be hard to have uniform illumination?

Yes, because the surface isn't flat. At these magnifications, any given field of view may be tilted toward one of the light sources, changing the relative contribution of the specular reflection off of the varnish as well as the reflected color from the paint surface. The apparatus doesn't have the ability to tilt to maintain a constant angle to the surface; it can only pan in the x-y plane and do focus stacking in the z-direction. Additionally, the left and right light sources were hand-positioned and there is no way to calibrate their exact geometry and relative brightness and color.

thanks for explaining!

Fantastic work, congratulations. Hope to see more of these.

Was the 3D effect made from different focus points?

Very likely. The focus stacking software (such as Helicon) can output the 3d object in addition to the stacked image

The very limited depth of field of a microscope necessitates doing depth stacking for every field of view. This is done automatically by the apparatus. As a side effect, it gives a heightmap, but with the lens used for the whole-painting scan (what HIROX calls "35x", corresponding to ~5 µm sampling resolution), the elevations are not reliable, especially near the edge of the field. For selected areas, a higher magnification was used that gives much more reliable elevations. Sadly, only a small fraction of the painting was imaged using this higher resolution, so the 3D data is spotty.

For purposes of browsing, you can source your top-level "zoomed out" layer from a single photograph and blend at lower tiles in the quad tree or normalize the downsampled capture data against a reference.

Nonetheless, thanks for doing this. It's an amazing piece of work.

Indeed, looks like with the illumination at hand there should have been more overlap in the scan process. The original is not striped!

Anyway, phantastic piece of art and it's great to have the opportunity to look real close!

Reply on news.ycombinator.com