Amazingly beautiful work. Thanks for sharing your knowledge and process!
It would be cool to combine this with some kind of classifier for surface types and simulate water, glass, reflections, sub-surface scattering (basically other types of materials).
Have you played with any of the new hardware raytracing API’s from Nvidia RTX stuff?