Very cool. One issue I saw was that the shadows actually show on top of other objects.
Yeah, I need to fix that, this is link to my wrong sampling implementation.
If you use only 1 sampling you will see it is fixed.
Might I suggest instead of a solution in real time, how about a static deal that essentially generates a lightmap to overlay on top of the scene? I can think of several uses for that.