slimbuck

Hi Wieland and congrats on your paper!

The issue you saw had been addressed in 1.7.0. We were generating codebooks for color and scale using k-means, which would result in bad values sometimes for the extremes. In 1.7.0 we switched to an optimal quantiser with outlier handling. It was actually surprisingly hard to get this right.

I just got opus to valid this now and it found another issue with our scale codebook gen, which I will fix now. Huzzzah!

The biggest update is adding alpha compensation (alpha > 1). With this in place we can addresses the gaps that appear after performing many rounds of decimation.

I had no idea SOG was being used this way, so cool! I'm busy working on V3 if there's anything you think we should include.

https://github.com/playcanvas/splat-transform/issues/362

We could get better compression using a custom encoder, however with webp the browser decodes off the main thread, copies the resulting data to GPU memory and caches the images for us. All very useful on the web!

Perhaps niche compared to the entire web, but we use lossless webp for gaussian splat data delivery for realtime rendering. (https://developer.playcanvas.com/user-manual/gaussian-splatt...).

Scenes often comprise gigabytes of lossless webp, some of which you can see at https://superspl.at.

The two challenges with something like this are:

1. Capturing the entire scene with photos or video and converting ("training") these into a 3d Gaussian splat. This is what Vincent did so well with Grace. Open source tools for training are Brush and Lichtfeld Studio.

2. Creating a user experience with the resulting capture. In the case of Grace it involved adding the peek effect, camera controls, moving cars etc. This was all done using PlayCanvas as the article describes, which is also open source.

There is still some art to doing 1 well, but with a capable LLM agent and our (PlayCanvas') awesome community on Discord, you could probably manage both.

(And if that doesn't work for you, please let me know!)

Hello, I'm wondering whatever you mean? You can absolutely walk around the church indoors on mobile. Just tap to walk.

If you can, please check any output/errors on the developer console?

We (playcanvas) use k-means clustering to generate a 64k entry palette of coefficients in our SOG compressed GS format.

The data savings are huge, but the compression step is slow and requires a GPU.

It would be awesome if we could find a better/faster method of compressing these 45 dimensional monsters.

Right, thanks for confirming. It seems firefox related. We'll get this patched asap!

Hello, playcanvas developer here. May I ask what phone/device you're on? Might be a bug. (No pun intended).

Have you tested the shader's runtime performance penalty after minification?

Also wondering how you handle named uniforms?

Some of these would be better converted to 3dgs.

This app was made in the PlayCanvas editor. The models were imported and a custom GLSL shader written to handle the particle animations.

I'm not really sure what you mean. Think of SuperSplat as the photoshop of gaussian splats?

- SuperSplat dev :)

I love this article! I used it as a basis for the multi-level grid in SuperSplat (https://playcanvas.com/supersplat/editor).

^^ https://playcanvas.com/model-viewer?load=https://code.playca...

Yes Dreams is actually a huge inspiration with the realtime rendering side of gaussian splatting. Specifically this presentation by Alex Evans https://www.mediamolecule.com/blog/article/siggraph_2015.

Dreams doesn't use gaussian splats as such, but we still learn a lot about how to compress and render a huge number of particles efficiently. (We're not doing half of this on PlayCanvas... yet).

Did you see that we credit Aras at the end of the post and link to his website?

Splats change color based on direction, but opacity is constant.

Yes! We are using this clustering approach for position and scale data.

It's definitely possible to compress/palletise the SH bands (see https://aras-p.info/blog/2023/09/27/Making-Gaussian-Splats-m... for one approach).

We just found that splats encode some view-dependent shininess anyway (for example the guitar body https://playcanvas.com/model-viewer?load=https://code.playca...) and so considered the extra data and rendering cost of SH not worth it for now.

It would be quite straightforward to add SH to the PLY file and we'll definitely consider it in future!