markisus

Gaussian Splatting allows you to create a photorealistic representation of an environment from just a collection of images. Philosophically, this is a form of geometric scene understanding from raw pixels, which has been a holy grail of computer vision since the beginning.

Usually creating a Gaussian splat representation takes a long time and uses an iterative gradient-based optimization procedure. Using RGBD helps me sidestep this optimization, as much of the geometry is already present in the depth channel and so it enables the real-time aspect of my technique.

When you say "big deal", I imagine you are also asking about business or societal implications. I can't really speak on those, but I'm open to licensing this IP to any companies which know about big business applications :)

Thanks so much! Even when I was putting together the demo video I was getting a little self-critical about the visual glitches. But I agree the tech will get better over time. I imagine we will be able to have virtual front row seats at any live event, and many other applications we haven't thought of yet.

Yes realtime capture and display. Locality is not required. You can send the source RGBD video streams over IP and in fact I have that component working in the larger codebase that this was split off from. For that use case, you need to do some sort of compression. The RGB stream compression is a pretty solved problem, but the depth channel needs special consideration since "perceptual loss" in the depth space is not a well researched area.

The paper does not give that many details about the evolution part. Normally, evolutionary algorithms contain some cross-over component where solutions can breed with each other. Otherwise it's better classified as hill climbing / beam search.

During the transition to industrialization it’s conceivable that many people became worse off because they could not find a place in the new economy. Even if society is eventually better off, the our generation or the next may have to sacrifice our way of life.

I think this has also existed in the form of force sensors and more recently image-like pressure sensors.

Can someone explain what “the framebuffer” is? I’m familiar with OpenGL programming where the OS can provide a framebuffer for an application but I’m confused about whether there is a global framebuffer for the entire desktop. Is this a Linux specific concept?

So the mathematicians have found an algorithm for producing the roots of quintic and higher order polynomials in terms of power series? Normally one would find these roots using an iterative algorithm so this seems like a big deal.

You can stipulate conditions to make the solution work out in either direction.

Suppose the bucket is the size of lake, and the leak is so miniscule that it takes many centuries to detect any loss. And also I need to keep the bucket full for a microsecond. In this case it is better to keep the bucket full, than to let it drain.

Now suppose the bucket is made out of chain-link and any water you put into it immediately falls out. The level is simply the amount of water that happens to be passing through at that moment. And also the next time I need the bucket full is after one century. Well in that case, it would be wasteful to be dumping water through this bucket for a century.

I have done a recent proof of concept to generate Gaussian splats from depth cameras in real-time. The intended application is for robotics and teleoperation. I made a post on reddit [1] a while back if you're interested.

I believe the quality of realtime Gaussian splatting will improve with time. The OPs project could help ROS2 users take advantage of those new techniques. Someone might need to make a Gaussian splat video codec to bring down the bandwidth cost of streaming Gaussians.

Another application could be for visualizing your robot inside a pre-built map, or for providing visual models for known objects that the robot needs to interact with. Photometric losses could then be used to optimize the poses of these known objects.

[1] https://www.reddit.com/r/GaussianSplatting/comments/1iyz4si/...

Maybe it will help to have a fluid analogy. You have a leaky bucket. What wastes more water, letting all the water leak out and then refilling it from scratch, or keeping it topped up? The answer depends on how bad the leak is vs how long you are required to maintain the bucket level. At least that’s how I interpret this puzzle.

Does this mean that all C programs invoke undefined behavior?

Is this a joke? It talks about how to help children smoke cigarettes.

> With this in mind, smoking should be commenced at as young an age as is reasonably possible. Children who have not yet developed a pincer grasp might require modified cigarette holders, safety lighters or both.

Indeed. Maybe the learned circuit does something like the following. For each token's feature vector, compute a representation of positions where it appears in the sentence. This could be made possible by the positional embeddings.

Token Features 0 => list[1, 5, 6, 10]

Token Features 1 => list[7, 8]

...

These "list features" would be invariant to Caesar cipher. So then the LLM could pass these list features to a learned Caesar cipher decoder unit to spit out the decoded text.

It's still unexplained, however, why the Byzantine Music Notation would trigger the this circuit while other Caesar cipher's wouldn't.

Does it make sense to talk about the index of a token? The model never sees such an index does it? I thought the tokens are converted to learned embeddings at the first layer.

Not sure but I decided on CBOR for my own project because I found a nice open source C library called TinyCBOR which can serialize and deserialize without any memory allocations. It works using an iterator that you manually move up and down into sub containers.

https://intel.github.io/tinycbor/current/

You’re talking about WebRTC which may or may not make use of a TURN server. And I assume there are other uses of TURN which are completely unrelated to WebRTC.

Use of a TURN server does not imply hiding of negotiation details. The TURN RFC [1] does not mention anything related to media encodings or WebRTC negotiations at all.

[1] https://datatracker.ietf.org/doc/html/rfc5766

You aren’t completely wrong apparently.

> TURN is considered an extension to the STUN. So, a typical TURN server would include a STUN server implementation by default.

https://www.100ms.live/blog/webrtc-turn-server

Won’t the firewall know to let return traffic through if my computer explicitly sends a request first?

I’m not sure what you mean by fingerprinting and supported protocols. None of that would be present inherently in a UDP stream unless the application included it. As for hiding IP address, that is a valid use case for a TURN server but I’m guessing 99% of TURN server usage occurs only because the NAT hole punch failed.

Very interesting. I even found some protocols that exactly implement the functionality I was thinking of!

https://en.wikipedia.org/wiki/Internet_Gateway_Device_Protoc...

https://en.wikipedia.org/wiki/Port_Control_Protocol

Oh I thought TURN was mainly about the NAT traversal. Can you elaborate on the other points? Especially security and privacy?

I don’t know much about IPv6 but it seems like even if we are stuck with IPv4 we could still make some sort of protocol where my computer could manipulate the NAT table entries allocated to it. Maybe something like DHCP but it deals with ports.

This has got me reading about TURN and it just seems like a huge ugly hack. Two computers want to send UDP to each other. They are electrically connected by sets of physical cables. They could even be down the street from each other. But they don’t know how to get their routers to set the right bits in their translation tables. Instead it’s just easier to send all their traffic through a third computer in another city thousands of miles away or maybe into outer space and back. Have I got it right?

I started reading it. It’s very interesting that the author thought computer vision would be the hardest part about robotics.

> Without vision, robots could not move around or manipulate objects. All of the other hardware was there. Legs and balance systems to allow bipedal motion had been in place for decades. Robotic fingers and hands with very fine motor control were easy to create. AI software to set goals and make decisions was getting more powerful every day. Everything was there but the vision system.

It turns out that dexterity has been much harder than vision.

This sounds right to me. A while ago on HN someone posed a thought experiment. “What would happen if literally every investor bought only index funds?” This made me realize that you need active investors. Trouble is that no one knows who the good ones are. And the good ones now may be horribly next year.

I thought it was the opposite of what you’re saying. Index funds buy more of stocks that are higher. And why would a 95 percent NVIDIA be against the law? It just wouldn’t be an index fund.

Article says it’s about stack addresses but I think it should apply more generally to dynamically allocated memory as well.

Here is his 1984 article published in PRL in which he highlights that cellular automata can be realized as physical systems and therefore computational intractabilities will manifest in models of physics.

https://content.wolfram.com/sw-publications/2020/07/undecida...

It's defined on page 5.

> In this paper we are considering the behaviour of ∆(HΛ(L)) in the thermodynamic limit, that is, when L → ∞

So it's the same as the regular mathematical limit. There must be some thermodynamic implications but my knowledge of physics is not enough to say.