Hacker News Reader

Comment by adamredwoods | Hacker News Reader

So I wonder if unusual angles that normally do not get photographed will be distorted? For example, underneath a table looking up.

View on HN

They reapply noise to the potentially distorted image and then predict the de-noised version like the originally rendered first frame. So the image is at least internally consistent for the frame (to the extend the the system generates consistency whatsoever).

The example with a squirrel wearing a hoodie demonstrates an interesting edge case, the "front" of the squirrel (with hoodie over the head) show a normal hooded face as expected, but when you rotate to the "back" you get another face where the hoodie is low over the eyes. Each looks fine in isolation, but in aggregate it seems like we have a two-faced squirrel.

Yes, this is often a problem. We use view-dependent prompts (e.g. "cat wearing sunglasses, back view") but the pretrained 2D model often does not do a good job of interpreting non-canonical views and will put sunglasses on the back of the cats head (as well as the front).

>cat wearing sunglasses, back view")

Bad prompt, missing implied antecedent/ambiguous subject...

You may want:

Back view of a cat which is wearing sunglasses, back view of a cat, but the view is wearing sunglasses, etc... I actually tried using projective terms from drafting books, and didn't get great results. Nor anatomicals either.

>Back view of a cat which is wearing sunglasses, back view of a cat, but the view is wearing sunglasses, etc... I actually tried using projective terms from drafting books, and didn't get great results. Nor anatomicals either.

In short: natural language is not good enough and you need a DSL. If only the last 60 years of language research had warned us of this.

Up next: English sentences are ambiguous and need context information to parse correctly. Machine learning community in shambles.

I mean... It kinda did. Sarcasm aside.

The trouble is all those darn uninitiated and trying to create a generalized oracle to map their inspecific ramblings to what they mean to free them of having to actually communicate properly...

Actually, funnily enough, this has cross section with philosophy in a way most programmers scoff at; but communication is frigging hard, and worse, detecting when someone is trying to get something across, but just needs a nudge in the right direction to be able to find the language to explain it is really damn hard.

I run into it every time I get a haircut. I have no idea how to speak their language, so it's always "Uh... A little off the top and rounded at the back, I guess?"

yep our fixed strategy for view-dependent prompting is silly and there is tons of room for improvement!

Some of the "mesh exports" used as examples on the page actually show this, to some extent. Look specifically at the plush corgi's belly and the weird geometry on the underside of the lemur's book, and to a lesser extent the underside of the bedsheet-ghost.

It'll be delusions and guesses, rather than distortions.

It'll just make up some colours and geometries that don't contradict anything it already knows from the defined perspectives.

Or leave it empty.

You can see that in some of these examples e.g. "plush toy of a corgi nurse"

Reply on news.ycombinator.com