GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images

In the short term you may be right, but in the long run it's a certainty you won't.

You can optimize the poses as part of the model.

I don't see it as a blocker... Especially if you alternate NeRF and SD iterations - ie. don't generate a whole image each time, instead just choose a random angle, do NeRF reconstruction, then run a single SD iteration on that NeRF, and do another training step of the NeRF.

That way, you know the exact pose for each image, because you chose it when rendering the NeRF.

This 100%. Most images in SD are paintings and have a certain compostion (for instance people usually looking at the camera, posed a certain way, etc). Trying to generate the same thing from a different angle will yield wildly different results which won't work well for Nerf. I do think diffusion models may work for generating 3d stuff, but right now a) We need a big labelled dataset of various 3d objects in different styles (yes, I know some exist but they are nowhere near what we have for 2d images since 2d images can be scraped from the web) 2) Memory/computation is a big bottleneck as 3d is orders of magnitude bigger in terms of memory and processing. That said, hopefully some smart person will come up with some shortcuts that make it possible on today's hardware.

Don't see why not, especially for low impact, high frequency decisions. Some AI guided assistance with option to automate. The next level for auto complete I guess.

I wonder what your input data would need to be for a competant AI in this space.

Finance, goals, delivery timelines, capabilities of the team, employee availability, which employees work well with each other, office politics, regulatory constraints...

The problem is finding a good training set.

Visions of the future where the consumer has to pick apart "human curated", "human assembled" and "human made" much the same way we do now for cage free and free range eggs at the grocer.

That would be the same as curating social media feeds.

Except it wont be, will it. Such things were promised for the internet and we had maybe a good 10 years or so before corps caught up and told us what to watch through their channels.

I imagine this being much of the same: AI trained on corp-approved training sets to give suggestions to your preferences that they want.

Sure, you could spin up your own and buy a machine that trains on it's own training data, but watch how no one will do that because of the cost, or the diminishing access to untainted resources.

Why would being a "head audio engineer" at Julliard give you any credibility on AI generated music and sampling/copyright law? Lol.

It's not sampling. Sampling is copying & pasting, but that doesn't happen, technically. Just like stable diffusion doesn't copy any artworks. AI learns from previous works, but doesn't copy them. It's quite similar to how humans learn and make adaptations based on other work.

That seems like good advice for professionals, but I'm wondering if it's going to hold up with new ways of distribution.

Would distributing a generative model that can sometimes generate such music would also be considered illegal? Will it actually stop people from doing it in practice?

Would it be illegal to share seeds and prompts?

Though these alternative methods, you could have a lot of people listening to music that's never distributed as audio or video files. And if there's an API for it, games could use such generated music via a plugin.

And then I suppose people start sharing on YouTube, and we see how good their copyright violation detection actually is.

People sample music all the time. Sometimes credit and royalties are given, but there are lots of popular music being made that samples without credit.

Against the law? Maybe, probably. I don't think the laws are as strong as they appear though.

To be fair, these models are no worse than the ones the iPhone makes with LIDAR. It's pretty impressive for being generated from a single static image.

At some point we'll have fusion and/or fully self-driving cars. Still the AI only can replicate learned ideas, so you can't do anything that hasn't been done before. Generated content will look 99% there but be just a bit off. Besides that, even if you could do 'anything' in a game, what's the point if nothing you do has meaning?

>Generating good video game assets from pictures is solved

lol no, not at all. It still needs tons of manual work to get it up to quality in terms of topology, material, etc.

I think what they mean by UV unwrapping is generating a UV map from. Textured model (here the texture is generated by a triplanes network).

It’s interesting for compression purpose, but kinda orthogonal to this method (a good UV unwrapping can be used once the model generated)

Generating an UV map from an untextured mesh. The UV map is stored in vertices as texture coordinates, and together with the topology of the mesh it defines how the texture (images) gets mapped to the mesh. A good UV maps preserve surface area (e.g. every region of the mesh map to regions of the texture proportionally, otherwise you get stretching), have few seams and have little empty space around the texture to reduce size.

There are ways to do this automatically but they're far from perfect. Artists usually take the mesh and literally unwrap it until it's planar, and convert this transformed mesh to the UV mapping. The advantage of this method is that it gives you very good control of seams and texture islands, but it's tricky to preserve surface area.

Those neural rendering methods are very cool because they use light/color fields, but they still have a lot of catching up to do compared to modern 3D graphics.