Hacker News Reader

Comment by mabbo | Hacker News Reader

The most interesting part, to me, of a release like this is the amount of "please don't abuse this technology" pleading. No licence will ever stop people from doing things that the licence says they can't. There will always be someone who digs into the internals and makes a version that does not respect your hopes and dreams. It's going to be bad.

As I see it, within a couple years this tech will be so widespread and ubiquitous that you can fully expect your asshole friends to grab a dozen photos of you from Facebook and then make a hyperrealistic pornographic image of you with a gorilla[0]. Pandora's box is open, and you cannot put the technology back once it's out there.

You can't simply pass laws against it because every country has its own laws and people in other places will just do whatever it is you can't do here.

And it's only going to get better/worse. Video will follow soon enough, as the tech improves. Politics will be influenced. You can't trust anything you see anymore (if you even could before, since Photoshop became readily available).

Why bother asking people not to? I guess if it helps you sleep at night that you tried, I guess?

[0]A gorilla if you're lucky, to be honest.

View on HN

I wonder/hope if a downstream effect of this technological change will be the end of the idea of a "shameful" or "humiliating" image. We all have bodies, we all have sex, and I agree that soon images of ourselves being nude/having sex etc. will proliferate because they'll be generated instantly via Siri shortcut as part of casual banter.

In a world where every celebrity is having sex with gorillas, doesn't such an image lose its charge? Will Norms and values around sex/body shaming change?

If we get to the point where we can't tell AI generated images from reality, I'm not sure "body shaming" will be on society's collective radar anymore.

Why?

Much bigger fish to fry.

Think things like forged evidence in trials.

It's the right time to get into the digital forensics business.

..and the Berlin gorilla nightscene

I second this.

Appropriately prioritizing problems has never really been society's strength...

That's because pesky things like "democracy" get in the way of getting shit done.

Don't you think?

I bet someone said in 1830, "by the time we can send robots to Mars, body shaming will not be a thing anymore". For better and worse, that's not how we humans do things generally.

> In a world where every celebrity is having sex with gorillas, doesn't such an image lose its charge? Will Norms and values around sex/body shaming change?

I hoped this would happen with social media. Everyone says stupid shit online and everyone has past beliefs they’ve outgrown. So what’s the big deal?

Instead we went the opposite way. Everyone is super self conscious and censoring at all times because you never know who’s gonna take it out of context and make a big deal.

This is not the same though, it's about being able to deny that that photo of you is real, not that it's taken out of context.

Well what’s a “real” photo?

If I take your photo and photoshop (very well) different surroundings. Is it a real photo if you?

If I photoshop your face (very well) onto a different body. Is that real?

If I feed your photos into a model that can create realistic versions of those photos in different poses or with different facial expressions. Is that real?

They all start with something that is very definitely a real photo. You can’t (yet? ever?) generate a realistic photo of a specific person from a textual description. The machinery needs a source.

A real photo is one created by photons outside the camera

You’ll be surprised to learn that doesn’t work without some amazing tech to process the photons. Different settings will produce a different photo.

Hell, just changing focal length makes a bigg difference to what your face looks like: https://imgur.io/gallery/ZKTWi no digital manipulation required.

Which of those faces is “real”? They’re all just recording photons hitting the camera, but look very different.

It gets even worse when we start talking about colors. For example: it took cameras decades before they could accurately capture black faces. Where accurately means “an average person would say it looks right”

https://www.vox.com/2015/9/18/9348821/photography-race-bias

Edit: here's a fun example of how journalists use perspective and focal distance tricks to support their desired story angle. No digital manipulation https://twitter.com/baekdal/status/1254460167812415489

Simple: a "real" photo is one in which a light field from the real world impinges on a photosensitive media (CCD, film), and directly encodes that information, with some allowance for global light levels, gain, ISO, and speed. Anything else is a modification therein. HDR, multi-exposure compositing, etc, aren't truly "real". They may be 99% real, but aren't 100% real. If you crop it, it's 99.9% real (we have models which can detect cropping and even from which region it originated, obviously it can't reconstruct the missing data).

Yes, by that definition, most photographs already aren't real.

So if I take a photo of a photoshopped photo, it is a real photo.

It is a real photo of a photoshopped photo, not a real photo of the photoshopped photo’s subject.

> Yes, by that definition, most photographs already aren't real.

What if I use no digital manipulation at all, but play around with focal lengths or perspective to produce the desired effect. Is that a real photo?

For example from covid reporting: https://twitter.com/baekdal/status/1254460167812415489

Sure, I'd say that's a real photo - you could easily pull it off with a single exposure of film.

Just because the photo is real, doesn't mean it's free from deception. I could take a "real video" of gorilla suit^W^W bigfoot and it'd still be deceptive.

It is an interesting point [the hope that values will adapt to reflect typical mischievous patterns of social dynamics within various clusters].. I used to marvel during the dotcom era at the salivating delight of "the internet never forgets" of college students caught smoking a bong and the subsequent impact on their career (virtue signalling?).. I saw hope that society could then adapt about ridiculous FUD biases. There is a strange relationship of scarcity & opportunity and these windows into souls painting a distorted picture of the darker side of our ambitions. It is a cancer. Also, humans are always testing boundary conditions - curiosity, discontent, security,insecurity. Some inherent desperation of people jockeying for the next path from frying pan to fire in the search for greener pasture breeds opportunism that seems certain to favor desperation and negativity.

That seems unlikely, given that making up hurtful stories about people and transmitting them via text or voice is still a thing. Everyone knows that anyone can make up any story they want without any technology whatsoever, and yet spreading rumors is still a thing.

Not really. I can't think of a recent "leaked texts" that the participants cannot not easily and plausibly deny (e.g. elon's supposed messages to gates), or even voice messages. Even most images can already be denied as photoshop if all the witnesses agree. The only medium that is somewhat hard to deny is videos, like sex tapes, but that's also not too hard. I think there will soon be a race to make deep learning pics look completely indistinguishable from phone pics.

Perhaps that's somewhat true for famous people, although there are plenty of examples of false stories (without any forged evidence, literally just stories) causing real embarrassment and damage to reputation.

But it's even more true for non-famous people getting bullied in their social groups, both online and offline, and that's more what I was responding to (the "asshole friends" in the original comment).

It'll be hard to deny crypto-signed photos https://petapixel.com/2022/08/08/sonys-forgery-proof-tech-ad..., especially if they include metadata that distinguish photos of AI generated images from normal photos.

any camera can be hacked to plant an image in its framebuffer

Maybe at some point for some cameras. But not soon after their release if they took steps to protect their pipeline with hardware.

There are a few different kinds of 'secure enclaves' implemented on chips, where you can have some degree of trust that it "cannot" be faked.

E.g. crypto wallets, hardware signing tokens, etc.

We could imagine an imaging sensor chip made by a big-name company whose reputation matters, where the imaging sensor chip does the signing itself.

So, Sony or Texas Instruments or Canon start manufacturing a CCD chip that crypto signs its output. And this chip "can't" be messed with in the same way that other crypto-signing hardware "can't" be messed with.

That doesn't seem too far-fetched to me.

* edit: As I think about it, I think more likely what happens is that e.g. Apple starts promising that any "iPhoneReality(tm)" image, which is digitally signed in a certain way, cannot have been faked and was certainly taken by the hardware that it 'promises' to be (e.g. the iPhone 25).

Regardless of how they implement it at the hardware level to maintain this guarantee, it is going to be a major target for security researchers to create fake images that carry the signature.

So, we will have some level of trust that the signature "works", because it is always being attacked by security researchers. Just like our crypto methods work today. There will be a cat-and-mouse game between manufacturers and researchers/hackers, and we'll probably know years in advance when a particular implementation is becoming "shaky".

How would such a camera ensure you're not taking a picture of a picture?

Just get a really really nice screen to display deepfakes on before photographing them

Maybe have the signed content include a timestamp and location? But... GPS can be spoofed I think.

In which case if found out will result in that person / publishers credibility going down the drain.

We will learn to trust sources (cryptographically signed) rather than just what we see.

Are journalists savvy or ethical enough to give a shit? What about the people reading/viewing/listening to the news?

>In a world where every celebrity is having sex with gorillas, doesn't such an image lose its charge?

No, because the image itself doesn't matter. What matters is how much the public wants to hate someone. If t he public is primed, any remotely plausible incriminating image will do as an excuse.

Fortunately, these images are actually far than what someone can cook up with Photoshop. Unfortunately, it's a part of a bigger trend where we get more and more tools to produce, manipulates and share information, while the tools to analyze and filter information are lagging by at least half a century.

Reminds me of the novel The Light of Other Days by Stephen Baxter: https://en.wikipedia.org/wiki/The_Light_of_Other_Days

I don't generally like his style, but this novel really captivated me. Without spoiling the plot (much), the end-result of everyone being able to spy on anyone anywhere at any time is a kind of societal disinhibition, especially related to sex, nudity, or similar taboos.

> doesn't such an image lose its charge?

Yep! Individual images, and individuals in the first place, absolutely loses significance as supplies increase.

> Will Norms and values around sex/body shaming change?

The bar skyrockets! Look at TV stars from before 1995 - they should already appear below-average amateurs to modern eyes.

As someone who witnessed AI Dungeon's GPT-3 model with an unlimited uninhibited imagination for erotica I would download everything now before they cripple the models. I would not be surprised if they very shortly completely stop downloads due to "abuse" and pursue a SAAS model.

I think it's funny how Yandex, a Russian company, releases these big language models without all the AI safety handwringing in their press releases. The Russians have a tradition of releasing technology without giving a lot of worry about what happens to it, for better or for worse. For example, they made between 75 and 100 million ak-47s, a not in any way limited machine gun, and it spread to every corner of the world. They even gave out all the plans and technical assistance so any one of the organizations they worked with could produce their own. 20 different countries make ak-47s currently. Of course you had to register every xerox machine in the Soviet Union, so maybe they just had different priorities?

The west is absolutely fascinated these days with the control of advanced technology. Drones, Blockchain, and AI models seem to be the latest things that the west is determined to exercise control over. For example:

"Many of the technological advances we currently see are not properly accounted for in the current regulatory framework and might even disrupt the social contract that governments have established with their citizens. Agile governance means that regulators must find ways to adapt continuously to a new, fast-changing environment by reinventing themselves to understand better what they are regulating. To do so, governments and regulatory agencies need to closely collaborate with business and civil society to shape the necessary global, regional and industrial transformations." -Klaus Schwab, "The Fourth Industrial Revolution", Page 70.

It's not about control of advanced technologies but rather making such technologies look more powerful by pointing out real or imagined destructive capabilities. If you want funding or "big up" a topic, claim it has the power to bring down the world (anyone remembering gray goo?).

The RTS game?

No, the concept it was based on: https://www.britannica.com/technology/grey-goo

> grey goo, a nightmarish scenario of nanotechnology in which out-of-control self-replicating nanobots destroy the biosphere by endlessly producing replicas of themselves and feeding on materials necessary for life.

StableDiffusion was trained by an academic lab in Germany, which is rather different from a company in California doing it. Though they both have the problem that the output is illegal in many countries with the wrong prompts.

"The West".

As if we're all one block, a single minded hive. Corrupted, degraded by a heavy capitalistic and egotistical mindset. It's fascinating to see and talk about our shortcomings isn't it? We are so inferior. The fact you wrote "The West" signifies you're one of "The Others". Someone possibly from Russia. Then you compliment Russia and degrade "The West" adding further strength to this hypothesis.

Using "The West", a simple minded sound byte that's been used in propaganda for centuries, a way of appealing to our ape-instincts to protect our tribe against others. "Us" vs "The Others".

You should make a little effort in seeing how ridiculous, infantile, and brainwashed it is to refer to countries that are not Asia or Russia as "The West".

> Politics will be influenced. You can't trust anything you see anymore

I've been wondering for a while now if this will lead to an unexpected boon: perhaps people will be forced to pay attention to a speaker's content instead of simply who is speaking.

The problem with this is that you will never know who is actually speaking. Deep fakes are already a thing, but as they get better and more accessible we will approach a world where anyone can make anyone say anything and make it hyper believable. In that world, it will be very difficult to tell what is real.

My personal hunch is that this will end up leading to a situation in which presenters do a cryptographic handshake that works to verify and prove authenticity. This isn't a new idea, and it has some very obvious drawbacks, but I don't see much of a way around the issue. The handshake could work great for something like official news releases, but for other instances that might come up in court, say, dash cam footage of an accident, it seems to me that the legal system is going to face some serious issues as these programs progress.

Looking forward to a future where all a politician’s quotes are on a blockchain, signed by their private key, and they chose to do so voluntarily out of fear of deep fakes.

Will remove all the useless “I didn’t say that”

Do politicians understand blockchains?

Neither do we[1], so it's sort of a mixed bag.

1. True for an overwhelming majority of the body politic.

Rename it as “verified speech transcripts.” I don’t need to understand video codecs to watch Youtube.

That's a start! I think their video appearances should be like a car in NASCAR with permanently displayed logos superimposed from all the interests that have funded their rise.

It will end up back where we started, "it's not on the Blockchain so I didn't say it", while making racist remarks with friends.

"We verified the validity of our source's signature and vouch for it's authenticity"

- Journalist from a respected newsroom, when they choose to keep the source confidential. So it won't be better or worse than it is now: all about transitive reputation.

The unspoken threat to further denials is the risk that the source may go public, or if unauthorized, doxxing the person who recorded the video (assuming signatures have nonrepudiation)

https://www.washingtonpost.com/news/the-intersect/wp/2017/09...

Why would this require a blockchain?

Unfortunately a speaker's content can also be auto-generated now, at least for brief enough snippets. And that means the content can (and will) be optimized to appeal to a target segment much more than has ever been previously possible.

If I recall from the interview with Stability.ai's founder, he has more or less the same opinion, and that humans will adapt to the new technology as we always have. I figure "please don't abuse this technology" warning stickers are more CYA. It'll make the vast majority of judges look at a motion to dismiss and not blink an eye.

Historically, he is correct. It is easy to find people that were against TVs, Cars, Trains, Electric Cars. Those people were not entirely wrong with their logic. Trains and cars did make it much easier for scammers to come into a town and then leave quickly.

In terms of the equilibrium, this is certainly a true observation. However, historically speaking new technology can be extremely disruptive in the short-term as society figures out the new norms, and the power-structures are disrupted and then re-equilibrate.

Concretely, it's probably true that children born with this technology will have adapted to many of the negative (and positive) aspects of it. But the current generation of elites, politicians, and voters might have a harder time adapting.

are we even sure that TVs are a good thing?

>make a hyperrealistic pornographic image of you with a gorilla[0]

I don't understand this irrational fear. This can be done today, just need some minutes instead of some seconds to create a good Photoshop.

Also, seriously this is the thing you fear? fake porn? there are much worse thing you can do with this tech, like phising, falsifications, etc. Not mentioning leaving millions of graphic designers out of job.

Unfortunately Anything that can go wrong will go wrong

Photoshop is a skill, not very widespread as we assume.

Typing something is literally at everyone's fingertips.

Photoshop is a skill possessed by more than enough people for the world to have already been transformed by Photoshops of famous people and anybody else graphic designers wanted to embarrass, if moderately-convincing images were that transformational a technology for adversarial use.

I agree that it's much easier to do low effort stuff to wind friends up, but universal access and low effort don't make it more likely to be impactful and believable.

That's what I was thinking. Whether a pornographic picture of me and a gorilla was made by photoshop or AI is irrelevant. People's reactions will be the same, and repercussions will be mostly the same (which doesn't means there will be consequences).

If someone really wants to hurt you, not having AI isn't going to stop them.

The real effect will be that you can publish a real picture of you fucking a gorilla and nobody would believe it because it's trivial to generate it with an AI.

> seriously this is the thing you fear? fake porn?

I'm being polite. Things will be so much worse.

Someone will make child pornography using your child's face as the input. Someone is going to take private videos of politicians and then edit them to have them say incriminating things. Someone is going to short the stock of a large company, then release a faked video of the CEO being shot, and profit from the immediate stock plunge.

And this is just what my mind can come up with. Imagine what 4chan will invent.

> The most interesting part, to me, of a release like this is the amount of "please don't abuse this technology" pleading.

> Why bother asking people not to? I guess if it helps you sleep at night that you tried, I guess?

I've perceived this as them doing the necessary amount of virtue-signaling and ass-covering to avoid the ire of groups that are loud/powerful enough to cause issues for them in the short-term. I've been following the developments in this space for a while now, and I don't get the impression that Stability AI cares too much about forcing Western ideals of "correctness" onto the public.

It's plainly obvious that this is going to be immediately used to produce content considered obscene, offensive and/or illegal by various groups of people. And it is what it is, we're going to just have to figure out how to live with it as a society. It's going to get far far worse (from certain points of view) as we continue to replicate functionality previously exclusively featured in human brains.

I never saw it create anything more NSFW than female boobs. It won't generate penises or genitals besides mounds of hair. Right now the "NSFW" flag/filter is 99%+ false positives. This is stuff that used to be allowed in G-rated movies in the US.

True to both Stable Diffusion and Dalle2, the more encompassing the content of your image is, the more incorrect the detail is. Both generally will make fantastic faces but anatomy and consistency starts falling apart with full bodies. Contorted limbs, missing fingers, too many limbs, etc. No one is going to be fooled here.

Of course the models could be trained on offensive images and over a long enough time period eventually will. But, for now, someone is going to have to spend millions of dollars on compute and have the human expertise behind it too. Then again, if we had an image generator making pictures of [insert X offensive thing] that is a whole lot less disgusting than the plentiful real photos and videos of that thing in reality.

It feels sometimes that there has to be small, influential group of people on some jihad against generated pornography on the Internet, with impacts in the order of $10-100mil.

I'm sure this is not a view shared by everyone, but it is a straightforward course of action to add such disclaimers in cases where you do agree with it and not wanting to be targeted and blacklisted from payment networks(that seems amongst their weaponry).

This.

They just have to cover their asses, any sane dev would make the same license due to the power of this tech.

On some level I can't stop laughing since OpenAI really got smoked. "OpenAI" my ass, this is what open TRULY means!

Cheering for these devs.

> you can fully expect your asshole friends to grab a dozen photos of you from Facebook and then make a hyperrealistic pornographic image of you with a gorilla

my prediction is that, as a result, people will start assuming pics online are fake until proven otherwise.

> my prediction is that, as a result, people will start assuming pics online are fake until proven otherwise.

"That worked well for quotations." — Abraham Lincoln

I much prefer a company that asks me to not abuse it but lets me, then a company that treats me like child and force filters it out for me.

That's not the only issue with NSFW; the larger problem is when you don't ask for it and get it anyway. Especially because this model is not particularly good at it and you'll get body horror.

Well it comes with an NSFW filter that's activated by default so that's a non-issue (but importantly you can turn it off if you're ok with accidentally seeing stuff like that without explicitly asking for it).

Like you mentioned near the end, all this has been possible with photoshop, with amateur level skill. Hollywood can CGI the entire Captain Marvel movie, so as far as state-level efforts go, AI can really only be an incremental improvement at best.

I think this is all just trendy popular sentiment moralizing AI.

Scale and access (≈quantity) matter.

It seems inevitable like it's "inevitable" that they Photoshop your face onto porn. Yes, of course it will happen but maybe not to most people? I'd guess inevitable for many celebrities.

It will be more now. Photoshop needs some skills. With AI it gets easier and easier.

But even today, we deal correctly with it. Fakes and real photos are mingled together in 9Gag/LatestNews reports about Ukraine. Under the fakes (and the real), people ask for confirmation. Someone says it’s true, no-one believes him, until a link to a newspaper is dropped. And 9Gag isn’t the highest IQ community around, so yes, general population does distrusts photos by default until proven.

They are laughed at anyway if they tell a story coming from a forged photo.

Sure, newspapers could forge stories, display pictures with, I don’t know, Biden’s son with a crackpipe, and make the populace believe untrue stories. But guess what, they already do it anyway, newspapers already “spin” (as they say, i.e. forge, suggest without literally saying) stories all the time.

The world deals correctly with fakes.

The problem comes from the early adult newsroom interns responsible for sourcing content. They don’t know it’s fake, it sounds like a good click-baity article to them, so they run it. It happens.

I wouldn’t shift responsibility on the shoulders of the last newcomer. The top of the management has had ample time to diagnose this. If it remains like this, it’s by design.

I have a quite different perception of 9gag. Yes, some ask for confirmation but it depends very much on the topic.

Wrong topic and facts get downvoted and the fake news prevail.

And not all links to newspaper are considered valid, especially if it's about "woke culture". Then you have to search the reasonable needle in the haystack of transphobia, homophobia and misogyny.

Significant progress has been made on video - inter-frame preservation of the surface level spatial invariants in 3D environments has been achieved.

But preservation of transtemporal spatial invariants requires understanding far more than that - dynamic lighting, density, flexibility, rigidity and momentum, the viscosity of the air, the skeletomuscular system, the flow of the fluids within, and so on ad infinitum.

And a lot of that is tacitly understood by the human mind (even when the human mind would struggle to generate a scene, it can often detect that something is wrong - try turning on the lights in a lucid dream).

It's going to be quite some time before it reaches the point where a human cannot detect that video was generated (or altered) and even longer before computers can't.

But then, it's going to be a shit-show - and I'm not talking about the bestiality videos.

Evidence, as we know it, will be meaningless - the implications for the legal system are terrifying.

I think you don't give enough emphasis on the "if you even could before" part. Sure, this makes realistic fakes easier and more available to to the masses. So what? We've had the technology to 'frame' people in similar ways for decades now, and other than entertain the 4chan teens it does squat.

There's enough money interest in bringing down certain politicians, if faking a sex tape or back-room conversation would make any difference to the world it would have been done already. Hell, politicians pretty much admit publicly that they are rapists and we don't do anything about it. Who's gonna care about a couple of gorilla-pics of a normie?

People don't and never have trusted any evidence that doesn't reaffirm their world view, regardless of how true that evidence is. More realistic evidence won't change that.

These images are good for satire, spoofs, and comedy but they definitely won't be fooling anyone.

This is great news for those that actually have sex with a gorilla, because now they can claim it's an ai photo. :)

Kidding aside, I think this is actually good. Humans need ephemerality. We are never getting the full version of it back, but with photorealistic ai video and image creation some freedom returns. I think without it a society in which everyone has a camera all the time would mean absolute ossification of social norms. Right now it's very, very new - I mean multiple generations living with the current, or better (eg. recording eye implants) technology.

I think that s a hopeful message.

At first we will see a lot of "prompt censoring mobs" that will try to "stop abusers because children and terrorists", but as the images multiply, and they will multiply, the line between real and fake photos will become hard to spot, for real. This is i think a pivotal and great moment, because everyone can now claim plausible deniability to any picture. No revenge porn will be believable anymore, nor will anyone know if that Bezos's weiner pic is real or not.

Really, the licensing is most interesting? There’s a lot of public info about training and development too.

The license itself is pretty irrelevant. What people will actually do with the training blueprints, and how fast things will evolve.. now that’s interesting.

Perhaps we should have started with artificial wisdom instead of artificial intelligence

> Why bother asking people not to? I guess if it helps you sleep at night that you tried, I guess?

They obviously are aware. They just put all that so they don't get "canceled". It's just virtue signaling and covering your ass.

Well we will be able to generate satirical political images of politicians, of religions easily and quickly. So there are some upsides for the technology to be given to every comedian out there. Politicians and religions, did have an easy ride the last few years, so we must set the record straight from now on.

Some fear porn was thrown out when GPT-3 was released. I love GPT-3, i used it just yesterday it is very good. I am just wondering when the total destruction of the world will happen because of GPT-3.

> Politics will be influenced. You can't trust anything you see anymore

How much could you trust media content previously? Staged footage, false narration, biased coverage are nothing new. A counter-intuitive side-effect of opening the Pandora's box could be a realization that media is a form of simulation. Perhaps this will lead more people to filter what they see through a prism of critical thinking.

We do a decent job of banning child pornography.

And bringing the two ideas together, is child pornography that is provably created by an AI still illegal?

As far as I'm aware countries fall broadly into two camps. Camp 1, USA for example, is concerned purely with the abuse of children, i.e., anything that depicts or is constructed of pieces of real children is illegal but other things such as drawings, stories, adults role playing, etc is not. Camp 2 outlaws any representation of it whether or not a child was involved.

Nowhere will a training set featuring pictures of naked children be legal.

>Nowhere will a training set featuring pictures of naked children be legal.

True, but generalizing beyond the training set is precisely the point of machine learning. A good generative model will be able to produce such images, no matter how heinous the content is.

> Nowhere will a training set featuring pictures of naked children be legal.

Appropriately from the recent news stories, but it's easy to imagine at least portions of such pictures being available for medical diagnostic purposes. I've sent pictures of my children to my doctor, so presumably in the future it's easy to imagine sending pictures to an AI to diagnose which would require a suitably fleshed out (pardon the pun) training set.

I imagine the AI which diagnoses your children won't be the same AI your neighbours use to make pictures with.

Actually ... we just did a good job of making generated child pornography ubiquitous.

I think the actual problem is that this gives plausible deniability against photographic evidence, which might result in increase of bad behavior. Even cameras which cryptographically sign their output can't prove that the input was actually photographed from the real world, or if it's just an image of an image.

> you can fully expect your asshole friends to grab a dozen photos of you from Facebook and then make a hyperrealistic pornographic image of you with a gorilla

... Someone is gonna do this to children. This technology is gonna end up on the news. Maybe they'll even try to ban it.

Seems less likely that everyone will "just be cool with" having pornographic images of themselves with gorillas spread everywhere and more likely that this is the impetus for demagogues to issue a digital ID for access to the internet.

Isn't it good if people learn not to trust photos and images shared in social media and learn to treat them as entertainment. So much productivity gains :-)

I had a thought for a utopian/dystopian world.

One day these image generators will also get video support... and pornography support. When that happens, a few things may occur that I think are reasonable to predict:

EDIT: Original post was way too wordy, TL;DR:

When AI-generated pornography becomes available, it could be likely that demand for "real" pornography disappears because the AI will match and surpass the "real." When that occurs, the "real" will become increasingly regulated and legally risky, and may just outright be as good as banned.

There will be a huge and unkillable market for non-AI generated pornography, even if people cannot tell the difference in an AB test. The demand will be too strong and I don't think there will be much outcry to ban it if it's all consenting adults.

Curious as to how you arrived at that conclusion.

A quick Google shows an estimate that it was a 100 billion dollar market in 2017. Seems there is a lot of demand.

But what portion of that demand is specifically for artisinally handcrafted authentic sex rather than for apparent sex acts?

Oh sorry, the OP was worded differently when I saw it. That's not what I meant.

I didn't edit my post if you were referring to me. Either way, I don't have a strong argument for why I feel this to be true, but I do think people are going to scoff at ai generated porn.

If people can't tell the difference in an AB test, how will the real porn out compete the generated stuff? Porn distributors aren't known for their truth in advertising or care in sourcing of material. And even if they were, how would the porn distributors be able to source the real stuff, when anyone can create porn of anyone they imagine? You might say PKA will save us, but people aren't going to be typing out `gpg --verify` when their hands are otherwise occupied.

Setting norms of asking folks not to abuse it reduces the likelihood people will step out of line with those norms.

Corollary: If you did not make it, someone else would have anyway.

You’re right. They call this an “ethical release”, but what ethics, I may ask. A profitable IPO is more likely to have been their consideration. As other researchers before them, they are willingly releasing something with the potential to do harm, or pave the way for it, washing their hands in innocence.

Reply on news.ycombinator.com