Hacker News Reader

Comment by jameshart | Hacker News Reader

Every time an AI model is released, people on this site show up to complain about how it has been censored and bent to some puritan agenda.

‘Why,’ they ask, ‘would I want a politically correct AI that’s been nerfed so it can’t offend anyone?’

Well, this is what happens if you build real applications using insufficiently censored AI.

This is why OpenAI and Google and stability always explain how they have restricted training content and aligned the output to avoid producing controversial output.

Because they want people to build applications using this technology and those people do not want this headline to appear about them.

View on HN

Each product has their own value proposition. I don't think anyone opposes that each business has control of their own AI pipeline, the crux of the problem is at what stage of the pipeline this censorship is injected, specially in open models.

Of course businesses have their right to release AIs with censorship, and we as costumers have a right to challenge that and ask for a product that is actually useful to us.

Saying it again: If a business releases a model, we have a right to demand that it is useful for us because we're the business' costumers.

If those demands aren't met by a business, another business should be able meet them in open competition. And this only happens when open uncensored models are not banned by law.

> ...specially in open models.

> Saying it again: If a business releases a model, we have a right to demand that it is useful for us because we're the business' costumers

Open models == free == no customers. For open foundational models, yours sounds like a demand that someone else invests millions in training a foundation model of your liking, but you're not motivated enough to fine-tune your copy of the model for hundreds or thousand of dollars?

If you're not paying for the "censored" proprietary models, you are not a customer either. If you are a paying customer of "censored" models, and would like them to change, remember businesses have the right to choose who they want as customers - and they can decide they don't want the "uncensored" market segment. I'd love a $25,000 electric Ferrari too, but Ferrari is not about that.

You're right, businesses can choose their market segment, there's nothing wrong with that. They just have to bear with the loss of costumers when the segment they were aiming for makes demands they're not prepared to fulfill.

In your example, it doesn't have to be Ferrari specifically. Someone could make an equivalent product at a lower price even.

But I have something else to say about your comment. It's that nothing is entirely free. Google lets us use their search service, but we're paying with something else than money.

When a business "gives away" an open model "for free" it's only on their self-interest, and this does not free them of the expectations/responsibilities with their costumers. No matter what form that self-interest takes, we're still their costumers even if it's not money.

So if they release an open model subpar with what we expect, they're gonna end up losing.

> If a business releases a model, we have a right to demand that it is useful for us because we're the business' costumers.

What makes you their customer? Are you offering them money for access to the uncensored model you say you want?

Do you have some business that you’re going to be able to build that tests on access to an uncensored model, and their failure to offer that to you is preventing them from getting paid by you to provide the underlying technology for your venture?

Because on the other hand there is a state lottery over here who absolutely is willing to pay someone money for a safe, censored model. They are an actual paying customer with a real use case and cash to back it up.

Of course, it turns out the AI they ended up using didn’t (if this story holds up) actually meet their expectations with respect to safety and alignment - but let’s be clear: they wanted a safe, censored model.

I find it hard to believe that the market for uncensored models - LLMs that occasionally output 4chan style racist tirades, diffusion models that occasionally generate pornography - is actually worth addressing, compared to offering models which don’t do those things.

> I find it hard to believe that the market for uncensored models - LLMs that occasionally output 4chan style racist tirades, diffusion models that occasionally generate pornography - is actually worth addressing, compared to offering models which don’t do those things.

Well I think that's for the market to decide, yes? If state intervention is out of the question, obviously.

> but let’s be clear: they wanted a safe, censored model.

That's absolutely right.

This confuses at least two things. Censorship is about intent and enforcing a world view. If I want a model to generate something for me and it doesn't because of censorship, that's one problem.

If I want a model to generate a fun pic of me at the beach and instead it makes a nude photo, that's completely different.

> If I want a model to generate a fun pic of me at the beach and instead it makes a nude photo, that's completely different

What if someone else's idea of a fun pic at the beach is a nude photo? Is it only censorship when the worldview being enforced is not aligned with yours? A truly uncensored model should accommodate everyone, but society will (and should) absolutely burn down any AI service that accidentally generates nudes from a beach photo of a family that has minors. "Intent" is a human attribute, AI is probabilistic.

This abundance of caution by the companies avoiding nudity is unfortunately read as promoting a worldview or pushing an agenda, while it's just risk mitigation.

Yes, but most people share the world view that a state lottery should not generate unsolicited lewd images of their customers.

This suggests there is a large, and perfectly reasonable demand for censored models that adhere to broadly ‘SFW’ cultural norms.

So why are people upset that commercially available models generally try to cater to that obvious need?

Way to completely misunderstand the issue at hand.

The issue isn’t “we’re mad at them for subscribing to a New Puritan agenda (although there is one playing out); the issue is that right now there is no way to censor the model for “undesirable” features without lobotomizing the acceptable content as well.

It’s not just removing nipples from training. We have evidence that the “safer” the models get, the worse they get.

The more heavy handed the weights are by manual influence, the less they are producing valuable output.

The cost of performance for everyone is directly coming at the cost of the perpetually-offended, and I suggest why bother because those people are never satiated. When you make your identity about being offended, there will always be something that is offensive.

I thought the issue at hand was that a state lottery generated fake porn of a customer?

Typically replies are referenced to a comment.

But… The spur here you are cha going the topic to is a side effect of government employees not knowing that these models aren’t yet so lobotomized that it is still possible to create “problematic” imagery.

I’m just saying that I don’t think the lottery player who got sent fake porn of herself is necessarily a member of the ‘perpetually offended’, to whom you’re ascribing the demand that model outputs be restricted.

I think she’s part of the class of ‘actually offended’. This image wasn’t ‘problematic’, it was potentially actionable.

And when you are a business, actually offending your customers is normally not a good idea.

Striving to create software systems that avoid actually offending people is a good idea, even if it comes at the expense of some other benefits. Whatever ‘performance’ we apparently give up by training systems not to generate nude photographs has to be weighed up against the risk that it might do so at an inopportune moment.

You take out the nipples and vaginas and you lose a large portion (most?) of human advertising.

Regardless of their intentions, current state of the art automatic censorship have dangerously bad false positive and false negative rates. So, they inadvertently and inevitably suppress the truth and broadcast misinformation. Their performance on images is similarly bad.

Of course, even if they did work, centralizing control of them is a bad idea. The end user should have the freedom to control such things. (In the same way you can type swear words into a word processor).

In practice, these AI models are centralized services. Few people are running them locally on their machine with knowledge of the training data, weights, or system prompts. There isn't a direct line between "user intent" and "system output," unlike the local word processor in your example.

Someone has to make value judgments on the underlying data and algorithms, if only to build the system in the first place. If users can't (or don't care to) run these models locally, who else will decide other than the service operator?

‘The censorship work done in AI alignment is insufficiently accurate to rely on for production use’ is a good and valid position

‘Therefore AI vendors should release uncensored models and let everyone else figure it out’ is not a logical consequent from it.

‘Therefore don’t use current AI tech to build apps that photomanipulate user submitted content and echo the results back to them with your logo on it’ would be a more logical conclusion to draw.

Thankfully that is how a lot of AI is working right now, open source models for both LLM and Stable Diffusion let people have access to uncensored reasonably decent quality AI tools.

In other categories of AI generated content though, such as speech and music, the open source alternatives are either lacking or completely absent. And with the company behind Stable Diffusion being on the verge of financial collapse, we may see Stable Diffusion fall far behind it's closed source counterparts.

Reply on news.ycombinator.com