Hacker News Reader

Comment by v7x | Hacker News Reader

>Also, tackling safety and bias in the models is a key focus area for us, and given the challenges related to this, we are not currently releasing an external research demo.

Read: We don't want the internet to turn our program into Tay 2.0 because we are super duper serious.

I get why these companies do this. But I can't help but feel like attempting to eliminate "bias" and tackle "safety" are counter productive to the goal of developing softwate desinged to mimic a species that is by and large biased and unsafe. Purposefully hindering a program because you're afraid it might produce results is not a healthy method of development, especially in an area like AI where we are still working to understand the basics of the field. Let the programs "learn" what's objective before trying to constrain it with the subjective.

View on HN

I suspect that the real reason is because non-cherry picked answers don't work as well and that if you interact with it you'll be no more impressed that decades of chatbots that aren't all that much more impressive than ELIZA.

I think part of the reason is that chat bots tend to be racist without any intentional effort to make them so. I saw a post last year where someone built the most default language AI setup using the most popular training data and then got it to rank words based on their positivity. Without any intentional effort the AI ranked English names as positive and African/Asian/etc names as neutral or negative words.

Maybe there needs to be a trail rating for chat bot safety. General public gets the bunny slope conversation, Tracy Morgan gets the double black diamonds. You work your way up by not attempting to cancel the project for something untoward that was said.

Maybe the last thing the human race needs is a machine that behaves like a human with the speed and precision of a computer?

These measures don’t seem to be too onerous, certainly less than what a government might come up with after the first Desaster following your approach.

But it seems that’s not your most salient concern here, considering the scarequotes around „bias“, and the reversion of objective nd subjective you introduce.

> we are not currently releasing an external research demo.

Without an external research demo this is all just hype. It is pretty easy for your bot team to come up with and release conversation where your bot does really well. Having it work in the real world outside the research lab is the big issue.

Disclosing their architecture and current thinking is not "just hype".

I guess I get the bias issue (though difficult to remain neutral on some subjects.) But safety? It’s a bot. No matter what it says to me there is no way I’m going to feel unsafe. I mean ok, don’t doxx people (I hope that’s kinda default behavior though).

"Our bot keeps asking for us to connect it to the internet and suggests that it'll be able to learn faster that way. We're really excited about this development, but a good fraction of our developers have started running around screaming about the apocalypse, so we're postponing public release until the chatbot tells us how to calm them down"

Maybe not you, but explicitly encouraging others to take harmful steps is a potential outcome.

For example, there's a clear safety issue with a conversational bot that tells you to kill yourself if you talk to it about feeling suicidal or depressed.

What if you talk to it about immigration and by picking commonly upvoted statements on certain online communities it starts to encourage violence?

There are clear ways to me that a chatbot can have safety issues.

That’s a decent argument but then should we go to Hollywood and sanitize their violence in their movies (even those wanting to teach us about the evil of violence) for fear they might incite the wrong people who see such as calls to violence?

What I mean is that that possibility exists in other media yet we don’t feel ambiguous about it.

First of all, Hollywood does occasionally sanitize films and TV shows. For example, the suicide scene was removed from "13 Reasons Why" after concerns from mental health experts that it might inspire copycats [1].

But even ignoring that, I think most people have a good understanding of what a movie is and that fictional characters are not real. Movies have been around for a long time, most people have been watching them their entire lives. I don't think at this point that most people have nearly as good an understanding of chatbots. If you can converse with one and it mostly delivers responses like a person would, some people will start treating it like a person and ascribe values to its statements, and that's where it potentially gets dangerous.

[1] https://www.nytimes.com/2019/07/16/arts/television/netflix-d...

So you’re saying people can tell fiction from reality in film (even if film re-enacts actual events), but can’t use that same construct to evaluate a “bot”?

Maybe... but it seems like a bit of a stretch.

People feel sad when their bomb disposal robots die. https://www.theatlantic.com/technology/archive/2013/09/funer...

We're just not great at keeping what we know and how we feel in line.

People absolutely do alter their output based on how they think others may act, Hollywood is a prime example where this happens because films even get age ratings. It's more extreme really because it's outside agencies and governments putting the pressure on what is deemed allowable.

What we're talking about here isn't someone stopping others from releasing a bot but stopping themselves until they're happy with the potential risks.

> Let the programs "learn" what's objective before trying to constrain it with the subjective.

What is "objective" and what is not is not always something that is obvious and implementable in code. The hot topic of course is politics and religion - feed a bot the speeches of Donald Trump and another bot the speeches of Bernie Sanders, both will not be seen as "objective". Train it with both and the bot will still not be seen as "objective".

I'm glad you mentioned Tay.

When I need a laugh, I just read about Tay running amok:

https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch...

https://en.wikipedia.org/wiki/Tay_(bot)

Reply on news.ycombinator.com