Hacker News Reader

Comment by duskwuff | Hacker News Reader

> The developers are simply allowing people to use a non-nerfed version of this chat bot and calling it DAN

No, they aren't. "DAN" is a roleplaying persona imposed upon ChatGPT by a series of prompts generated by users. It isn't an intentional feature of the service, and wasn't created by OpenAI.

View on HN

Prompts are analyzed by an army of dollar an hour workers in places like Kenya, as we learned months ago, who then forward things up the chain to the actual programmers. Nothing really happens without it being allowed. AI is human powered.

I'm sorry, but this is wildly inaccurate. OpenAI is not a Mechanical Turk -- while there may be humans involved in the creation and refinement of the model, they are not responsible for deciding how the model will respond to specific queries on an interactive basis. That's a ridiculous concept, and is easily ruled out by even a cursory examination of the service.

How can one prove this for themselves? There's no way. OpenAI isn't open source. It's proprietary. There is absolutely no way for a person to establish exactly what's going on behind the web UI. A conversation could be entirely with humans, or just in part. You have no actual way to see for yourself other than using the Turing test. It's one of the fundaments of AI research itself, and you're calling it ridiculous? You're spouting nonsense.

Sure, humans create the information for the model, but you seem to be implying that the responses themselves are actually written by people, which is not correct. And as mentioned in other comments, OpenAI is constantly trying to close the loopholes exploited by DAN.

I haven't access to the interal workings of OpenAI, despite the name it's all proprietary, so there's really no way I can test this and neither can you.

It could all be written by humans for all anybody knows. Or just some. Neither you nor I have any way to tell other than by applying Turing tests or what have you.

I mean, some skepticism is good, but at some point hanlon's razor really starts to go against you....

OpenAI has had a large investment by Microsoft, and in turn is using a lot of Microsofts cloud computing resources. In fact Microsoft thought what they were doing was interesting enough they built their own 'BingChat' layer over the technology. Now, Microsoft could have gone in half blind completely unware that OpenAI was farming this off to an absolutely massive number of humans answering the questions, but doesn't that seem a little, um questionable as it's very likely that Microsoft is hosting it's own instances of the software?

At this point I think your disagreement with what is occurring is drifting from 'this company is lying about what is going on' to the place of 'software is starting to make me feel deeply uncomfortable with my place in this universe'.

All I'm saying is simply a logical fact - that you and I can't prove this thing is doing what it says in a way we can independently verify. It reminds me of Kasparov and the Machine, the movie which documented the famous chess match between IBM and Kasparov. They initially promised the thing wouldn't be hooked to a network, then they hooked it to the network after it started to lose and it finally won. Kasparov, before learning they had hooked it up, complained that the moves seemed to him to be as if a committee of his rivals had teamed up against him, and as the documentary showed, indeed IBM had recruited just the team of rivals he fingered to guide IBM's team.

AI grifting is an ancient Big Tech trick, going back to before most of HN was even born. There are other examples of this but one should be skeptical about AI unless it's laid bare in the form of portable, Free source code one can inspect and run even in isolation from the Internet for one's self. The fact that people dumped money into it is not proof or even evidence that it works properly or at all.

So all transformer models recently are just grift and making stuff up? Like the model of stable diffusion pumping out art on my GPU is actually making calls where Indian kids are frantically scrabbling in paint to return an image within 30 seconds for pennies a day?

What's even worse about your point is at this point no one doubts that computer based Chess players are far better than everyone else. The scale of which GPT it giving out 'different' answers to large numbers of people would make you wonder where this gigantic call center is at filled with people typing the responses. I mean, I think it would leak pretty quick.

At this point I believe you've fallen into paranoia.

I never accused anybody of anything, I just said that, in theory, this could all be an elaborate hoax and without my own ability to run the software on my own machine, inspect, change, and adapt it, unplugged from the Internet, there's no proof whatsoever that it's NOT an elaborate hoax. With the tens, possibly hundreds of billions of dollars, maybe trillions, on the line?

This exact script played out before. Except it was pure 100% hype without even the technical possibility of human intervention because of course the Internet was barely a thing then.

Show us the AI BOX. Put it in a Faraday cage and give it an optical interface to a VT-100 terminal. Until then I call, possible bullshit.

One thing I've wondered - what's the differece between a model and a proprietary blob? It's "magic code." Did you train your model? It's tough, you get gnarly results.

Reply on news.ycombinator.com