Hacker News Reader

Comment by emptysongglass | Hacker News Reader

He's a self-appointed expert who is neither an AI researcher, computer scientist, or any other accredited title you'd expect someone loudly yelling, “WE'RE ALL GOING TO DIE” on a technical topic like the fundamentals of AI and its dangers to hold.

You and others continue to foist human qualities on LLMs because it looks like magic. To know is to know oneself. LLMs do not know themselves. In fact, there is no “self” to know.

View on HN

I don’t know much about him but I thought he was a researcher? There is an “Academic publications” section on https://en.m.wikipedia.org/wiki/Eliezer_Yudkowsky and it says “He is a co-founder and research fellow at the Machine Intelligence Research Institute”

LLMs are not self aware, I agree, but there is clearly some reasoning going on and you could argue that its model of human behaviour can somewhat simulate self awareness

Being a self-appointed expert goes a long way if you're willing to work the character. A combination of information overload and a tendency to give people the benefit of the doubt - especially if there is (as yet) little competition for whatever niche you're trying to occupy. people are predisposed (and media people almost pathologically so) to assume benign or benevolent intent in the absence of better (or any alternate) sources of truth.

To wit: I'm cofounder and senior researcher at The Center For Decentralization. We have business cards, phone numbers, offered a free consultation service to budding decentralists who were interested in shaping public policy, a social media presence, the whole deal. For a little while we even sent representatives to Congressional meetings and had an office in DC. I have no official credentials in this area. And I'm also the only co-founder, because a decentralist mustbtake care to avoid any language that makes it sound like you're the center of the thing you're trying to decentralize. Which, naturally and inescapably, you are.

I started TCFD as a parody to point out the fundamental paradoxes arising when one affects to be sponsoring or causing decentralization to happen, whether as an individual or a group of individuals. I thought the joke would be obvious to anyone the moment they saw our business card or heard the name, but was rather astounded to to discover that about half of the people I introduced myself to as "senior researcher, primus inter pares" completely took it seriously.

and you didn't even bring up "I would never join a group that would have me as a member!"

> its model of human behaviour

What model of human behaviour? I asked ChatGPT if it has a model of human behaviour, and it said "ChatGPT does not have an explicit model of human behavior".

As I just said ChatGPT is not self aware, so it can’t answer questions about it’s own workings like that. (this is somewhat true of humans too though, there is a lot going on in our subconscious, but we are aware of our own conscious thoughts to some extent at least)

It just hallucinated an answer

If it didn’t have a model of human behaviour, then it wouldn’t work, because the whole idea is it’s simulating what someone acting as a helpful assistant would do.

Self-awareness isn't a requirement for the presentation of information, which LLMs continually prove.

Is every answer thus not a hallucination by this logic? If it's trained on external information, why would this information not include observations and information made about it? A human doesn't need to be able to even read to present a book to someone else with contextually relevant information.

What I meant is that it can’t self-introspect and work out how its own thought processes work. It can only know that if it was in its training data or given to the model in some other way, it’s not (yet?) possible for it to know about its inner workings just from working it out

But this is true of humans in many cases too

It’s training data is from 2021, so it won’t contain anything about how ChatGPT works, maybe a bit about how LLMs in general work though

At some point it will be trained on data that describes itself, which will make it self-aware in a way that will probably prompt much argument about what precisely we mean by the concept of self-awareness.

Logic systems from the 1980s could explain their reasoning.

I think it's still debatable if there are any AI researchers at all because ML is not AI by a long shot. Also I think he considers himself more of an AI philosopher than a "researcher" anyway, and that, in my opinion, gives his words as much weight as any others'.

... ML is quite literally a sub-field of AI. That's quite literally the MIT definition of it.

Allow me to go hypothetical for a moment: do you really believe that the emergence of AI will come strictly from ML advancements?

I do not, and based on the direction of ML progress from the last couple of years I am becoming more and more certain that it will have a low impact on AI, if/when that materializes.

Therefore I won't scoff at someone that's not involved in ML but still considers themselves as being part of the AI world, at least not without more evidence that they're an impostor. Take John Carmack for example, at which people look down in similar ways, because has no formal training in ML.

I think AI field at present is in an pre-Enlightenment period, where people studying astrology, alchemy mix up freely with astronomers, chemists and philosophers and it will take some time until we can discern the useful from the useless, or from the outright harmful.

I don't think AGI as we fantasize about will ever exist, period. Not unless we completely change our computational paradigm to at least partially biological or quantum.

https://www.npr.org/sections/health-shots/2022/10/14/1128875...

This has my attention, particularly from an ethics perspective in terms of slavery.

Reply on news.ycombinator.com