s1dechnl

> I'm not sure what you're arguing but it seems like my key point wasn't communicated well.

It was and quite well. Were speaking the same language. We just have different conclusions.

> Yes, every narrow AI is a system of systems to an extent. So expand on that concept but outside of a single firm/system. Such that the self driving car system is one single solution path solving "transportation" which would comprise automated flight/rail etc... and is a node in a larger general system - like hub and spoke.

And you still have nothing more than a hub and spoke system of systems authored for specific problems spaces and you're spokes will increase with every new problem space until you overwhelm your hub. A horrible architectural approach that if not caught in the initial stages will result in catastrophe down the road... Weak AI is weak AI no matter how you scale it.

> You seem to be arguing that a single entity will fail if it attempts to take a narrow AI system and make it generalizable. Of which I am in agreement with.

This is a start in the right direction...

> If however there were 10,000 or 100,000 or 1,000,000 narrow AI companies/systems (like a self driving car system or alphago etc...) those could fill the corpus of solutions which an executive function system could utilize depending on the application and together they would be what we call AGI.

No, its strung together weak AI. It will require significantly and unreasonable amounts of resources. Its capability will increasingly reach diminishing returns and you'll end up with a frankenstein monster code base that no one can manage or understand.. Sounds a lot like the path Weak AI is already heading down.. At such a point, it's best to just scrap it and start all over. Something that Hinton and other prominent figures are finally admitting. Something I concluded year ago which lead me down a different path. Now, you're more than welcome to state : Well hey man that's your opinion and you're wrong and I'll wish the 10s,100s, million of narrow AI companies the best just as was conveyed to me a umber of years ago. Weak AI is Weak AI. It is a class of optimization algorithms. You can jerry rig this all you want.. You still have nothing more than a system of systems of optimization algos. If you think this is what intelligence is, I'm not sure what to say.

What I stated was that there isn't even a sound/reasoned example as to how a it would escape its bounds unless purposely authored to and even then you run into the scenario of human limits (its creator). Also, having no understanding on how AGI is structured, it is quite foolish to talk about bound leaping in the traditional sense of tech which is mainly associated with computer viruses that are purposely written for that sole purpose. There's nothing to even suggest you can purposely author AGI in such a fashion. So...

> How would it start, etc.

There has been no credible framing on how it would even start. As such, it's a null point of discussion.

> If you can't define it, you can't analyze it sort of thing.

With no definition, you can 'attempt' to analyze it and you'll likely be horribly off the mark, wasting tons of resources, and likely produce something that has no bearing on the real thing. Instead of admitting this, it's like people put full faith in these efforts being sound when the reality is the exact opposite. Why engage in this, unless your aim is profit/notoriety, when you could be working on the actual problem? Define intelligence [first]. Time to be honest with oneself. Time to stop projecting ones shortcomings on others. Time to stop using fear/uncertainty/doubt to obscure one's true intent. Time to stop pushing the disinformation cloud. Time for TRUE intelligence.

> But what will stop them at the beginning?

Themselves of course. You are after-all your own worst enemy. I don't think people appreciate the incredibly slim number of individuals who are actually capable of pursuing this problem through to fruition and what their natures necessarily have to be. Do people not know of the types of individuals whose work is still being dusted off and relabeled as modern day achievements by others? The thinkers and creators of the previous AI age had certain characteristics. Certain character flaws and natures preclude you from probing certain subject matter. If you don't know much about yourself as a human being and your creation... If you haven't conquered aspects of yourself, how do you imagine you magically become capable of replicated it another domain? You don't. It's a non-starter which is why there is so much wheel spinning going on even with billions of dollars of resources and top ranking engineering talent. I really don't think people appreciate how fundamental AGI is. Look in a mirror sometime. All the answers are right there if you can find them/recognize them. Always has been.

Firstly, there are fundamental limits that all things are subject to in this universe. When you push towards these limits, you discover this...

Developing AGI pushes certain limits. There aren't many who have this capability as it requires a vast range of understanding/know-how across an incredible number of domains and into ones that have yet to be discovered. There are multiple disjoint leaps and barriers. In order to make and surmount them, you yourself are subject to certain considerations/restrictions. I resolving these, you end up w/ a less free and more controlled system.

> improve faster and execute more efficiently.

This occurs with order not chaos

> If an AGI can indeed learn and act at the human level or above, there is reason to believe that a more free variety will improve faster and become more powerful than a less free one.

There is no reasoning to suggest this... quite the opposite actually. Also, you're mistaking the capacity for learning with the successful execution of learning. Chaos leads to destruction not boundless construction. That being said, there is order to all things.

> That is a big incentive for its creators to let go of some control. The question is how much control they would retain and whether that would be sufficient.

There isn't any magic going on... You really need to convince yourself of this. You're speaking of this technology as if it gets booted up today and eclipses all human intelligence. It doesn't occur that way. A human being will have to teach it and guide it and therein lies the same control you have today. Of course, there are further steps because you have the capability to see exactly what's going on inside. You'll really have a hard time establishing a case for doomsday scenarios. Also, a destructive/chaotic individual is necessarily limited by their own flaws such that they wouldn't be able to conceive of the underpinning necessary to develop AGI. So, I hate to tell you this but the hollywood image of such people is wrong...

Both of you are correct. One need only have understanding. You must maintain a yet to be discovered understanding of the human equivalent and have a depth of understanding in computational systems. While the understanding is non-trivial, the translation effort from one domain to the other isn't much effort. Computing resource capability scales with $$. I decided to do something unorthodox and start with limited computational resources. It drives the innovative spirit ^_-. If your processor is to 'slow', you can simply create a simulated abstraction of time and go from there. Computational power doesn't bog down/limit the effort, one's own understanding of the problem space/domains does.

You are correct. You don't need to do a whole brain simulation to achieve AGI. Instead, you need to understand an incredible amount about its processes, design, and overall nature. At which point, you need to translate this into the computational domain. A lot can be 'left on the table' so to speak. The fundamental problem arises with how deep your understanding is so as to know which parts you can leave on the table and which parts you can't. As far as then putting this into a functional computational system, you would need an extensive knowledge in this domain as well so as to know how to structure the software to best exploit the hardware. Lots of prototypes, performance testing, scaling, etc until you have a sound 'feel' of what you can expect and where things and you need to go. That being said.. Yes, I can run my stack on a consumer grade CPU/GPU. I have designs for hardware architectures that don't quite exist yet but all of that can be emulated in software. Latency is the only consequence of current hardware which can be trimmed with effort. Latency when too high can be abstracted with time scaled simulation. So, there is absolutely no blocker in hardware for developing AGI and yes it can be done on affordable consumer hardware .. If GPUs don't continue to be resigned to ponzi schemes and RAM prices come back to earth. That being said, if I need to, and if things don't change down the road, I'd be more than happy to spin my own hardware to keep costs in order.

Sure. I am Jovan Williams from : http://www.monad.ai. I know its true because I am sitting in front of working aspects that I have been researching and developing full time for approximately 4 years. Some years ago, I was operating a full stack on a 4-core Intel processor. While resources were pegged, it only contributed to increased latency which is why I created my own simulation layer to continue my proofing work. I utilized various software to create a simulated virtual environment w/ time scaling and continued w/ my work.

I charted out some ways hardware had to evolve over the years. So far its beating my estimates. I conducted a simple upgrade to my hardware a year ago and saw various latency figures cut in half which is exactly what I estimated. The industry continues to push hardware towards capabilities that will only increase performance.

So yes.. Currently, I can run my stack in real-time on an 8-core consumer grade computer. I have already proofed it beating human response times in various tests by 100%. This is before any specific optimizations. Structure matters [Software] and I have other unspecified hardware in the loop. I intended on applying to Ycombinator in march, but will extend this out a bit. I'll be going more public with proofed functionality and capability. Just making my rounds in various communities/mediums as I have been doing for some time to correct the record and get a gauge of people's sentiments.

> From what I gather from your comment above and another one, the AGI you talk about does not include general intent or free will to act differently from what its creators anticipate.

It actually does and its actually the nature of my work. That being said, I still have a high fidelity of control near complete absolution. I can still set immutable laws/restrictions and prevent undesired behavior.

> I believe a genuine AGI can behave outside of our predictions in multiple ways, unless we truly solve the problem of constraining its will to a range that is acceptable to us.

It sure can. However, it cannot act beyond laws/restrictions that I set forth and indeed my control functionality centers on very deep percepts. If you have a crappy architecture/limited understanding, you end up with an overly complex, flawed, and limited control algorithm... One that can be even more complex than the underlying system it attempts to control. This is evident in : Weak AI. It is not the case in AGI at least not in my work.

> I do not believe anyone has solved that

No one in their right mind has published it... As it's valuable IP and has substantial power [which is why it shouldn't be publicly disclosed].

> I agree that time is required but an AGI can multiply itself and collect all requisite information from the world quite quickly.

Incorrect. It cannot do so unless its creator has allowed for it to do so. In the case of it being allowed, what hardware does it migrate to? It needs to be provided by its creator(s). Hardware takeover? Sorry, this is again sci-fi fantasy. Are you able to take over someone else's body/brain in totality? No. Same rule applies here. Let the fantasy/fear go away. There is no grounding. It's a position pushed by people hoping to falsely profit/gain attention/get article clicks...

> There is so much available to learn just from the Internet if it knows how to learn independently like humans do. Computational resources might be a bottleneck but presumably a human-level AGI can at least do online work at a minimum wage (e.g. translating documents, simple accounting, ...).

Sure. What's the problem with this? Its progress can be overseen, audited, and/or halted at will. So what's the issue here?

> It can parallelly execute many 'brains' to accomplish more work to acquire more resources, to do more work profitably....

You're drifting back into the flawed fear/uncertain/doubt armageddon scenario. It cannot execute on anything other than the hardware I consign it to just like you. If I decide to scale it, it's what I decided. At any given point in time I can halt it or power it down... Just like any program/computational system today. So, what's the issue here?

> Humans are limited by 24 hours a day. An AGI can over a fairly short amount of time (months) accumulate sufficient resources to make thousands or millions of its copies perhaps with variations to specialize for different kinds of work. Over time, it should gain experience to perform more and more highly-valued work as well.

Repeating the same flawed scenario doesn't make it true. See answer above.

> A smart AGI can form alliance and share benefits with inside players and execute any cunning strategies not available to humans who at least need to take into account law enforcement. There are many other advantages for an AGI over human organizations (some of which I mentioned above).

All of what you mentioned above was debunked. If you have a more sound proposal for how this could occur, I'm all ears. Alliances can't occur without human intervention. None of these systems are connected and there is no sound argument for 'viral' takeover. Your scenarios are flawed and you've been infected by the : fear/uncertainty/doubt propagandist who structure ventures to take advantage of the wallets/attention/mind share of people who buy into this nonsense. Focus your attention on the problem of intelligence [first]. Until you grasp a sound understanding of it, all of this theoretical hand waving is all for not especially as its not grounded in anything possible in the real-world. Put your engineering hat on if one is avail. Less theory and more practical grounding. Life isn't a fantasy level dystopian sci-fi movie and its sad that certain people have created this image so as to profit. Talk about [cunning strategies] [manipulation]...

> Has the industry always been able to prevent smart, persistent actors from breaking the access locks? Why should we assume that an AGI which can accumulate experience over time, gain more knowledge, and make connections with others, including human actors, will not ever be able to break the locks?

Yes, the industry has persistently been able to do this. It's why the whole world isn't falling apart as we speak. What limits the locks most often is cost not capability. As such, you are possibly mistaking one's business decision not to use a more costly lock for the lack of capability to create a capable lock. Furthermore, you mistakingly attributing the actor in this case. The actor in the case of AGI is in a carefully controlled/monitored box. Actors in the real-world are not. As such, please tell me how an absolutely monitored/restricted actor has the ability to go playing with locks that aren't within its reach? I have a more fundamental question even : Have you been able pick 'your locks' yet? Do you even know what they are? Where they are? Those capable of 'creation' hold certain things close to their chest .. The act of creation necessitates it and is [built in].

> Why should we assume that an AGI which can accumulate experience over time, gain more knowledge, and make connections with others, including human actors, will not ever be able to break the locks?

Show me how you're able to break your 'locks' and you'll have an argument for how AGI can break its locks. I don't think you're grasping the level of 'locks' that I'm speaking about. Humans have been around for how long and still don't even know what their [locks] even are... or where they are. It's quite easy to example at a certain level of visibility how your scenario is unwarranted. I can draw direct parallels to eons of human history.

I disagree. The naming convention [Artificial Intelligence] still is a large shoe that these purpose built applied engineering solutions have yet to fill. Meanwhile, for profit/notoriety/marketing people want to trample on yet another name space? What you just described is essentially the architecture of a self-driving car. It's yet another applied engineering solution of Artificial Intelligence. It is not Artificial General Intelligence. Scaling/Distributing the computational space of an applied Artificial Intelligence solution is not Artificial General Intelligence. This is the same thing that lead to optimization algorithms being called Artificial Intelligence. If you aren't able to maintain foundational distinguishment, you lose track of what you're searching for and trying to achieve. Outwardly, you capture more money and attention. Inwardly, you become unraveled and lose your capability to solve the elusive problem. Eventually after much fame, wealth, and feigned 'success', one asks themselves : Was it worth it? Depends on what your original aim was.

I would just like to echo this comment as it exactly and indefensibly strikes out Weak-AI from Artificial General Intelligence. I fail to see how specially crafted optimization algorithms continue to receive higher attribution. First, they were renamed to Artificial Intelligence. A great deal of cash exchanged hands based on this attribution. To retain the spirit of the true definition of Artificial Intelligence, the naming convention : Artificial General Intelligence was crafted. Even then, the allure of cash/fame progresses to taint that whereby people are calling expositions of Optimization/search algorithms Artificial General Intelligence. It saddens me that mankind has progressed the ubiquitous and beautiful information age into the disinformation for profit age from top to bottom and bottom to top.

> That's quite a bad argument when it comes to "existential risk for all of humanity".

It's not a bad argument. It's the standard for creating one. Engineers get paid to resolve risk/problems. So, if you claim there is one and it is fantastically an 'existential risk for all of humanity', one should at least be able to explain how exactly it exists in the most basic technical terms. I haven't heard a single example of this and in my work it was resolved from focusing on the hard problem : What is intelligence?.

> Of course, so far no one has any idea how pragmatically it could play out.

Then there is no argument. Let it go. What you don't understand is that millions/billions are at play over this argument. Humans love to make up fantastical nonsense/games for profit. You're being played. If someone truly claims they care about this issue, why haven't they done the first step? Define how exactly this pragmatically could play out. Were talking about people with PhDs, industry titans, and millions if not billions are at play and you're telling me that between them all they can't come up with a signle pragmatic way this scenario plays out? It's a scam that's why.

> Maybe it'll start with something as banal as VisualStudio + Cortana providing an advantage to AI design, that helps designing a better Cortana, that helps designing a better Cortana, that ...

Garbage in garbage out. Weak AI in Weak AI out.

> Yes, of course, it's unlikely in the coming next 1-2 decade, because the best "AGI" we know (the human mind), is a hodgepodge of specialized faculties jury rigged together by ancient code lossy compressed and stored in a multi-stage bootloading wetware that takes years to learn reading the magnificent ~26 letters of a latin alphabet (but similarly takes the same time to learn walking, talking, hearing, rudimentary thinking, a minimal concept of self, a theory of mind and so on, and for some reason it has a shitty API, it must go through all of the aforementioned to even begin learning chess).

And if I told you the foundation of AGI has already been defined, the core computational model proofed, and that a functional system exists today how would anything change? AGI is here and there is no doomsday playing out. It's running on an 8-core processor, a GPU, and 32GB of RAM completing interactive exercises and going through a series of tests defined by its creator. The bowl of spaghetti above your head was decoded and a functional equivalent instantiated in computer hardware. The locks are compiled binaries, read only code regions, and a power button.

> So far AI is focusing on computer vision, some NLP, (strategy or easy to evaluate) games. At least the current progress/results are clustered around those areas. We don't know when will be the next big leap, maybe in knowledge representation, or goal formation, maybe in other kinds of very useful, more general intelligence related areas.

Instead imagine a child AGI with no bearings on the world... The human equivalent eons ago... Being carefully cultivated and learned in a controlled environment. No takeoff intelligence scenario. No end of the world. A child simply stumbling its way through controlled scenarios being oversee and tweaked by its creator. Not so scary huh?

It is difficult to make arguments in something that has no grounding. I'd expect someone who claims to have a valid argument for an [intelligence explosion] event to have formal education and industry experience designing computational systems such that they could clearly define how exactly it could occur. I have yet to see such an individual w/ such a viewpoint. Instead, I see a ridiculous argument being forwarded by people who are least informed/experienced so as to push fear, uncertainty, doubt either for profit/attention or because it fulfills some sci-fi oriented religious prophecy. Internally, companies require extensive and well reasoned documentation before funding an initiative. Externally, someone w/ no expertise/proof throws their hands up in the air speaking about armageddon and they are able to secure considerable attention and money.

[AGI Developer]

I'll instead give you something to consider : * The totality of mathematics is not fully defined. As such, there are can be completely new mathematics that destroys previous conceptions especially as it relates to the limits of computability. Theoretical Limits often reflect one's limited scope of perception. Your limits or the limits someone theorized ages ago aren't necessarily my limits if I find a path beyond them.

Information theory and computational complexity theory are theories... Solving fundamental aspects of intelligence will invalidate a host of theories, create new mathematics, new algorithms, and new theories. Continuing to frame something as ground breaking as an understanding/implementation of intelligence in yester-years theoretical limits is flawed.

A series of connected games [society] is constructed by various groups in mankind. They all have common traits. A human being having no understanding of intelligence creates a measure of potential success in these games [society]. It is called an IQ test. It has nothing to do w/ actual intelligence as there is no fundamental understanding of it. Instead, it correlates to a potential for success in the games certain individuals have constructed. Change the game and the potential for success does as well. Far too much weight is put on the intelligence Quotient especially given a lack of understanding as to what intelligence is and the clearly skewed games mankind creates [society].

Also, I'm still confused as to what you mean by : AGI concern. What's the concern? What's the big fear?

[AGI Developer]

A controller/overseer can easily limit/block this sufficiently and securely. Were talking about hardware/software. There are systems/standardized approaches to solving this problem. The 'Control/Safety' problem for AI are lauded as theoretical and new. However, they are not. They are solved by industry standard approaches day in and out. Any seasoned/experienced engineer in this field could solve this with known approaches.

> Since an AGI can copy itself and be available at a multitude of access points at once

Same comment above applies. This can only occur if done by a controller/overseer. Real-life isn't a sci-fi movie... There's engineering involved.

> AGI changing x,y,z

Not possible unless it is given access. Solved easily in industry standard ways.

[AGI Developer here]

> the real barrier for AGI is software and not hardware.

This is indeed true. It has actually been true for some time. AGI is computable on present day hardware if one has enough knowledge on how to correctly structure it. A fundamental understanding of intelligence is the first step. How one crafts this understanding into software is the second step. Hardware reached capability in recent years.

> exploding intelligence

There is no such thing. Time is still required like with all things. Teaching/learning/interaction is still required. Furthermore, A controller/overseer of the system can more than adequately limit progress they are not comfortable with. I find the idea of exploding intelligence/overnight super AI to be pure fantasy not at all aware as to the structure of AGI.

> If an organization can marginally predict stock price movements better than the rest of the world..

The problem is this kind of thinking... AGI is achieved and people rush to apply it to games to get rich. Sorry, that will not occur. It will not occur because the stock market is fundamentally a [game]. A game with disadvantaged players. A game with incomplete information. A game whose rules/dynamics change frequently to suit inside players. One could make all of the accurate predictions they wanted, if the game changes underneath you or before you can act, your lofty predictions have no real world value and that's exactly how the market behaves.