Fear of sentient robots | Hacker News Reader

Fear of sentient robots

allartburns.org

26 pointsiamwil17 comments

I remember my college days in the early 1970s at a liberal California university when computers were continually portrayed as soulless monsters having no redeeming value. That was before the personal computer. Same functionality - same destructive potential as before, but now seen as friendly toward human advancement. It is all a matter of perspective - computers themselves are a tool, one that can be used for good or bad.

The corporation is impersonal by design. It is a structure designed to facilitate enterprise.

It does so: 1) by giving an enterprise perpetual existence; 2) by allowing investors to put up risk capital without having to risk their personal assets in the venture; 3) by separating management from ownership such that the operations of an enterprise can be flexibly handled; 4) by creating discrete units of ownership that can be used to create special incentives for founders, investors, employees and others; 5) by creating a vehicle for widespread public ownership of the enterprise.

Is this good or bad? It is all a matter of perspective.

Consider that, without the corporation, we could not have the modern startup. Try to gather a serious team together without being able to promise them limited liability should the venture fail. Try to raise funds for your sole proprietorship from strangers. Try to offer equity incentives to employees when all you have to work with is a family business or a partnering-type vehicle. You will not get far.

Robotic or not, a corporation is a legal tool that serves an important purpose. Yes, it can be used for good or for bad. Just as with the computer, it all depends on the people behind it.

a corporation is so much more than a legal tool because of the effects that emerge from it. a simple example is your number 3 - a description of the conflict of interest inherent in corporate structure (and the source of much controversy lately).

while a corporation is a collection of people if it is successful it buries every one of them, until the end of time. the behavior that emerges from the simple act of self-preservation over time is complex and very different from entity to entity (see evolution).

more personal, accountable structures do not require feudalism or regress. what they require is communications technology like that we are utilizing right now (and real names help).

Fascinating idea of having more personal, accountable structures - is there a write-up somewhere of what legal form these might take? Or are you talking about LLCs and the like? I have been doing startups for a long time now and cannot imagine them (nor most other forms of enterprise) without some form of limited liability vehicle.

Or is it the idea that communications technology will somehow transcend the form of the structure and lead to better results simply because people relate to one another differently?

I am asking this sincerely. I guess my imagination does not easily comprehend the potential alternatives. If you can give me some pointers to discussions on this, I would appreciate it.

I have often seen corporations criticized but have never heard of any formal structure along the lines you describe (again, unless you mean LLCs or the like).

fair enough q but the legal structure of organizing people is a reactionary force and well behind the changes taking place in society (for instance web businesses with distributed operations still need a shopkeepers style business permit in some jurisdictions - "the business has to be somewhere where do they get their mail?").

in the case of corporations and their modern alternatives tax-engineering is the driver more than the things we are talking about.

w/r/t liability the corporation is an extremely inefficient mechanism for the elimination (or lately society's assumption) of the liability. the underlying risk of the entity has no bearing on the price of this risk (a break even bomb factory pays less than a profitable pillow store). the increasing importance of director's insurance shows the market has a way of addressing this and may provide the harbinger (if not the "write-up") you seek.

I am referring to the risk that investors face of losing all their assets (beyond what they invested in a company) in case the company fails. I don't understand why people would put up money for investments in such cases. Are you saying that insurance would cover them?

Not to belabor this but the idea of investors being shielded from losses beyond their risk capital was what gave the corporation its huge expansion over the years. That is how public companies came to be formed, for example.

Still not understanding how investors would be motivated to invest in most companies without such protection.

its pretty simple, corporate ownership/government fiat isn't the only way to protect investors that aren't willing to assume risk. you've done a good job of deconstructing most of this now just look at the pieces and see they are separate.

the most fearsome self-sustaining entities are also self regulating, they are called governments. legacy interfaces like representative legislatures and voting have limited influence on their path towards omniscience and omnipresence.

Pretty dissapointing. I was hoping for a paranoia-filled full-on psychosis, serialized to HTML, and all I find is a bait-and-switch.

Look, if you want to rant about corporations, make scary movies about them. And no, the corporation from Alien doesn't count, the scary part was the Alien, not the corporation.

In the mean time, I'm going to stay right here, work for a corporation, and keep winding more copper for my EMP.

This is a thread that's reflected extremely well in Charles Stross's Accelerando. It took me a couple of readings of the novel to really make the connection between intelligent corporations in the story and what's actually happening in the real world.

Accelerando is one of the strangest books I ever read.

The 'rapture of the nerds' is a theme worth exploring (the singularity), but I've never given it much credit. After reading accelerando I'm not so sure anymore.

It certainly is food for thought. Given the pace of progress in technology in the last 30 years or so I'm pretty ambivalent about how fast it goes. Sure, there is lots of movement but I don't see anything that is really mindblowingly new. Networks existed, the internet is already decades old.

We're able to communicate faster and better, the cost of a bit has decreased further and further.

But we are not much closer to for instance real AI than we were 30 years ago (only then we were much more optimistic that we'd have it in 30 years).

>But we are not much closer to for instance real AI than we were 30 years ago

This is very hard to evaluate. It seems unlikely that progress towards strong AI will be linear - and I'm not basing that on Kurzweil or claiming it will be exponential, I just don't see how the notion of making steady progress towards strong AI even makes sense.

That's a good point.

Ok, how about this then, if IQ would be some measure would you accept that even lower animals have an IQ of sorts and they can solve simple problems on their own ?

And that we still can't get a computer to solve anything on its own without very precise instructions ?

A first step towards AI would then be to get a computer to be goal driven enough that it could solve a simple problem without it being foreseen (and the solution spelled out) by the programmer.

It could still be years before that would lead to a true AI in the sense that it could do something that we could not but it would be a step along the way.

I think genetic algorithms have been able to surpass that standard a long time ago.

While the developer has to define the playing field and the rules, the solutions the computer comes up with are way better than randomly searching through the solution space. check out http://www.popsci.com/scitech/article/2006-04/john-koza-has-...

Of course that is no way near strong AI, it is still a step in the direction.

And we are working on simulations of the cortex of simple mammals. (read a bit on Blue Brain).

I am not saying we're close to strong AI, but we are making small steps in the right direction, and if philosophically speaking you are a materialist like me (believe that consciousness can be explained in theory using nothing but physical/chemical terms) than it is just a matter of time until it happens.

True, genetic algorithms are definitely a step in the right direction. There is the one famous example of the filter circuit where even the designers of the software couldn't figure out how the damn thing worked, but work it did. Parts connected in the weirdest ways and if you took them out the circuit stopped to function. Apparently the genetic algorithm had in its feeback loop discovered some none-trivial parasitic effects of some components. No sane designer would have done it that way, but that's beside the point because the filter had less parts than any designed by humans directly.

I'm very strongly a believer in the material consciousness,

The emperors new mind was an interesting read but I found it to be a tremendous buildup to an almost magical reliance on quantum mechanics to provide us with our consciousness.

I don't have much with magic so I'll stick with regular physics and chemistry until it has been proven otherwise.

There are too many brain-computer analogies to avoid the strong suggestion that the brain computes. It doesn't do it in a way that we can hook up a debugger to just yet but it does seem to have a lot in common. It's more of an architecture/interface problem.

But strong AI probably won't come from being able to interface with the brain or from a sudden leap of insight in how it all works. I think strong AI will follow from a seed AI which in turn will follow from some relatively simple breakthrough or insight that we still haven't gotten around to.

The simulation of the cortex of simple mammals is another such step in between, do you have a reference for that ?

I wasn't talking about brain computer interfaces, although that is an interesting field. Note to self: try our algorithms on EEG data...

My point regarding the materialism was that if you can simulate the physical/chemical processes that happens in a brain, and I am talking about the molecular/quantum level, the result will be indistinguishable from the original.

The reference you asked for: project Blue Brain by IBM http://online.wsj.com/article/SB124751881557234725.html

I meant that to figure out how the brain works we will have to figure out it's architecture first, then we will be able to hook up to it to refine that knowledge.

Thank you for the link, I'll read it today. The rate at which hacker news produces good stuff to read is getting higher than I can read fast!

In addition to corporations we also have humans. Breaking news, 100% of murderers are human!

Reply on news.ycombinator.com