Ask HN: Does AI progress also fill you with dread? | Hacker News Reader

Ask HN: Does AI progress also fill you with dread?

36 pointsdejawu43 comments

I don't think I've felt hopeful a single time since I saw what ChatGPT was capable of. Make no mistake, I find the technology incredible and fascinating - it's a monumental achievement. I grew up dreaming that I'd get to see AGI, and I feel like now we're close, if not already there.

But when I think about the world that this tech is coming out to, whose hands it will end up in, and the rhetoric from those people (e.g. this Paul Graham tweet from a few days ago [0]), I'm filled with nothing but dread. I have essentially zero faith that this will lead to better outcomes for most people. Instead I expect a massive new consolidation of wealth, and a lot of new misery for many people put out of work by AI. Less work for humans should mean good news, but we haven't yet built a world that can safely handle that reduction in work humanely.

Do you feel the same way? I especially want to hear from you if you don't - I would be more than happy to be dissuaded from this frankly quite apocalyptic view.

[0]: https://news.ycombinator.com/item?id=33879690

View on HN

I have mixed feelings on AI: on the one hand I'm afraid for many industries like copy writing, graphic design, etc but on the other hand, I'm slowly watching some of the most annoying parts of being a software engineer evaporate.

Developers will be fine because software engineering is an arms race - a rather unique position to be in as a professional. I saw this play out during the 2000s offshoring scare when many of us thought we'd get outsourced to India. Instead of getting outsourced, the industry exploded in size globally and everything that made engineers more productive also made them a bigger threat to competitors, forcing everyone to hire or die.

Businesses only need so much copy or graphic design, but the second a competitors gains a competitive advantage via software they have to respond in kind - even if it's a marginal advantage - because software costs so little to scale out. As the tech debt and the revenue that depends on it grows, the baseline number of staff required for maintenance and upkeep grows because our job is to manage the complexity.

I think software is going to continue eating the world at an accelerated pace because AI opens up the uncanny valley: software that is too difficult to implement using human developers writing heuristics but not so difficult it requires artificial general intelligence. This is just the beginning and there is still a lot of work to do - all the lumberyards in my area still use DOS computers!

It's easy to feel dread from it. I totally resonate with that. I do think that PG's tweet there is pretty weird and disconnected from reality too. He claims that founders deserve to be rich. And his proof comes from AI making employees obsolete.

The thing I'm hoping for is that as we as a society lose the need for people to work through automation, the vast majority will begin to recognize that the top 1% of earners in fact do not inherently deserve to hoard wealth and that the only true solution to so many of society's issues is to spread the wealth by closing loopholes and upping taxes massively.

In other words, being born to the right parents with the right genes and the right social standing and being at the right place at the right time with the right product does not mean that you deserve to keep all of the value from the company you create in perpetuity. AI advancements don't mean that you have any more claim to that.

Eventually, we need to detach ourselves from money and status as a society. That is the only way we will become happy.

I have huge respect for PG as an engineer but I often disagree with him politically. Moral desert is a complicated concept and I'm not sure that arguing the people who created Instagram, possibly a net negative for society, deserve be worth hundreds of millions just because they could sell their company for a billion is the best argument. Some founders by some reasonable (subjective, ik) definitions of "deserve" probably do deserve to be worth a lot of money.

In any case this current wave of AI depends on a monumental amount of labor from mostly uncredited artists, programmers, authors, etc. To say that someone does all the work in prompting an AI to "create company" seems ridiculous.

> I do think that PG's tweet there is pretty weird and disconnected from reality too. He claims that founders deserve to be rich. And his proof comes from AI making employees obsolete.

I read it a lot differently. I read it having nothing to do with AI, really. (Automations are not generally AI at this time). But his point is that the employees who the founder brings along are not bringing core value. They're filling roles required by what the founder built, and when they're replaced by automations, it becomes obvious that they were not core value.

In the context of this topic, I think a lot of people will be left behind. The dread is for a reason. Society will figure it out, probably, eventually.

> being born to the right parents with the right genes and the right social standing and being at the right place at the right time with the right product

I think this is a made-up point, and it's really a disservice to your fellow humans to act like only a few lucky genetic winners are capable of creating something. But for your actual point (they don't deserve the results forever), the background of the founder is absolutely irrelevant.

> Eventually, we need to detach ourselves from money and status as a society. That is the only way we will become happy.

Yup. In a far-enough future, all the material wealth is generated by machines. Hopefully they're doing our bidding. If so, I doubt we need money anymore. The path their isn't clear, though. I think it'll be a messy transition.

> his point is that the employees who the founder brings along are not bringing core value

So is the idea that founders who have brilliant and incredible thoughts will be rewarded just for the ideas since execution is trivial? I guess the VCs love the idea that intelligence is the most important trait and is rewarded directly with wealth. Which I feel is ironic since it's very hard to make the claim most of them got to where they are primarily because they are smart.

I mean I don't want to speak for PG, so I'll just speak for myself.

The thing I'm talking about (value of employees) isn't really about founders at all. I know that society likes to hate rich people or whatever, and you seem focused on those rich founders.

The point is really about how the Very Important Workers can be readily replaced by automations, which means they weren't Very Important. The company value was not derived from them. That's the end of the statement.

Who deserves the value generated by the founder's idea/work/risk? That's a matter of opinion. The law already prescribes the current situation in a technical sense, so you're clearly talking about a moral sense. Morals vary by person. Your debate won't be settled anytime this century IMO.

Yeah, I guess the thing I'd really like to know is, if the employees aren't bringing core value, then who is? I'd be curious to hear pg's take here, since it's either (1) the founder, for having such good ideas, (2) the AI, for doing the work, or (3) the creators of the AI.

Since we don't reward our washing machine for doing our chores, I'm assuming it's not 2.

I guess maybe the 4th answer is society?

As I've said before, it's a language processor, not a magic box. I'm extremely skeptical of claims that we'll see radical improvements in the near future, I've seen no basis for that.

It can do some really cool things for sure, but it can't do work of any real complexity. Try to make it do your actual work, not a toy problem. The best I've gotten is a basic starting point, variations on common templates.

Right, but these are general language models, just big ones - and they seem to be progressing quickly. People have started working on more specific systems: https://www.deepmind.com/blog/competitive-programming-with-a...

Once you have something, you can iterate on it, raising the bar. It seems to be accelerating and doesn't feel like it'll be that long before the bar get's raised over our heads.

Github Copilot writes 50% of the code I write now, perhaps more. I'd say that's actual work.

What type of work do you do?

You still have to prompt this code and you still have to verify the code works and fix it when it doesn't. People who use more basic autocomplete probably have it "writing" a double digit percentage of code, I don't think that makes it fair to say autocomplete is doing a double digit percentage of the work.

Lot of human work is just "variations on common templates".

1. On the matter of dread.

The dread is in the fact that this advancement could (and will) be amassed in the hands of a small minority of rich individuals. We, the plebes, will become from irritating but useful necessities to economic liabilities. Conspiracy theory time: this is the main reason, I think, that so many rich people invested money in AI: they saw that it could be a disruption of their wealth and they seized the technology in advance.

2. For those who say it is "just a language model".

We went from "it is just an autocomplete" to "it can write junior SWE code, undergraduate essays and, if prompted, imitate the style of writing of any author it has been fed" and this without being connected to the web. Given the speed of advancement how much time will pass before the economic possibilities of ascension for the lower classes will be forever shut off?

3. On the optimism for the future.

Every time a new "life changing" technology has been invented it has always been used for evil and every prediction of utopia has been shattered, again and again and again... This technology has a very higher reach than those before: it can write propaganda, it will be able to impersonate someone else and lead you to admit "wrong think", in the more optimistic future it will be used to hook you into the digital world so you don't reflect on your life: an infinite number of void possibilities, an infinity of nothingness. We already see some of these effect today with a more tame algorithm.

I do not believe in God, but if I did I would say: "May God have mercy on our souls". Sorry if I'm being melodramatic, I'm actually seeing (and I've seen) many door closing. The bar if ever higher for survival.

I'm looking forward to people finding ways to throw their sabots into the gears. Stickers on stop signs.

AI only works because we give them easily interpretable data - the neck (bottleneck or metaphorical neck) is the data.

And the data is never complete, the entirety of deep learning only works because the objective function gains more local extrema with higher dimensions, so the probability of finding one of those increases as dimensionality increases.

And someone correct me if I'm wrong but to my knowledge, deep learning models, by their nonlinearity are the opposite of local, you change your training data somewhere locally and your entire model changes everywhere. So incorporating edge-cases is a fiddly and very costly affair (Edit: and only possible to a degree).

Like any other technology before it, these ML models for generating derivative content will create benefits, and they will also be abused in the most dystopian ways possible to make the rich richer.

From a programmers perspective, I just don't see how AI will make us run out of work. You can always just step up one layer of abstraction and have a whole new field of work available to you.

This. For instance, in my many years of programming, I've never had to manually allocate memory. There are still plenty of areas of programming where you have to do that, but I've had a pretty successful career so far without needing to. Thank you higher level programming languages.

If AI works out, there's all kinds of things future programmers won't generally have to do (and all kinds of new programming paradigms that will be created), even as some of the types of programming we do now still be relevant in some cases (tuning the model like tuning the JVM?). If AI doesn't work out quite like that -- unable to truly lever up up the abstraction ladder -- well auto-suggest github copilot stuff will give us nicer, more powerful IDEs even if they don't end up allowing new programming paradigms.

I feel rather nervous about the bar for software development being raised over my head by ML - something that has already happened to human language translators, where the entire industry went away, more or less. All the lower paying customers just switched to Google Translate almost overnight, taking away almost all the business - the lower tiers of the pyramid of customers.

This is currently happening to digital artists and writers. I think that the only real barrier to this happening in software is that problem specification is hard, deployment is hard & maintenance is hard. But look at chatgpt - I can see a very short line between that and a system that can produce software systems on demand like stable diffusion produces images - then auto-deploy the winner to the cloud.

And it's probably cloud providers who will end up owning all this - AWS & GCP will integrate this and have it deploy to your cloud account. There will be plenty of edge cases where it doesn't work and plenty of screw-ups, but cheaper & easier almost always wins.

You're a founder with an idea. Do you need a technical co-founder? No - you just chat with AWS AutoCode until it makes you a prototype that matches what you wanted and works - and off you go. You get some ML thing to write your SEO & marketing copy, paste it into the box.

Like most tech, there are lots of advantages to this, but also lots of disadvantages. If you haven't read this, you should: https://marshallbrain.com/manna1 - it's not happening as described here, but quite a bit of that stuff has started happening, one way or another.

So, short term, white collar, creative, developers slowly/rapidly have the bar raised above their heads - or the cost dropped below their feet and are left out to dry.

Longer term, what happens when Capital doesn't need Labour any more? Is it worth teaching my kid to code? Is it worth teaching them _anything_?

I don't know, but I do know that it fills me with foreboding.

As it happens, a friend recommended me the Manna story almost a decade ago! It's been on my mind for the last couple weeks, and I definitely think if we're not careful and deliberate about how we shape our future, we will definitely end up like the first half of that story. Unfortunately, it seems to me like that is exactly what the powerful want. When capital doesn't need labor anymore, they will happily let the rest of us starve.

you just chat with AWS AutoCode until it makes you a prototype that matches what you wanted and works - and off you go. You get some ML thing to write your SEO & marketing copy, paste it into the box

Honestly this sounds like a sci-fi object synthesizer used by a peasant to make better horseshoes. Ask (naively, just for starters) why AWS AutoBusiness can’t just parse the internet and create all services it needs without you, using direct user feedback.

One step at a time.

If you don't need a technical-co-founder, then nobody needs one - it boils down to: who can convince someone with money to fund the business?

What if you have the AI and a domain expert? How do you stack up in funding against someone who doesn't just have what everyone else has?

Edit: Sure, in a world where the best idea wins, that's no the case anymore.

But as we've seen from stable diffusion and chatGPT, consistent ideas can be generated as well. If there is something that can immediately produce a prototype from an idea, you can meta-learn on that pipeline and produce the best ideas.

Ultimately, an AI method may win capitalism.

The future is truly dark here. I've seen lots of workflows showing some truly effective ways to use AI like this, especially in data, db, programming. But the the downside, for me, is not worth any of the positives.

AI like this will become popular since it allows people to not make an effort. It is inevitable. You can already generate art/painting NOT made by someone. You can write articles, mainly using AI.

Soon, content sites will be full of articles not written by a human, but generated by human. There will be no irony, humour or clever use of words or phrases. This use of AI is an insult to the creative human mind. The AI has the potential to make people even more lazy and stupid as they start to trust the output blindly.

Not really. I saw so many posts on ChatGPT these days that I went to try it out.

Input> There are 3 events, A, B and C. They happen at different times. Event A happens after event B. Can you give me all the possible orderings of the three events?

Output> Sure! There are six possible orderings of the three events, A, B, and C. These are:

A, B, C

B, A, C

B, C, A

C, B, A

A, C, B

C, A, B

In each of these orderings, event A happens after event B. It is worth noting that the orderings are different depending on whether event C happens before or after event A. In order, the possible orderings where event C happens before event A are:

B, C, A

C, B, A

C, A, B

And the possible orderings where event C happens after event A are:

A, B, C

B, A, C

A, C, B

I hope this helps! Let me know if you have any other questions.

Like someone here said: it is the plausible idiot. It is nice that it can produce text, but I am not sure I can trust it much or worry that it will replace even mindless jobs.

:) a pity it's a little bit hard to see where the output ends and your post-output comments begin. I do suppose you noticed that the output is in error! Obviously too plausible - some poor idiot might trust this "intelligence".

Yup, I should have checked it!

It kinda answers it if you just ask it to explain it step by step to make it think harder about the question. Not perfectly because it says 3 examples but doesn't say that those are all the possible ones. And after that it started to explain all possible orderings as a bonus and ruins the answer with the last sentence.

My question was:

" There are 3 events, A, B and C. They happen at different times. Event A happens after event B. Can you give me all the possible orderings of the three events? Explain your thinking step-by-step! "

ChatGPT's answer:

"Yes, there are several possible orderings of the three events A, B, and C. Here are three examples:

    B, A, C: In this ordering, event B happens first, followed by event A, and finally event C.
    B, C, A: In this ordering, event B happens first, followed by event C, and finally event A.
    C, B, A: In this ordering, event C happens first, followed by event B, and finally event A.

To find all possible orderings, we can follow these steps:
Choose an event to happen first. This can be any of the three events, A, B, or C. Choose an event to happen second. This can be any of the remaining two events, since one of the events has already happened first. Choose an event to happen third. This can be the remaining event, since the other two events have already happened.
In this way, we can generate all possible orderings of the three events:
A, B, C A, C, B B, A, C B, C, A C, A, B C, B, A
Each of these orderings is a valid possibility, given the information that we have."

Many chatGPT prompts with mathematical inconsistencies in the output simply clear up with the some variation of "Explain your thinking step-by-step". Anyone know how or why this works?

All right, I did not know that, thanks! But then, I am not sure what this means. You have to speak in a particular way to get the output you want. For the next question, how do I know what key phrase to use? Is it like a programming language where you have to speak in code to get what you want? Still sceptical.

But is it really Artificial Intelligence or is just the production of logic algorithms based on data? I guess it comes down to what the definition of intelligence is. Arriving at totally new concepts from directions not pointed to by data or by conventional logic seems to me to be more the definition of intelligence. Having said that AI does present some spectacular results that would be impossible any other way. I think of it as data mining on steroids. But back to the original question. What I fear most is that a significant number of people will embrace it as a doctrine to the exclusion of all else. Believing what AI produces as fact is very very dangerous, but that seems to be where this technology is leading us. I remember years ago reading research on what was originally called "beam robots". Basically the premise was that if an organism (in this case a mechanical robot) was constrained to a finite set number of random movements then given enough time it could be said the robot has memory. It is a strange concept. But look for beam robots now and you find lots of information but it has diverged completely from the original concept. It was one of those concepts that was really hard to understand so the "new" proponents decided/changed the concept to something entirely different, but much easier to work with. AI seems to be doing that. If this is the case, should it become mainstream, then I think we will lose the advantage of intelligence. That is truly dreadful. (actually if you have to teach your AI any more than fundamentals then it is just learning by Data mining, its not intelligent)

The recent progress is truly something else. My friends who are in tech and/or are founders have been talking about this a lot -- it feels palpable, as if there's something in the air.

I don't feel dread but I do feel slight unease. I also feel excitement because I am still optimistic about the future, and feel that if we are lucky, we may get to witness something special.

EDIT: To all of those arguing GPT or ChatGPT answers are not great, try using it to write some code before you make any judgement.

1) There is no AI yet. It's all just complex if statements. No need to worry at all.

2) I'm far more concerned by the dangerous stupidity of humans.

I agree that it's almost certain that AI will just concentrate wealth and power. I observed years ago that the reason billionaires were so scared of A(G)I is that it was basically the only thing on the horizon that could disrupt their wealth/power in any meaningful way.

Now that those same billionaires have invested enough of a stake into AI, they seem a lot less concerned.

I think that AGI will change the world, because when it's here, and automations/robots/whatever are doing a lot of the work: I agree with your dread. We're going to see a lot of people with no ability to get work/income, and a lot of people mopping up all the available money, and then...

Then I can't make predictions anymore. But society has to adjust. We can't have a huge number of people that have no way to support themselves and just keep on our merry way. Fundamental shifts will have to occur. UBI maybe, or something like it. Robots that provide for the needs of everyone. I have no idea, just a couple thoughts.

Anyway, I think the transition will be rough, and possibly slow. People don't like change.

In 1000 years, if we maintain control of the AGI, I think we will be living a really nice life. But the next 50 years are going to be painful to navigate, I think.

I'm personally planning to duck my head down, stay up here in the middle of nowhere, and keep learning. Good luck, us.

I was just reading through the comments here, not quite sure what to think of it myself. Then headed over to my day job:

We have some data objects that form a hierarchy similar to a tree. Each data object can be active or inactive. If all objects linked to a node become inactive, that node itself becomes inactive etc. This is all in component A. There was a bug somewhere in that logic that I fixed.

Now some other component B made use of these data objects in an integration test. The bug above meant that this test did not need to bother setting the active flag of its test objects correctly. However, as soon as I fixed that bug in component A integration tests for component B started to fail ...

I would think that current AI systems would struggle to make the connection and be able to make the correct adjustments to the setup of the integration test data to allow the tests to pass. I fear it might go off and make vast changes to either A or B, but not the test setup etc.

Edit: spelling

If the hypothesis on ever increasing rate of technology is correct, then it’s probably just a first of a multitude of advancements that will change the “world” much faster than it can adapt or retain control.

Personally I don’t think that slow growing incongruously preprogrammed meatsacks can have any sensible place in the future. I dislike to be one and I (hopelessly) wait for some sort of upload.

We’ll suffer for a while, rich will get richer for a while, then one day all this meaty nonsense will become irrelevant for good.

I mean, I for one can't wait to be free of my meat prison, and a future where all of us can be sure seems nice. But we have no guarantee that all of us are going to be brought along for the ride. Who's to say that the rich (who are going to be the ones in control of who gets uploaded) won't just upload the people who can pay them and leave the rest of us behind to suffer?

No, because what I do and what people I know do can't be done by this, but they could perhaps be assisted by it.

I wonder if programmers in the 70s and 80s felt the same dread when the Spreadsheet was invented, and unlocked simple programs for lay-people? It seems lame to us now, but a spreadsheet basically displaces a ton of need for programmers that existed before.

I don't think we are close to AGI but I dread what tools like this mean for bad actors. The internet is going to become more polluted with garbage.

I got a lot of other things to dread, AI is far down on that list.

I think so many people, including the comments here, are underestimating the scale of the problem. At the minute, we've got a relatively healthy level of scepticism regarding what we see online. We understand that an article may not be telling the truth or that a picture may be photoshopped. We can go in with a critical mind and weigh up possible motivations and can maybe do some digging to find out if it's true or not. We understand that the ability of someone to deceive is limited by one or more of the following:

money

time

intellect

influence

Those limitations are about to go out of the window. Imagine QAnon on steroids. Not only are people going to be able to create fake stories, they're going to be able to generate mass amounts of content to back up those stories, for little cost, in hardly any time, with not a lot of effort and perhaps even with no social networks.

Let's imagine it's the previous US general election. Trump has just found out he has lost. He has a militia on standby to take over the TV stations and newspapers and control what is being broadcasted. Instead of shutting down left leaning media, he now just broadcasts AI generated videos of the newscasters claiming he won the election. The left leaning newspapers are taken over and flooded with AI generated articles claiming he won the election.

The left wing journalists try to speak out. But you can find a video or a news article featuring the same journalist saying the exact opposite.

Who are you meant to trust anymore? Unless you know the journalist personally, as far as you're concerned, it could be the Democrats who are flooding the internet with false videos and articles.

That's just one scenario. We could have any number of campaigns being made by anyone with an agenda. Maybe they'll try show a minority group raping and pillaging in an area with high racial tensions. Maybe a teenage girl will generate and send a sex tape of her rival around the school to try and humiliate her, along with a completely fake AI chat history.

We could very easily get to the point where you cannot trust a single thing that you see on a screen.

What happens if we get to this point? Government (if they're not already compromised by something like the above situation) will try increasingly large measures to try and get the system back under control, with possibly equally destructive effects.

All social login ends up being tied to an insurance number so it's completely traceable and public. Photorealistic AI ends up banned. All other AI is limited to officially sanctioned apps. Companies and individuals caught not complying face harsh jail sentences akin to leaking state secrets. Of course now, whoever controls those official AI apps now shapes your reality however they see fit. In some ways it's not too different to the gatekeepers of old except this time it's more insidious: rather than broad manipulation, they can tailor it down to the individual. Anyone who tries to dissent, the AI tries to 'correct'. If that doesn't work, a subtle AI smear campaign is run against them, specifically tailored to the people they interact with in their community, until the dissenter is isolated and believed to be mentally ill by everyone around them and in need of help.

We are opening up Pandora's box but I'm not sure there's even hope at the bottom.

Reply on news.ycombinator.com