1,142 karma · joined March 18, 2023
Please don't cite the CDC for anything, they're utterly discredited.
"Why the WHO took two years to say COVID is airborne
Early in the pandemic, the World Health Organization stated that SARS-CoV-2 was not transmitted through the air. That mistake and the prolonged process of correcting it sowed confusion and raises questions about what will happen in the next pandemic."
Companies did exactly what the government wanted them to do: make hand sanitizer as fast as possible and at massive scale. The lesson here is about leaving disaster response to governments. The FDA had a bunch of tactics it could have used. It could have continued to do inspections and changed the rules to say if you don't pass you have to display a notice of that prominently on the bottle. It could have kept the rules and just let companies scale up within the limits.
Or most obviously, it could have bothered double-checking and realized that hand sanitizer wasn't going to do anything about a virus that spreads via cloud-like aerosols. The fact that SARS-CoV-2 spread this way was widely known to independent skeptics by the time the company in the article was applying for permission to make sanitizer. What happened on board the Diamond Princess cruise ship demonstrated that the virus must be spreading through air ducts, rendering lockdowns, hand sanitizers and masks all useless. It was also expected because SARS-1 spread this way too, and because masks had never had an impact on respiratory illness before either.
All this was ignored because the medical establishment didn't know what to do if this fact was recognized. They preferred to tell people to engage in pointless and self-destructive behavior rather than say, ok, maybe there's nothing we can do except let doctors get on with it.
This aerosol denial was never formally ended and so large parts of the population is still unable to process reality even today, especially in academia and healthcare. Instead they invent ever more bizarre and desperate attempts to try and explain the abject failures of all these mitigations.
Here are two graphs from the same government agency (NASA), measuring the same thing (temperature), for the same time period, drawn twenty years apart. The data has been fundamentally altered such that the story it tells is different:
https://realclimatescience.com/wp-content/uploads/2019/08/NA...
The 2000-2015 pause is the same. You've been told that only "unorthodox" people were talking about it. Here's a brief history of the pause, as told by climatologists publishing in Nature.
https://www.nature.com/articles/nature12534
https://www.nature.com/articles/nature.2013.13832
In 2013 we read that "Despite the continued increase in atmospheric greenhouse gas concentrations, the annual-mean global temperature has not risen in the twenty-first century". The IPCC 2013 report is reported with the headline "Despite hiatus, climate change here to stay". Climatologists claim that maybe the missing heat has disappeared into the oceans where they can't find it.
https://www.nature.com/articles/nature.2013.13832
Two years later everything changes. "Climate-change ‘hiatus’ disappears with new data", there's a new version of history and the past 15 years are gone:
"That finding [that global warming actually did happen], which contradicts the 2013 report of the Intergovernmental Panel on Climate Change (IPCC), is based on an update of the global temperature records maintained by the US National Oceanic and Atmospheric Administration (NOAA)."
Climatologists have an interesting methodology - when observations don't fit the theory, they conclude the observations must be wrong and go looking for reasons to change them. Given a long enough search they always come up with something that sounds vaguely plausible, then they release a new version of the old data that creates new warming where previously there wasn't any. Although this isn't entirely secret they also don't tell people they're doing this, and the media certainly isn't going to tell anyone either.
And that's how it goes. You look at the edited graphs, remember people talking about a pause and think golly gosh, how badly informed those awful skeptics were. We have always been at war with Eastasia!
Where we depart is point 4. Actually, both point 3 and 4 are things I agree with, but it's implied there's a logical link or progression between them and I don't think there is. The problem is the definitions of "outperform humans" and "smart".
Current AI can perform at superhuman levels in some respects, yes. Midjourney is extremely impressive when judged on speed and artistic skill. GPT-4 is extremely impressive whilst judged on its own terms, like breadth of knowledge. Things useful to end users, in other words. LLMs are deeply unimpressive judged on other aspects of human intelligence like long term memory, awareness of time and space, ability to learn continuously, willingness to commit to an opinion, ability to come up with interesting new ideas, hide thoughts and all that follows from that like being able to make long term plans, have agency and self-directed goals etc ... in all these areas it is weak. Yet, most people would incorporate most of them into their definition of smart.
Will all these problems be solved? Some will, surely, but for others it's not entirely clear how much demand there is. Boston Robotics was making amazing humanoid parkour bots for years yet the only one they seem able to actually sell is dog-like. Apparently the former aren't that useful. The unwillingness to commit to an opinion may be a fundamental trait of AI for as long as it's centralized, proprietary and the masses have to share a single model. The ability to come up with interesting new ideas and leaps of logic may or may not appear, it's too early to tell.
But between 3 and 4 you make a leap and assume that not only will all those areas be conquered very soon, but that the resulting AI will be unusually dangerous. The various social ills you describe don't worry me though. Bad governments will do bad things, same old, same old. I'm actually more worried about people using the existence of AI to deny true evidence rather than manufacture false evidence en-masse. The former is a lot of work and people are lazy. COVID showed that people's capacity for self-deception is unlimited, their willingness to deny the evidence of their own eyes is bottomless as long as they're told to do it by authority figures. You don't even need AI to be abused at all for someone to say, "ignore that evidence that we're clueless and corrupt, it was made by an AI!"
Then by point 6 we're on the usual trope of all public intellectuals, of assuming unending exponential growth in everything even when there's no evidence of that or reason to believe it. The self-improving AI idea is so far just a pipe dream. Whilst there are cases where AI gets used to improve AI via self-play, RLHF and so on, it's all very much still directed by humans and there's no sign that LLMs can self improve despite their otherwise impressive abilities. Indeed it's not even clear what self-improvement means in this case. It's a giant hole marked "??? profit!" at the heart of the argument. Neurosurgeons can't become superintelligences by repeatedly performing brain surgery on themselves. Why would AI be different?
To clarify my own thinking here, it's totally reasonable to me that people are surprised if:
1. They weren't previously aware of AI research (surely 99% of the population?)
2. They were but had stopped paying attention because it was just a long series of announcements about cool tech demos nobody outside big corps could play with.
3. They were paying attention but thought scaling wouldn't continue to work.
My problem is that people like Sam Altman clearly aren't in any of those categories and Hinton shouldn't have been in any, although maybe he fell into (3). I personally was in (2). I wasn't hugely surprised that ChatGPT could exist because I'd seen GPT-2, GPT-1, I'd seen surprising AI demos at Google years earlier and so on. The direction things were going in was kinda clear. I was a bit surprised by its quality, but that's because I wasn't really paying close attention as new results were published and the last InstructGPT step makes such a big difference to how the tech is perceived. Actual knowledge doesn't change much but once it's housetrained, suddenly it's so much easier to interact with and use that it makes a step change in how accessible the tech is and how it's perceived.
I think I was more surprised by the joining of LLMs with generators and how well AI art works. It does feel like that happened fast. But, maybe I just wasn't paying attention again.
So I guess where we differ is that I don't take their surprise at face value. The direction was too clear, the gap between the threats they talk about in the abstract and the concrete proposals are too large and too lacking in obvious logical connection; it feels like motivated reasoning to me. I'm not entirely sure what's really going on and perhaps they are genuine in their concerns but if so it's hard to understand why they struggle so much to make a convincing case given they are certainly intellectually equipped to do so.
The two posts you linked are interesting and much better argued than the website this thread is about, so I'll reply to them directly.
1. Yes I believe independents are more reliable than full time researchers because the latter are deeply conflicted and independents aren't.
2. They don't work for the oil industry. I've checked. That's propaganda designed to stop people listening.
3. There was in fact a very real pause, not simply due to statistical chance. Climatologists didn't predict that and fixed the problem by altering the historical record to erase it. That's why you can't see a pause now - not because there wasn't one, but because any time temperature graphs don't go according to plan, they change the databases with the measurements so they do. Look into it. There's another pause going on right now! In fact temperatures seem to have been stable for about 20 years modulo an El Nino in ~2015, which is natural.
4. A big part of why they're unreliable is that these guys don't engage with the real world. A major embarrassment for them was when people started driving around and looking at the actual ground station weather stations and discovered what an unusable data trash fire the ground network was - this was something climatologists themselves hadn't bothered to ever look at! You'd expect these experts to know more about the quality of their data than random bloggers but it wasn't so.
5. Where do you think Al Gore got his original claims? He didn't invent them out of whole cloth. They came from climatologists, of course.
You can go back and forth on climate related claims all day and get nowhere because the field is so corrupt that half the data is truncated, manipulated, tipped upside down, cherry picked, or wholesale replaced with the output of models whilst being presented as observation. It should go without saying but if the people who control the data also make the predictions, then they will never be wrong regardless of what happens in reality!
Yes, you can assert that an AGI would be a superhuman level hacker and so no rules would apply. It gets back to the same place all discussions about AGI risk get to - a religious argument that the entity in question would be omniscient and omnipotent therefore it can do anything. Not interested in such discussions, they lead nowhere.
Terrorists aren't programs or even a well defined group, so I don't see the analogy.
https://news.ycombinator.com/item?id=36123082#36129011
Arguments like yours are very subjective. What is "rapid". What is "surprising". I don't find them particularly surprising myself - cool and awesome - but I was being amazed by language modelling ten years ago! The quality kept improving every year. It was clear that if that kept up eventually we'd have language models that could speak to like people.
So the idea of a surprising change of pace doesn't really hold up under close inspection. LLM capabilities do seem to scale linearly, with the idea of emergent abilities coming under robust attack lately. To the extent big LLMs are surprising to a lot of people this has happened primarily due to throwing a previously implausible quantity of money at building them, and OpenAI releasing one of them from their lab prison that other companies were keeping them in, not due to any major new breakthrough in the underlying tech. The progress is linear but the visibility of that progress was not. The transformers paper was 5 years ago and GPT-4 is basically an optimization of that tech combined with RL, just executed very carefully and competently. Transformers in turn were an improvement over prior language models that could speak like a human, they just weren't as good at it.
> It doesn't though, does it?
It does. Arguments that consist of "everyone knows that" are also called rumours or folk wisdom. It's fine to adopt widely held beliefs if those beliefs rest on something solid, but what we have here is a pure argument from authority. This letter is literally one sentence long and the only reason anyone cares is the list of signatories. It's very reliant on the observer believing that these people have some unique insight into AI risk that nobody else has, but there's no evidence of that and many signers aren't even AI researchers to begin with.
This thread is really a perfect demonstration of my point. Our society is so in thrall to self-proclaimed intellectuals that you can literally make a movie presenting falsifiable claims with 100% confidence, people can say at the time "this is absurd and will not happen", you can spend years attacking those critics, it can then not happen and still you will have an army of defenders who dodge behind weasel-words like "generally right".
Of course the usual trick is to express only 95% confidence. Then when it doesn't happen you say, well, I never said for sure it would, just that it seemed likely at the time.
See? It's a winning playbook. Why would anyone not deploy it?
> His opponents at the time were mostly claiming that it either wasn't happening at all, or wasn't largely the result of human activities. What do you think is the current credibility of those claims?
Pretty high, having looked at the evidence. The usual rebuttal is to express disgust and displeasure that anyone might decide these claims via any method other than of counting "smart people". But those "smart people" are who Al Gore was listening to when he made that claim about Kilimanjaro, so they can't be that smart can they?
If it runs on millions of hacked devices, how do you turn it off? The same way any botnet gets turned off: virus scanners clean it up, the C&C servers get taken down, etc. This is not a new problem.
The usual response to this is to claim that superintelligence will develop so fast that we'll go from crappy chatbot to Skynet in 5 seconds flat, and nobody will have a chance to turn it off. That whole scenario is unmoored from reality: it assumes multiple massive leaps in tech that aren't anywhere on the horizon and where it's unclear why anyone would pay for such massive CPU overkill in the first place.
To get around this the usual Star Trek analysis (by fans, the series itself doesn't talk about it much) is that after replicators were invented, there didn't need to be capitalism anymore and so there's no money in the future and everyone just works on perfecting themselves. It's a wafer thin social idea that was never fleshed out because the writers themselves didn't believe in it. Roddenberry insisted but the writers often couldn't make it work which is why there are so many weird hacks, like saying the replicators can't replicate things as big as star ships and they mostly just ignore the whole topic. Also the replicators kill a lot of drama because they mean there can't be any valuable objects.
There are obvious and basic objections to this idea that replicators = communism (in either direction). One is that you can't replicate services, and much economic activity today is the service economy. We see that the Enterprise has staff who do things like wait tables, cut hair and sign up for red uniform missions in which they will surely die, but why they do this in the absence of money is never explained. There's just this ambient assumption that everyone works because work is awesome.
Getting back to the thread, the lack of philosophical rigor here is all on McFarland unfortunately. He doesn't actually have a critique of capitalism. He doesn't even seem sure what capitalism is, appearing to use the term to just mean contemporary society and anyone he doesn't like. Even his arguments against his strawman enemies are garbled and useless! He shits on Musk, saying that if Elon invented a replicator he'd patent it and hoard the tech to himself, ignoring that Tesla gave away its entire patent pool so anyone else could build electric cars using their tech. Musk - arch capitalist - did the OPPOSITE of what McFarland claims capitalists do, and he didn't even notice! All the rest of his argument is also like that. He makes absurd claims about governments, Republicans killing animals in TV ads, some non-sequitur about meatless sausages ... it's just this total grab bag of incoherent thoughts that make no sense and don't seem connected to each other, wrapped as "capitalism sucks, communism rules".
If this were an essay I'd grade it an F. But in the end it's just a set of tweets. Those looking for philosophical rigor on the idea of an abundance machine need to look elsewhere.
> Has it occurred to you what happens if you are wrong?
Has it occurred to you what happens if YOU are wrong? AI risk is theoretical, vague and most arguments for it are weak. The risk of bad law making is very real, has crushed whole societies before and could easily cripple technological progress for decades or even centuries.
IOW the risk posed by AI risk advocates is far higher than the risk posed by AI.
Where is the thought provoking idea here? It's just an excuse to attack his imagined enemies. Indeed he dunks on conspiracy theorists whilst being one himself. In McFarland's world there would be a global conspiracy to suppress replicator technology, but it's a conspiracy of conspiracy theorists.
There's plenty of interesting analysis you could do on the concept of a replicator, but a Twitter thread like that isn't it. Really the argument is kind of nonsensical on its face because it assumes replicators would have a cost of zero to run or develop. In reality capitalist societies already invented various kinds of pseudo-replicators with computers being an obvious example, but this tech was ignored or suppressed by communist societies.
https://research.facebook.com/downloads/babi/
There's been a lot of progress since then, but it's also nearly 10 years later. Progress isn't actually instant or overnight. It's just that OpenAI spent a ton of money to scale it up then stuck an accessible chat interface on top of tech that was previously being mostly ignored.
Al Gore: "Within a decade, there will be no more snows on Kilimanjaro due to warming temperatures" (An Inconvenient Truth, 2006).
Everything is not solar. Snow is still there. Gore literally made a movie on the back of these false claims. Not only has there been no social penalty for him but you are even citing him as an example of someone who was right.
Here it is again: our society systematically rewards false claims of global doom. It's a winning move, time and again. Even when your claims are falsifiable and proven false, people will ignore it.
- Such exploits happen already and don't lead to extinction or really much more than annoyance for IT staff.
- Most of the computers attached to the internet can't run even basic LLMs, let alone hypothetical super-intelligent AIs.
- Very few cars (none?) let remote hackers kill people by controlling their acceleration. The available interfaces don't allow for that. Most people aren't driving at any given moment anyway.
These scenarios all seem absurd.
The social taboo here is saying that a position taken by lots of highly educated people is nonsense because they're all locked in a dumb purity spiral that leads to motivated reasoning. This is actually one of societies biggest taboos! Look at what happens to people who make that argument publicly under their own name in other contexts; they tend to get fired and cancelled really fast.
> there's been a pattern of people confidently predicting AIs won't be able to perform various particular feats of the human mind (either fundamentally or in the next few decades) only to be proven wrong over increasingly shorter time-spans
That sword cuts both ways! There have been lots of predictions in the last decade that AI will contribute novel and hithertofore unknown solutions to things like climate change or curing cancer. Try getting GPT-4 to spit out a novel research-quality solution to anything, even a simple product design problem, and you'll find it can't.
> the claim that it seems far too obvious that we should
They're not arguing that. They're saying that AI risk proponents don't actually have good arguments, which is why they so regularly fall back on "it's so obvious we shouldn't need to explain why it's important". If your argument consists primarily of "everyone knows that" then this is a good indication you might be wrong.