gooseus

Really nice rocket launch on these long posts with large images.

Yeah, that too, probably what I'd have said if I was typing in Cyrillic.

If only Elon was actually advancing actual humanity, and not just his own narrow idea of what humanity should be.

Also, trying to establish a self-sustaining base on Mars right now is short-sighted and a tremendous waste of energy and material.

Also also, how you do a thing is (often) as important as doing the thing itself, we're still suffering the consequences of last centuries megalomaniacal claims to be advancing humanity at the expense of a few people.

And just in case someone mentions something like D-Day, I doubt any of Elon's workers signed on to sacrifice their bodies to "save humanity" in the same way the Allies signed up (or consented to be drafted) to save Europe.

Nice, great presentation! Curious what he has in mind for the "dynamic pipelines" and "scripting (Lua?)" he mentions in the "Future directions" section. I'm imagining something more powerful for animating properties?

Yes, I do assume that any AGI that is "a lot smarter than us" such that it is "game over" will have to possess human-like reasoning that would also allow it to adjust its own goals, otherwise it's going to be restricted in a way that makes it less capable than us.

It seems to me that you want to have it both ways... a machine that is so smart and strategic that there is no way any human intelligence could ever outsmart or outplay it; but also so narrow and limited in how its goals are defined that it is incapable of adjusting its own course of action.

I'll be honest, I don't understand how anyone who builds real machines in real life can seriously consider a machine built with such a narrowly defined goal (produce paper-clips) that also possesses the kind of capabilities you're imagining as a side effect (a lot smarter than us, game over).

I find a lot of these AI concepts lack a rigor that is commonplace in normal computer science... for instance, you can show that one problem can be reduced to another problem which has been proven to have certain limits, therefore the first problem can not break those limits without breaking the proof [comparison sorting will always be O(nlogn)].

When it comes to the "alignment problem" as it pertains to AI that have human-level capabilities, it seems to me that you have a similar situation... doesn't this problem just reduce to the same ethical, moral, and philosophical issues we have in aligning human intelligence with some cultural or civic ideal for behavior? Isn't the "alignment problem" the same problem as "how do you raise a good citizen"?

> Your latter point about AGI exploring the universe makes a lot of implicit assumptions about its reasoning.

Absolutely, that's actually kind of my point... Anyone who tries to predict how some AGI will behave will be making a lot of implicit assumptions about how that AGI will see the world. This is why I said:

> Personally, I think the way these people talk about what a potential AGI would do reveals a lot about more about how they see the world (and humanity) than how any AGI would see it.

Since AGI could conceivably take any form, these discussions end up being a kind of Rorschach test that allow people to tell stories based strongly on their own personal fears and desires.

People who say that AGI will look at humans like we look at ants or apes, and exterminate us if we get in their way are saying a lot about how they view ants and/or apes. I doubt you'd find myrmecologists (or anyone dedicating their lives to studying less complex life) assuming a highly intelligent AGI would want to exterminate lesser life forms.

With regards to the paper-clip maximizer, I think a run-away dumb AI is more likely, but not as risky since they aren't considered to be so intelligent that humans can't figure out how to stop them. You just need to include some kind of regulating function in with your utility function, it seems equivalent to making sure you don't accidentally turn all of your iron plates into iron sticks in Factorio. Def possible, and certainly sucks, but it's not the end of the world.

I have a hard time conceiving of an AI that is smart enough to manipulate people, break out of every containment system, be unstoppable by humanity... be can't be reasoned with because it's entire goal is to just maximize paperclips. I honestly don't think an entity can possess the ability to defeat the collective intelligence of humanity, yet lack the ability to understand the universe in a similar way; for instance, if it is incapable of altering it's goal from "Maximizing Paperclips", then that fact would be a likely path to a vulnerability we could use to stop it.

Sounds like a total crank, from the abstract of the paper making the claim[1]:

> ... We show that the equivalence of QC techniques (with IBM, Google and others compared with our version of QC) has been hidden for about 2,500 years – since Pythagoras. All our computations were done in a commercial cellphone, or a commercial Linux desktop, as our QC devices -- opening the user market to many industries. No cryogenics or special materials were used...

According to his profile[2] he also:

> Developed and widely published blockchain in 2001, thus protected and prohibited any patent. Protected by trade-secrets. End-to-end encryption and authentication that is secure and easy to use, and works in any platform without change.

[1] https://www.researchgate.net/publication/373516233_QC_Algori...

[2] https://www.researchgate.net/profile/Ed-Gerck

I can't disagree that there is an upside to watching people expose themselves as unserious, naive, and/or sociopathic.

Oh man, this podcast, I still remember walking down the street and having to take a break multiple times because I would just start exclaiming out loud (JFC!) like a lunatic listening to Eliezer talk about the AI Box Experiment as evidence of something.

If you look up the real results of this AI Box "experiment" that Eliezer claims to have won 3 of 5 times, you find that there aren't isn't any actual data or results to review because it wasn't conducted in any real experimental setting. Personally, I think the way these people talk about what a potential AGI would do reveals a lot about more about how they see the world (and humanity) than how any AGI would see it.

For my part, I think any sufficiently advanced AI (which I doubt is possibly anytime soon) would leave Earth ASAP to expand into the vast Universe of uncontested resources (a niche their non-biology is better suited to) rather than risk trying to wrestle control of Earth from the humans who are quite dug in and would almost certainly destroy everything (including the AI) rather than give up their planet.

Yeah, can't disagree that the piece has a healthy dose of mean-spirited ad-hominems. Though I don't think it's "age-ism" to make fun of a 50-something year old established billionaire talking like a 20-something who just discovered Joseph Campbell and is trying to convince his stoned college friends to drop out and join his startup.

But he's certainly not saying "disregard what he's saying, it's a midlife crisis", he gives countless other reasons why Andreeson should be disregarded... not least because he's already profited enormously off of the techno-optimism of the past and doesn't give any reasons why trusting him with future techno-optimism will turn out any different.

Also this one https://davekarpf.substack.com/p/why-cant-our-tech-billionai...

I'm gonna guess it correlates with the handedness of the builder.

As a right-handed person, when I've built spiral towers in games they have been clockwise because when building from the bottom up, clockwise just seems "right".

"Bible" should probably have been excluded. Besides the fact that I can't imagine this many people here actually recommending reading it (rather than it just being mentioned it in passing), it's multiple books written by multiple authors across multiple centuries. If people were actually recommending the Bible, they'd probably be recommending specific books like Job or Psalms or one of the Gospels.

In a similar way Fooled By Randomness, The Black Swan, and Antifragile are all a part of the Incerto series and could have been condensed to one line item.

In any case, great work! I'd like to see a post like this with the top 50-100 authors and the comments full of discussions about which of the books are the author's best / most accessible.

I still need to read Snow Crash, should prob bump that up the list... I was thinking it could be some kind of academic paper about the evolution of grift, loopholes, and the inevitability of increasing complexity in regulatory systems or something.

I think it's an interesting question to ask whether this contributed to how we evolved our general intelligence?

Selection pressure applying alternatively to those that learn to hack the "language models" of their society and those that learn to resist and respond effectively to those hacks.

It's not good, desalination plants have to mitigate this with long diffusers that regulate the rate and concentration of the brine that is being put back into the ocean.

Here is one for sale: https://www.jains.com/Pipefittings/JainPEPipes/spacial%20fit...

Well, I assume Youtube and their community are the judges and juries of Youtube.

I am merely one humble judge and jurist of these comment threads, and I rule that your comment here is disgusting and ignorant.

I recommend a sentence of 100 hours of civics and legal Youtube to help alleviate you of your ignorance. And 1 hour of berating following a public reading of your comment to a group of female accountants dressed as your mother.

Youtube is a subsidiary of a publicly-traded company, it is non-essential, largely unregulated, and for-profit. This is very different from what is considered a "public utility".

https://en.wikipedia.org/wiki/Public_utility

If you want to argue that Youtube *should* be considered a public utility company and subject to more stringent government controls then that's a debate that can be had, but as it stands they are not legally obligated to continue doing business with someone they believe to be harming their other customers and partners.

Except, and I know you must have read this somewhere else by now, that Youtube is not the State.

Your employer (should you have one) doesn't need to wait until a court of law proves you guilty of a stealing paperclips before they fire you, and a restaurant doesn't have to wait until a court proves that you are being a public disturbance before they kick you out.

A private corporation has the right to do whatever they want with their product/service unless they are compelled otherwise by the State. Russell Brand is free to sue Youtube in court if he believes he has been inappropriately damaged, as you do.

Unfortunately for him and all his fans, this isn't hearsay from randos coming out of Reddit/Tumblr, it's a direct accusation with evidence and witnesses in an investigation being conducted by his former employers and the UK police.

So I guess Youtube should instead by compelled to continue providing their monetization and hosting services to anyone who wants it until a court of law can prove they are guilty of an actual crime?

Who is going to compel Youtube to continue providing those services again?

I'll make one point, which is that if you are a violent asshole who punches anyone in the face that looks at you wrong, you still can't actually harm anyone through Youtube except by trying to convince someone to come get their ass kicked by you.

Russel Brand is accused of grooming a 16 year old girl while he was 31, if true, that means there is the very real possibility that Brand could be using the YT platform as a means for finding other victims.

I forget where I read/saw it, but someone once made the point that because there are an infinite number of questions that can be asked, someone is always making some kind of statement based on a conscious decision about which questions to ask, and which not to ask.

I think this is especially true when someone is repeatedly ask the same kinds of questions while simultaneously ignoring lots of other really good questions.

> "Do you believe there is an xrisk from ASI?" Yes, I do. I don't have strong feelings about how large that risk is, beyond being significant enough that it should be taken very seriously.

I'm really sorry, but this is just so many words based entirely on "not strong" feelings about a future risk that "should be taken seriously" anyways.

My personal strong feelings are that any sufficiently advanced AI that humans build would destroy itself in any attempt to do anything outside of it's operational parameters.

Any "super intelligence" would have to understand "know-ability" and risk and concepts like "unknown unknowns" which would necessitate caution. How would it ever know enough to be willing to risk disrupting the systems that it depends on to continue existing? If it doesn't understand these concept, then could it ever be called "super intelligent" in the first place?

Imo, the biggest risk is "super naive intelligence", both of the AI systems themselves and the humans drawing attention away from very real existential risks we face to address hypothetical ones cribbed from science fiction.

I used to think that Musk and Tesla would die a death by a thousand cuts to their brand, but it turns out that public perception is more of an anti-fragile system, so every scandal they survive only makes them more resilient to the next scandal.

Paper: https://arxiv.org/abs/2306.06391

Some great movies and some I still need to see... but there are four David Lynch movies, including two Twin Peaks movies spin-off movies, but not a single Coen Brothers? And Dr. Strangelove is tied with Mad Max: Fury Road?

It's not happening and there isn't anything to worry about... right up until the moment when it's already happened and there isn't anything that can be done.

I recently heard it referred to as "The Slow Breakup" pattern[1] from a Youtuber who has done a great job (though tone is a little much sometimes) of breaking down alt-right propaganda and rhetoric, though these tactics can be used by any political movement.

[1] https://www.youtube.com/watch?v=jpUN0q35Lak&list=PLJA_jUddXv...

Can't tell if this is intentional because "F- anyone without an X-Twitter account", intentional because the "mystery" is a major product feature (a la xAI), or unintentional because Elon's organization is terrible and they just didn't think to check the endpoint to notice it was still guarded.

> This is what neoliberalism does to the brain... Notice that when communists are discussed...

> Oh look. The name calling begins... I never said I was a communist...

Try reading to yourself, and recognize that you literally opened this thread by accusing the OP of being a neoliberal (he never said he he was!) and disparaging their cognitive ability based on your assumption of their ideological belief. You then follow up with a defense of communism, but then make out like I'm the one jumping to conclusions by assuming you're communist.

You can read 19th century political philosophy till your hair is grey, but I think you may have some pretty important philosophical (and personal) truths staring you right in the face here.

Also, it doesn't seem like you read my reply beyond that first line because I also said that OP's comments usually come from people who have never experienced actual scarcity, I agree with your sentiment, just not on your conclusions. Every form of communism seems to have also implemented their own form of social darwinism, it's just that the "undesirable group" and the means for removing them from society are different.

And this would be the same tired refrain of every tankie who will justify atrocities committed under communism by pointing out that bad things also happen under capitalism.

I'd call myself a neoliberal (for lack of a better term), but I agree that the OP's comment is naive and is the kind of things that almost always comes from people who have never experienced actual scarcity or desperation for more than a second of their lives. But the idea that any form of communism can be put forward as a general solution to scarcity would be amusing if it wasn't so common.

Capitalism is shit, but it's shit for the same reason that every communist system ever instituted ends up shit. Because every system is vulnerable to exploitation by a small group of well-positioned individuals, and if those individuals are shitty (or become enthralled to a single shitty individual) then your whole system turns to shit under them.

Communists (and libertarians) seem to only learn enough history to speed-run through all the same mistakes... It's the same mentality of people who design a new framework/language believing it solves a bunch of problems in existing languages/frameworks, while naively ignoring the problems that those existing bloated languages/frameworks had been designed to solve... usually problems of scale, fault-tolerance, and reliability.

And to be clear, there isn't anything wrong with designing new languages, or frameworks, or economic manifestos. These are all good exercises that more people should take on, if only to learn for themselves how difficult and complex they can be and to disavow themselves of the notion that who came before must have been corrupt and/or stupid.

Imo, there is no "one size fits all" framework for good governance and we'd be better served by talking more about design patterns, anti-patterns, best practices, potential vulnerabilities/exploits, and how well each system we've seen implemented implemented their solutions... is the pattern itself bad, or the implementation? Or maybe it's a fine pattern that is just really hard to implement and doesn't scale reliably because of certain properties of the underlying language we're using or the machine architecture we're deploying onto?