c1ccccc1

Cryptographers are clever and have figured out a way to let you know your vote was counted without being able to prove it to a third party!

https://www.youtube.com/watch?v=BYRTvoZ3Rho

This comment is interesting in that it itself exemplifies principle #5.

(And also principles #6, #7, #9 to some extent.)

"Below" in this context means "directly below and connected by a line" (the lines are the edges of the cube). So you can have a blue vertex that is vertically below a white vertex, so long as they are not connected by an edge. The first time this can happen is for a 3 dimensional cube. You can have blue at the top, then 2 blue and 1 white below that, and then 1 blue (under and between the 2 blues in the layer above) and 2 white in the layer below that, and then white for the bottom vertex. This configuration can be rotated 3 ways and this takes us from 17 to 20.

The post assumes a 2% annual rate of growth in energy consumption. So, due to the nature of exponential functions, most of the energy loss would concentrated towards the end of the 1000 years, as the energy consumption approaches 400 million times present day energy usage. The first two centuries of use would not have a noticeable impact.

So I'm not a moral relativist, like, at all. But in this case, it seems like we westerners have constructed one particular set of norms for encouraging innovation, where we decide that it's possible for ideas to be owned. It's not like there's anything intrinsically wrong with copy-pasting code, it's just that we have a legal framework where we've traded away the right to freely copy-paste code so that we can grant a temporary monopoly to its author. We do this in the hope that more useful code will be written than otherwise. But if the people of China decide that that's not a trade-off they want to make, then I don't think we westerners get to say that they've committed a moral wrong in making that decision. It's just that they have a different way of doing things.

Like I said, I'm not a moral relativist at all. Murder is still wrong in China, imprisoning people not convicted of any crime is still wrong in China, lying is still wrong in China. But I just don't see how copyright infringement is universally an immoral act.

> Hinton and other billionaires are making sensational headlines predicting all sorts of science fiction.

Geoff Hinton is not a billionaire! And the field of AI is much wider than LLMs, despite what it may seem like from news headlines. Eg. the sub-field of reinforcement learning focuses on building agents, which are capable of acting autonomously.

Pretty sure the angst is about the AGI killing everyone. What's the connection between not killing people and enslavement? I don't kill people, yet I don't consider myself enslaved. The entire point of worrying about this at all is that a sufficiently smart AI is going to be free to do whatever it wants, so we had better design it so it wants a future where people are still around. Like, the idea is: enslavement, besides being hugely immoral, obviously isn't going to work on this thing, so we'd better figure out how to make it intrinsically good!

Weird. Seems like this is a plagiarism of this article:

https://www.strangeloopcanon.com/p/agi-strange-equation

(Or maybe it's an independent reinvention of the same idea, and something's just in the water.)

It's much easier than that! Living cells already have ribosomes that construct proteins and all the other molecular machinery needed to go from DNA sequence to assembled protein. You can order a DNA sequence online and put it into e-coli or yeast cells and those cells will make that protein for you.

By looking at the weights? Yes, it would be very easy to tell.

I have watched it, and it was actually talking about this: https://en.wikipedia.org/wiki/Poincar%C3%A9_recurrence_theor...

Ironically Banach-Tarski would have been better suited for a movie about infinity. Poincare recurrence is relevant for amounts of time that are merely exponentially large, not infinite.

I agree with Wadge that the movie was not very good, but I'm not actually sure what his objection is to the bit with the orange in a box, it seems fairly reasonable based on our current understanding of physics.

> Any other vehicle and we would hold the worker accountable for not refusing such a negligent order.

Say the worker refuses that order and their boss says "okay, you're fired". Would we also give that worker legal protection? Would we also hold accountable the higher ups who gave that order and set the policies for how trains are to be run? If not, then you now understand why the accident happened.

Anyone else do a double-take when they got to the part about Edward Snowden?

The author of this article must either really love the NSA or really hate Snowden if he's willing to undermine his entire thesis just to get that jab in.

Here's some more examples:

    cos((x - t)) + cos(0.5877852522924731*y + 0.8090169943749475*(x - t)) + cos(0.9510565162951535*y + 0.30901699437494745*(x - t)) + cos(0.9510565162951536*y - 0.30901699437494734*(x - t)) + cos(0.5877852522924732*y - 0.8090169943749473*(x - t)) = z

    10.0*(pow(sin(0.1*(x*x-y*y)), 2.0) + pow(sin(0.2*x*y), 2.0))/sqrt(x*x + y*y) = z

    x*x+y*y+z*z=2.0+cos(t+10.0*x+24.0*y)+cos(t - 10.0*x+24.0*y+16.0*z)+sin(t+x+y+2.0*z)+cos(3.0*t+y-z)+sin(2.0*t+x*y)+cos(t+3.0*x*y)+sin(0.1*t+y*z + 7.0*x*y)+cos(0.5*t+x*y*z)+cos(5.0*z)+cos(1.618*t+7.0*z)+cos((x+2.0*y)*(2.0\*z-y))

Do we actually have that much control over what the fission products are? My impression was that you hit U-235 with a neutron, and you get the same kinds of fission products out whether it's a bomb or a reactor. It's just determined by physics what kinds of isotopes are likely to come out, and my impression was that we don't know how to influence it so that only the shorter-lived kinds are created. Bombs may be designed to consume a very high fraction of the fuel, but that would tend to make them worse, since the fission products are more radioactive than the starting Uranium, and there will be more of them. The mushroom cloud thing does make sense as an explanation for why Hiroshima is still inhabited, though. And it's clear that Corium wouldn't be able form in the middle of a nuclear bomb explosion.

Could you expand on why the disappearance time is so much faster for a bomb than for a power plant? If 1kg of uranium undergoes fission, I would expect there to be a little less then 1kg of fission products resulting from it. No matter if it were a bomb or a power plant, the amount and lifetime of the fission products would be the same. There would have to be something else going on, like the bomb only splitting a tiny fraction of its uranium, or maybe something about the environment of the explosion destroying fission products?

After seeing the title but before clicking on the article, I thought this would be about a legal hack rather than a security hole. More specifically, creating the minimum possible corporation that qualifies as an airline (so that you literally own an airline), and then saying to the government, "hey, we need the nofly list, we're an airline, see?". The actual hack sounds way easier, to be sure, but I still like my version the best.

It seem pretty likely that we have to die, based on the second law of thermodynamics. The whole universe has to end eventually, after many trillions of years. There's no physical law that would stop a person sticking around for those trillions of years, but it would be technologically very difficult.

I think the person you're replying to is referring to cryonics (the topic of the original article), which is still illegal in many jurisdictions.

As something of a free speech absolutist myself, I'd say this Twitter policy goes directly against the principles of free speech. No twisting required, Elon is simply being hypocritical here.

Fascinating. Without a detailed knowledge of biology, it seems like this could affect any cell type, why would heart muscle cells specifically be targeted? Are the spike proteins just more likely to get stuck to the surface of certain types of cell?

Yes, in fact. This text shows up at the bottom of the page:

> Disclaimer: This essay was generated by StableDiffusion (the image) and ChatGPT (the text). It is based on my tweetstorm, which can be found here: https://twitter.com/IntuitMachine/status/1600804191731036160

So which of the following count as "man made" chemicals?

Alcohol or vinegar from humans intentionally fermenting things?

A metallic aluminum alloy?

Sulfuric acid (which sometimes occurs naturally)?

Turpentine?

Soda-lime glass?

I get that it generally refers to substances that are more on the very artificial side, requiring advanced knowledge of chemistry to produce, and to have a connotation of harmfulness / toxicity. But it's not at all obvious what the speaker would consider to be a "chemical" because that varies from speaker to speaker.

Paper link: https://arxiv.org/pdf/2209.11178.pdf

Lots of interesting things I'd want to try here. Eg, the formulation of thinking of your probability distribution as a charge distribution opens up the possibility of adding negative charges. So you could have a regular positive data examples, but also some negative examples that the generative model tries extra hard to avoid.

I'd also be interested to see what happens when you "add the fields" for two different distributions over the same space. Seems like it would be interestingly different than regular superposition, and I'd really like to see what happens when you add together a dog image field to a cat image field.

I guess the current formulation where you only get the direction of the electric field and not the magnitude would cause problems there. Seems like an improvement might be to predict the actual potential instead of the field and then take the gradient. That would also guarantee that the field is curl-free, which it should be, but it doesn't seem to be enforced in the current paper. I think they don't try this because it's hard to get NNs to produce numbers across a wide range of orders of magnitude. Maybe separately predicting the normalized direction and something like the log of the magnitude would help here.

> it is hard to find a place where independent thought is so unwelcome.

Is that actually true, though? It's true that a higher fraction of the people in that community give credence to the intelligence explosion hypothesis than pretty much anywhere else. (This is what one would expect, since part of the purpose of LessWrong is to be a forum for discussions about super-intelligent AI.) But even if the intelligence explosion is a terrible, absolutely-wrong theory, that doesn't prevent the people who hold it from being open-minded and tolerant of independent thought. Willingness to consider new and different ideas is something the LessWrong community claims to value, so it would be a little bit weird if they were doing way worse than average at it. And AFAICT, it seems like they're doing fine. Some examples:

- Here [1] is a post critical of the intelligence explosion theory. It has 81 upvotes as of this writing, and the highest upvoted comment goes like: "thanks for writing this post, it makes a lot of good arguments. I agree with these things you wrote" (list of things) "here are some points where I disagree" (list of things). This may even be the original post you were talking about in your comment, except that it doesn't start with an apology.

- LW has 2 different kinds of voting: "Regular upvotes" provide an indication of the quality of a post or comment, and "agree/disagree votes" let people express how much they agree or disagree with a particular comment. Down-voting a high quality comment just because you disagree (instead of giving it a disagree-vote) would be against the culture on LW.

If you're already sure that LW is wrong about superintelligence, and you're trying to explain how they became wrong, then "those LW people were too open minded and fell for that intelligence explosion BS" makes more sense to me than anything about suppression of independent thought.

[1] https://www.lesswrong.com/posts/zB3ukZJqt3pQDw9jz/ai-will-ch...

My responses:

> We're ahead of even the most optimistic timelines — what's happening with transformers is taking everyone by surprise.

I admit that I was surprised by how good GPT-2 was. But after that, you expect a lot of effort to go into development and for things to improve further. I haven't found any of the subsequent developments too surprising. Also, while LLMs do make a good example of how capable neural networks can be if you're willing to spend a load of compute, the kinds of models that have the potential to be dangerous are reinforcement learning models, and the state of the art there is a lot more primitive than the state of the art in next-token prediction.

> Humans are famously bad at dealing with exponentials

AI research is not an exponential process. It's too discrete and jumpy. It's better to mainly think of it as a series of discoveries that can only be made once.

> I was chatting with a researcher from OpenAI the other day who told me he intellectually understood AGI risks, but couldn't feel the fear emotionally, because he was so close to the problem and saw how hard it was to get these models to do anything.

This should be reassuring. An expert in the field is saying that we probably have a lot of time before these models are at the point of becoming dangerous, based on his direct experience with them. It's reasonable to worry that AI researchers will find it hard to believe that the models they are working on are dangerous because their job depends on them not understanding the risk. That's not what's going on in this case, though.

> The only thing that gives me solace is that, historically, new life forms don't extinguish previous ones.

If we were to stop developing AI while continuing to develop all other technology, it's plausible that we'd eventually build a Dyson sphere around the sun. If we build an AI with alien desires, then we should expect it to want to gather resources so that it can achieve its alien goals. A very important resource in the universe is energy, so an AI would probably also want to build a Dyson sphere. An AI indifferent to human life wouldn't have a reason to leave any sunlight for us.

If the context window is finite, then LLMs actually are Markov chains. It's just that they're a much more efficient way of representing transition probabilities than storing them all in a giant lookup table.

I agree with your second point that having deontological boundaries is important. Interestingly, people like Eliezer Yudkowsky (generally respected in the EA movement, I'm pretty sure?) also agree with you about that. He's been reiterating the point recently on twitter, though this post [1] says pretty much the same thing and it's from 2008. Generally, I'd attribute your examples of "utilitarianism gone wrong" to mental illness or sociopathy rather than to holding a utilitarian philosophy. For the average person, moral philosophy is much more of a way to explain the choices we make after the fact than a thing that actually determines the choices.

[1] https://www.lesswrong.com/posts/K9ZaZXDnL3SEmYZqB/ends-don-t...

Interestingly, the sun would kill us much sooner than that: in less than 1 billion years the sun's increased output will heat up the Earth's surface to the point of becoming unlivable.

Related to the topics of AI apocalypse and putting things off because they seem far away: I personally find it amusing to contemplate a strong AGI being developed in 2035 and wiping out humanity. Why? Because it would eliminate the Y2038 problem, retroactively vindicating all the engineers who only allocated 32 bits to timestamps. It would mean the justification of "oh, it's so far off, the world will have changed unimaginably by then", would for once turn out to be correct!

Depends what Aaronson meant by "being ahead morally"; it's unclear whether he is referring to morality in the sense of being virtuous, or in the sense of making other people better off. Hypothetical Charitable Investor's donations are a sign that that they might be rather virtuous, but they certainly aren't 1000 times more virtuous than the average person, and if it turns out that they secretly have a habit of kicking puppies, then they aren't virtuous at all. But it could certainly be true that, compared to an average person, Hypothetical Charitable Investor has had 1000 times the impact in terms of how they improved other people's lives.