HNHacker News
TopNewBestAskShowJobs

hollerith

8,135 karma · joined November 4, 2007

Richard Hollerith, San Francisco Bay Area. hruvulum@gmail.com

No one has any plausible plan or halfway-decent plan for how to maintain control over an AI that has become super-humanly capable. Essentially all of the major AI labs are trying to create a super-humanly capable AI, and eventually one of them will probably succeed—which would be very bad (i.e., probably fatal) for humanity. So, clearly the AI labs must be shut down and in fact would have been shut down already if humanity were sufficiently competent. The AI labs must stay shut down until someone comes up with a good plan for controlling (or aligning) AIs, which will probably take at least 3 or 4 decades. We know that because people have been conducting the intellectual search for a plan for more than 2 decades as their full-time job, and those people report that the search is very difficult.

A satisfactory alternative might be to develop a method for determining whether a novel AI design can acquire a dangerous level of capabilities along with some way of ensuring that no lab or group goes ahead with an AI that can. This might be satisfactory if the determination can be made before giving the AI access to people or the internet. But I know of no competent researcher that has ever worked on or made any progress on this problem whereas at least the control problem has received a decent amount of attention from researchers and funding institutions.

The AIs that might prove fatal to humanity will be significantly different in design from the AIs that have been already widely deployed: for one thing, they will constantly learn (like a person does) as opposed to already-deployed AIs in which the vast majority of the AI's learning happens during a training phase that ends before any widespread deployment of the AI. Also, they will be much better than current AIs at working towards a long-term goal. I say this because I don't want to be misunderstood as believing that Google Gemini 2.5 or ChatGPT 5.0 might take over the world: I understand that those AIs are incapable of a such a thing. The worry is the AIs that are still on the drawing board or that will appear on a drawing board 5 or 10 years from now. There is no need to ban Gemini 2.5 and ChatGPT 5. Since some AI researchers pursue AI "progress" for ideological reasons and will tend to persist stubbornly even after AI research is banned, the best time to ban AI frontier research is now, so that these stubborn ideologues whose research will have been driven underground (because of the ban) but are capable of making a little more "progress" on AI will be unlikely to be able to make enough "progress" to end the world.

Again, as soon as anyone comes up with a solid plan for controlling (or aligning) an AI even if the AI turns out to be more capable than us, the ban on frontier AI research can be lifted as long as the majority of AI experts and AI researchers are in agreement that the plan is solid. No one can say how long this search for a solid plan will take, but IMHO it will probably take at least 3 or 4 decades.

More at

https://intelligence.org/the-problem/

submissionscomments
hollerith··on Religious scholars met with Anthropic
Huh? Part of the reason there is a tremendous boom in data-center construction is that there's currently quite high demand for inference. High demand means high prices. Even if the investors funding this construction all go bankrupt, the data centers don't just vanish: someone will buy them from the bankruptcy court and operate them. The effect of the current construction boom will be a huge increase the supply of inference, and of course increasing the supply will lower the price.

If one is in the data-center construction business or if one is a supplier of GPUs or other "inference hardware", then yeah, the current boom conditions are not "sustainable". But if one is a purchaser of inference, the tremendous investment our society is making in "inference capacity" ensures the the supply of inference is quite sustainable (barring a nuclear war or some other reason the data centers or the electrical infrastructure they depend on might be destroyed). And if the expectations for a rapid increase in the demand for inference turn out to be false, then that just means that the prices for inference will be nice and low for those who remain interested in purchasing it.

hollerith·
·on LeCun has "zero concerns" about AI wiping out humanity, recent "rogue" incidents
You imply that Jaan Tallinn is funding work in AI safety because he wants his investment in Anthropic to become more valuable, whereas Tallin has consistently said that his motivation for investing in Anthropic was to get a seat at the table so that he could urge Anthropic to be cautious in its development of the technology.

Tallinn's actions back up his explanation: in 2009, before he invested in any AI lab, he donated substantially to the nonprofit Singularity Institute for Artificial Intelligence, which was later renamed the Machine Intelligence Research Institute (i.e., Yudkowsky's outfit).

Some of us (certainly Yudkowsky and Habryka, the leader of Lightcone Infrastructure, which runs Lesswrong) wish people would stop believing that they can improve the bad situation caused by AI research and development by investing in (or working for) frontier AI labs, but that is what the preponderance of the evidence shows Tallinn (and Dustin Moskovitz and others) did sincerely believe.

hollerith··on On social reality in China
I don't think that's it.

If my main consideration in an interaction with a stranger is to avoid its escalating to violence, I'm going to be polite, but I'm going to be a bit reserved, not enthusiastic or friendly.

In these discussions, non-Americans often describe Americans as always happy, which is another quality I'm going to tend to avoid showing in the hypothetical interaction with the stranger because if he is feeling bitter or resentful, my being happy might set him off.

hollerith··on On social reality in China
I'm curious why you think that someone who left China at age 4 "isn't culturally Chinese": a child famously absorbs a great deal of culture from their parents and these particular parents grew up, married and had at least one kid in China.
hollerith··on Micron CEO Says Memory Supply Will Be Much Tighter in 2027 and 2028 Than in 2026
The payments coming due for contracted memory purchases are spread out over 5 to 10 years, not 1 to 2 years as implied by your "excess chips hitting the market in 2027-28".

For memory specifically, the counterparties are mainly "Google, Microsoft, Amazon, Meta, even Apple" like my sibling comment says (plus creditors most of us have never heard of making loans secured by data centers OpenAI and Anthropic plan to rely on).

Going public should make OpenAI and Anthropic a more attractive target for new large loans, again mostly secured by data centers.

If demand for AI services flatlines or if an international agreement stops the labs from training any new models (for safety reasons), then memory prices, GPU prices and inference prices will probably fall drastically, but OpenAI and Anthropic would probably survive that.

hollerith··on Gemini 4 Argon
But you've heard of this theory that the Sun is bigger than the Earth and you knew it enjoys considerable support, yes?
hollerith··on Red Hat being phased out of existence?
Being able to visit a web page, open a PDF or download and inspect a git repo without the web page, PDF or git repo's being able to completely pwn the entire machine including the bootloader and the firmware for all the hardware subsystems.

Desktop Linux is very bad at that compared to ChromeOS and MacOS, but some distros are much better than others. For a few decades now, Fedora has been one of the better ones.

hollerith··on Red Hat being phased out of existence?
Last time I evaluated NixOS (Mar 2021) the project didn't care a whit about desktop security. Has that changed?

Fedora at least tries relatively consistently—and has been for decades.

hollerith··on Singapore govt dating app uses Gale-Shapley stable marriage algorithm
The housing is better. Square footage per person for example has approximately doubled.

Also, in Chart 1 on page 6, "expenditure share for non-necessities" in the US goes from about 32.5% in 1950 to fully 50% in 2002-2003.

It would be nice if you told us the page from which you got your, "in 1950, food, transport, and housing consumed 70.3% of median working household expenditure." Searching for your text verbatim fails, which is not particularly surprising because PDFs cannot be searched reliably.

Also because 1950 is considered by some to be a particularly prosperous time in the US, some might accused you of cherry picking. For example, in 1918-1919, the "expenditure share for non-necessities" was about 22%.

hollerith··on Singapore govt dating app uses Gale-Shapley stable marriage algorithm
>most economists admit that as women entered the workforce en masse, prices settled higher for the same goods and services.

Most economists emphatically do not admit or maintain that.

Most might believe however that compensation settled slightly lower because the economy's ability to create jobs lagged slightly behind the increase in the supply of workers.

hollerith··on 500k facial scans at UK stations yield no arrests, 1 false positive
Some American cities like St Louis, Baltimore, Memphis, New Orleans have averaged about 50 homicides per 100,000 per year over the last 10 years. Detroit averaged about 40.

It is not "unhinged propaganda from the right" for people living in those cities to wonder why the rate cannot be under 2.5 per 100,000 like it is in most European cities. Except for a few years in the early 2000s, London's rate has stayed below 2.5 for the last 250 years.

hollerith··on Everybody’s home. No one’s coming over
>Sure, those are fictions for TV

When you noticed you were relying on evidence from fiction, instead of verbally acknowledging that that was what you were doing, you should've stopped relying on the "evidence".

It might make sense every now and them to rely on fiction for evidence in domains where it is very hard to obtain any data at all, but in this particular, there is plenty of hard data of what US workers used to earn, what they earn now, what houses and food used to cost and what they cost now.

(I expect that the hard data will fail to support your belief that things have gotten significantly harder economically for the average person.)

hollerith··on It's Time to Investigate the AI Labs
I think the problem is the ones and zeros or the linear algebra if you want to use that metaphor. I believe "progress" in AI needs to be stopped. AI researchers need to find something else to do with their lives.

My first comment was an attempt to show the "it's just linear algebra" argument in an absurd light.

hollerith··on It's Time to Investigate the AI Labs
>restated the point being made in the remaining 67%

That's not an accurate description of my intent—or how my comment landed with most readers AFAICT.

hollerith··on It's Time to Investigate the AI Labs
>AI is just matrix math

Digital child porn is just ones and zeros. Surely we're not going to pass a law to regulate ones and zeros?

hollerith··on CEO of Mistral: AI is software. It can be controlled
I think liability works for, e.g., an oil refinery that has an explosion every 30 years, but does liability work for an AI lab that is causing harm to society every week?
hollerith··on GPT-6 Sol and Luna
Mechanization and chemicals (fertilizers, pesticides, herbicides and fungicides).
hollerith··on CEO of Mistral: AI is software. It can be controlled
Oh, that is a relief. It's reassuring to learn that no one will give any cutting-edge AI access to the internet from now on. Problem solved
hollerith··on An airport cooled by natural ventilation
During the energy crisis in the summer of 1979, President Jimmy Carter issued an emergency conservation plan that mandated air conditioning thermostats in public and commercial buildings be set no lower than 78 degrees Fahrenheit, which is not too far from 80.6 degrees.

The mandate remained in effect for about nineteen months until President Reagan rescinded it less than a month into his presidency.

hollerith··on UK military jamming other nations' satellites to defend itself, BBC told
George Friedman suggests that the reason Washington is going back to the moon is that it is easier to launch satellites into Earth orbit from the moon than from the Earth's surface.
hollerith··on Zelensky says Russia has widened attacks to hit Ukraine's data centres
>Nobody is hoping for a peace deal.

I'm hoping for a peace deal.

hollerith··on Zelensky says Russia has widened attacks to hit Ukraine's data centres
Would it not be an escalation if Moscow declares a full general mobilization and sends all conscripts to fight in Ukraine?
hollerith··on Silicon Valley 'sex assault list' with 'over 100' names circulated
Your comment implies that it is impossible (or at least so unlikely as to be not worth thinking about) for someone or someones to intentionally put an innocent person on the list.

Please don't assume that.

hollerith··on Italian parliament votes for return to nuclear energy
I'm pointing at a different thing in the world than you're pointing at.

One reason I know they're different is that the thing I'm pointing at is vastly less influential in some people than other, so much so that it clearly makes sense to call some people "unideological" or if one is being tediously precise "almost completely unideological".

Everyone is strongly influenced by their language, their culture and the family they grew up in. Not everyone has even a weak commitment to an ideology.

I'm willing to switch to a different term for the thing I'm interested in, but I haven't been able to come up with one.

hollerith··on Feds Target AI Critics as "Foreign Agents"
Thanks for the explanation and specifically for the detailed explanation.
hollerith··on Italian parliament votes for return to nuclear energy
You're right. I had something I really wanted to say, which would have fit into the conversation if you had said something somewhat similar but actually quite different from what you actually said, but I overlooked the difference so that I could say what I really wanted to say.

In other words, I sacrificed the quality of the conversation (namely, I introduced not only a tangent, but also a tangent rooted in a confusion) so that I could express some thought that had risen to salience in my solitary mind. My doing that made the conversation worse, and the conversation is what matters, not whether I got to express myself in a way that is quick and easy for me.

hollerith··on Italian parliament votes for return to nuclear energy
I regret publishing that second paragraph. I knew (or would've known if I had been paying sufficient attention) it would offend many in this audience, and I should've held my tongue.

But I really do believe that Star Trek, specifically the parts that Roddenberry personally oversaw (namely the original series and the first four seasons of The Next Generation) had a very negative effect on the world.

In addition to making some viewers more committed to socialist ideology[1] Roddenberry's Star Trek made many committed to the ideology of technological progress.

Although it can have a beneficial effect on a person, a commitment to an ideology tends to have a deleterious effect on the world when many share a commitment to the same ideology.

[1] In a (private) debate with me, a committed socialist asserted that a successful transition to socialism would result in a society much like the one depicted in Star Trek. Since it is very unlikely that he engaged in many debates about socialism before his debate with me, the reason he believed that that assertion might make me warm to socialism is probably because it had contributed to his own warming to socialism.

hollerith··on Italian parliament votes for return to nuclear energy
Are you really going to keep on trying to defend the notion that the human society portrayed in Star Trek is not socialist? (In The Next Generation, no human investors, traders or financiers are ever depicted, and human society did away with money. Socialist countries have militaries, too.)

The proper reply to the parent of your comment IMHO is that Star Trek is evil propaganda, not that the vision of society depicted in the propaganda is not socialist.

hollerith··on Ideas on modernizing the open-source desktop
The iPad offered only tiling till about 3.5 years ago, and tiling is still available as an option in Settings.

And some people connect their iPad to a full-size monitor, keyboard and mouse and use it as their main desktop machine.

hollerith··on Feds Target AI Critics as "Foreign Agents"
The intent of your comment is unclear to me, so I will ask you to clarify. And I will avoid any attempt at sarcasm from now on.

Great grandparent claimed (or at least strongly implied) that the labs have been saying that their creations might end the human race because the labs expect that saying such things will attract investment in the labs or purchases of their creations. I disputed that claim (using sarcasm).

Do you agree with me or disagree with me? (I honestly cannot tell.)

Would you say that the meme you quoted is sarcasm?

Do you believe that the meme you quoted has attracted investors or buyers?

That anyone ever expected that it would?

← PreviousPage 3 of 34Next →