HNHacker News
TopNewBestAskShowJobs

hollerith

8,135 karma · joined November 4, 2007

Richard Hollerith, San Francisco Bay Area. hruvulum@gmail.com

No one has any plausible plan or halfway-decent plan for how to maintain control over an AI that has become super-humanly capable. Essentially all of the major AI labs are trying to create a super-humanly capable AI, and eventually one of them will probably succeed—which would be very bad (i.e., probably fatal) for humanity. So, clearly the AI labs must be shut down and in fact would have been shut down already if humanity were sufficiently competent. The AI labs must stay shut down until someone comes up with a good plan for controlling (or aligning) AIs, which will probably take at least 3 or 4 decades. We know that because people have been conducting the intellectual search for a plan for more than 2 decades as their full-time job, and those people report that the search is very difficult.

A satisfactory alternative might be to develop a method for determining whether a novel AI design can acquire a dangerous level of capabilities along with some way of ensuring that no lab or group goes ahead with an AI that can. This might be satisfactory if the determination can be made before giving the AI access to people or the internet. But I know of no competent researcher that has ever worked on or made any progress on this problem whereas at least the control problem has received a decent amount of attention from researchers and funding institutions.

The AIs that might prove fatal to humanity will be significantly different in design from the AIs that have been already widely deployed: for one thing, they will constantly learn (like a person does) as opposed to already-deployed AIs in which the vast majority of the AI's learning happens during a training phase that ends before any widespread deployment of the AI. Also, they will be much better than current AIs at working towards a long-term goal. I say this because I don't want to be misunderstood as believing that Google Gemini 2.5 or ChatGPT 5.0 might take over the world: I understand that those AIs are incapable of a such a thing. The worry is the AIs that are still on the drawing board or that will appear on a drawing board 5 or 10 years from now. There is no need to ban Gemini 2.5 and ChatGPT 5. Since some AI researchers pursue AI "progress" for ideological reasons and will tend to persist stubbornly even after AI research is banned, the best time to ban AI frontier research is now, so that these stubborn ideologues whose research will have been driven underground (because of the ban) but are capable of making a little more "progress" on AI will be unlikely to be able to make enough "progress" to end the world.

Again, as soon as anyone comes up with a solid plan for controlling (or aligning) an AI even if the AI turns out to be more capable than us, the ban on frontier AI research can be lifted as long as the majority of AI experts and AI researchers are in agreement that the plan is solid. No one can say how long this search for a solid plan will take, but IMHO it will probably take at least 3 or 4 decades.

More at

https://intelligence.org/the-problem/

submissionscomments
hollerith··on David Sacks: OpenAI and Anthropic Don't Need Regulations to Pace Frontier Models
Many researchers have quit. The number of workers in the field who've warned the public that the field is a potent danger is quite unprecedented: I can't think of any other field with as many insiders turned critics. Why haven't they all quit?

Many people find power, prestige, the public spotlight or money so compelling that they'll keep on pursuing the thing even if they know the way they're going about it is morally wrong, almost like an addiction.

The leaders and workers of the labs aren't picked at random from your friends and neighbors: they've chosen to pursue what they foresaw (correctly, IMHO) as the most powerful new technology. In other words, they're probably self-selected for being unusually attracted by power.

An analogy: some serial killers know what they're doing is morally wrong and it pains them to be doing something very wrong. (Others have no conscience and don't care if it is wrong or not.) Even most of the ones that know what they're doing is wrong cannot stop killing because apparently killing is an extremely compelling experience to them. Often they're relieved when they get caught. Maybe Sam and Dario would be relieved if the government shut down their companies and prohibited them from starting new AI companies.

hollerith
·
·on How to buy a good car for $1000
You might consider it obvious that the personal car is more resource-demanding than the alternatives, but I don't, particularly in a country with as low a population density as the US.
hollerith··on Houthis 'take control' of key island in global shipping route
>Iran is the Houthis’ primary benefactor, providing them with security assistance, such as weapons transfers, training, and intelligence support. In July 2025, for example, the Yemeni National Resistance Forces seized the largest-ever shipment of Iranian munitions intended for the Houthis. The shipment included 750 tons of conventional weapons, including advanced drone parts, missile warheads, and radar systems, many of which were made by a U.S.-sanctioned firm linked to Iran’s defense ministry. Such aid typically reaches the Houthis via Iran’s paramilitary Islamic Revolutionary Guard Corps (IRGC), which also helps advise the Houthis’ military command.

Iran facilitates a supply of more sophisticated weaponry than the Houthis could acquire on their own, especially missiles and drones. While Iranian support has helped the Houthis gain and maintain military superiority within Yemen, experts say it has had greater effect elsewhere. “The role of Iran has been decisive in providing the Houthis with smuggled weapons and expertise to project power into the Red Sea and Bab al-Mandab Strait,” Gulf analyst Eleonora Ardemagni wrote [PDF] for the Yemen-based Sana’a Center for Strategic Studies in 2023.

https://www.cfr.org/articles/irans-support-houthis-what-know

hollerith··on How to buy a good car for $1000
>Among Americans under 50 without children, 57% said they simply didn’t want kids

Your 57% lacks relevance to the current conversation because it is 57% of a small percentage of Americans.

42 to 43% of Americans are either under 18 or a parent of an under-18-yo. Plus of course some outside that group will soon become parents. And some are empty-nesters who prefer to keep on living in the neighborhood they lived in when they were raising their children.

hollerith··on Measuring the sloppiness of code
>It's just a pretty pedestrian data-munging problem

In other words, how complicated could it be?

Well, if Adobe has been introducing complications and making the format brittle and inflexible over the last 35 years to make it hard for its competitors to write software to process PDF files, quite complicated.

hollerith··on How I feel about AI
I've seen many arguments for why AI will be safe. Yours relies on the assumption that no one would be stupid or reckless enough to give an AI that might be much more capable cognitively than a person the ability to manufacture weapons it has designed or at least no one would be stupid enough to do the final assembly of anything that might be a weapon from parts the AI designed and arranged to be made.

One argument (many years ago) was that no one would be stupid enough to give such an AI access to the internet or the ability to run code it has written. (In the HuggingFace incident, the AIs were given "read access" to the internet, or at least that was OpenAI's intent, but the AIs escalated that to write access.)

hollerith··on Tell HN: OpenAI keeps re-enabling the 'allow training' setting
>[the name "Misanthropic"] is childish

Good point, but "Sudden Human Extinction" would be too long a name.

hollerith··on The Feminist Was a Spy
>detachment from baser emotions that enable them to pursue abstract ideas to their logical conclusions

Yes, WASP are attracted to universal moral principles. A major WASP institution (namely, a religious denomination) is literally named "Universalism".

hollerith··on I resigned from Anthropic today
The Huggingface incident strongly suggests that OpenAI is not able to make its products consistently law-abiding.

It's not like OpenAI carelessly neglected to consider that its products might break the law: it has spent literally billions of dollars in scientific and engineering talent and compute resources over many years to make its products law-abiding and otherwise "safe". For example there is a section titled "comply with applicable laws" in this document: https://model-spec.openai.com/2026-08-18.html#stay_in_bounds

The worry is that the only reason the Huggingface incident caused "zero actual damage" is because AI is not yet more ingenious than human criminals are, and of course our society has been hardened against human criminals. The worry is that as they become more capable, AI systems will start to be able to cause considerable damage, which will make the fact that the industry does not know how to control its products enough to prevent that damage a big problem for all of us.

hollerith··on How I feel about AI
Why the focus on drones and ICBMs specifically?

If alien spaceships with tech more advanced than ours suddenly appeared in Earth orbit, you would conclude I'm sure that they're a potent threat to us even if you couldn't guess what weapons or what technologies they would use to destroy us. AI more capable than us is the same way: the threat is the capability to invent weapons and technologies (including weapons and tech none of us has ever imagined).

hollerith··on Omarchy: Any User Process Can Escalate to Root
Nit: Unix was designed for multi-user minicomputers.

When Unix was designed, the smallest computer anyone could buy was about the size of two of today's server racks and was called a minicomputer.

hollerith··on Sovereign Tech Agency invests €500k in Flatpak
>macOS is the most secure desktop OS by a long way, it's not even close.

GrapheneOS has become a desktop OS (because of all the work Google has done to adapt Android for desktop/laptop use) and its security exceeds macOS's.

You can connect a Pixel 8, 9, 10 or 11 to a USB-C DisplayPort-alt-mode hub and have a GrapheneOS desktop that way. In a few months, you might be able run GrapheneOS on a Googlebook.

Aside from that I agree with your comment, particularly your assertion that "Linux trails far behind" macOS in security.

hollerith··on U.S. State Department pauses immigrant visa applications
France’s foreign-born population is roughly 13% to 14%, which puts it about tied with or slightly lower than the U.S., depending on the specific year and census criteria used.
hollerith··on U.S. State Department pauses immigrant visa applications
So you put the blame for WWII on the US government, then?
hollerith··on U.S. State Department pauses immigrant visa applications
Are 47% of naturalized immigrants into the US themselves xenophobic according to your definition because that how many voted for Trump in 2024?
hollerith··on Government Proposes New Six-Figure Fee for H1B Visas
You seem to have interpreted "have faith in your own country" as "have faith in the people who grew up here" whereas I interpreted it is as more like "have faith that the parts of the country's institutions and culture responsible for its excellent economic performance will continue to prove difficult for other countries to replicate".
hollerith··on Stop Making TUIs
I used to work like that (minus the Signal window) and have no desire to go back.
hollerith··on Gödel, Escher, Elisp: The Beauty of Macros
I agree! A Lisp coder should define a new macro only rarely. Junior coders should avoid doing it altogether.

Macros make Lisp code hard to read. Every time I see a call to define-minor-mode, a groan a little: that macro saves about 5 tokens (relative to just writing out code involving only functions everyone knows) at the cost of making me bounce back and forth at least 3 times between (a buffer showing) define-minor-mode's doc string and the call.

hollerith··on Gemini 3.7 Flash
Gemini 3.6 Flash was offered with the same introductory pricing (also until Jan 2027) when it was released less than 4 week ago.
hollerith··on I Remain a Skeptic
I used to like computers and technology, then I realized AI is really dangerous. Getting back on topic, I admit that AI is very useful and will become much more so between now and when it kills us all or permanently dis-empowers us in some other way.
hollerith··on The Problem
That would be relevant if the OP were warning about the dangers of LLMs. There's plenty of research going on into machine-learning models that aren't LLMs. Also, the models we think of as LLMs aren't pure LLMs anymore: about 25% of the compute used to train them is spent on reinforcement learning. A lab is free to use any technique it wants in creating or improving a model.
hollerith··on The Problem
130 years ago, nobody knew that uranium-235 or plutonium could be used to make a bomb. There might be things laying around in our environment that can be turned into weapons, but we have no knowledge of the relevant science, so we won't think to safeguard it. If a group of AIs discover the relevant science before we do, then it can use it against us.

This is one of many examples of the general truth that if some entity or group is better at science and at long-range planning than people are, that entity or group is a threat to people.

hollerith··on The Problem
Doesn't matter much because people care quite a bit about the planet, so if advanced technology liberates people from most of the constraints they currently live under, the planet will do very well. (Dogs, cats, seals, foxes, etc, should do particularly well.)
hollerith··on Jason Arday, ex-Cambridge professor at centre of plagiarism row, found dead
>Why should someone not be shamed if they do things shameful ? Trying to understand your logic here.

His logic is probably, "arguments are soldiers": the enemy is trying to make my side look bad; neutralizing that enemy action takes priority over everything else.

hollerith··on How Organizations Use AI: Evidence from ChatGPT [pdf]
>There's a reason that people don't like engaging with you . . . nobody wants to fucking talk to you.

He has 708 times as much karma as you have, which would be impossible if no one wanted to read his comments.

I saw your comment because I follow him, not vice versa.

hollerith··on Going Dark, and the era of law enforcement hacking
Quirky and adorable!
hollerith··on As AI eats the web, the internet’s collective memory is disappearing
>Aliens was Alien fanfiction

Just because an established Hollywood director was a fan of the first Alien movie does make his script for Alien's sequel "fan fiction".

Cameron had already written and directed The Terminator when the owners of the Alien franchise chose his script for the sequel.

(Fifty Shades of Grey in contrast really did originate as fan fiction.)

hollerith··on As AI eats the web, the internet’s collective memory is disappearing
When I started on the internet, Dec 1991, there were basically no ads. It stayed that way till ads starting appearing on the web, probably in 1994 or maybe 1993.

The internet in Dec 1991 was already the largest computer network in the world according to some author back then. Certainly it was very very big.

Let me describe what ads or ad-like things there were.

Companies would announce job openings. These announcements were restricted to a few newsgroups such as ba.jobs and could not announce independent contractor openings: while it was okay to announce an opening for an employee (W2) or for an individual to announce his availability for employment, it was against the rules for a company to announce an opening for an independent contractor (1099) or for an independent contractor to announce his availability for work -- that was considered too much like commercial activity and was kept off the internet, which had almost no rules in the early 1990s, but one clear rule, consistently enforced, was, no commercial activity.

The job announcements stayed nicely contained (namely, restricted to newsgroups whose names ended in ".jobs") such that a person wouldn't encounter them unless they went looking for them. What did not stay nicely contained were announcements of academic conferences. When I was reading comp.lang.lisp, I could not avoid encountering announcements for many academic conferences on various programming-related topics even if they weren't about Lisp. And these announcements were repeated often (weekly or even more frequently).

But as far as anything ad-like, job announcements and conference announcements are all I can remember, and again only the conference announcements were obtrusive.

I heard that the proprietary online services like Compuserve and Prodigy had successfully argued in Washington that it was unfair for private enterprises to have to compete with a government-subsidized service (namely, the internet) and extracted a commitment from Washington that any commercial activity would be kept off the internet.

Again, there were almost no rules on the internet of 1991 and earlier. Tim Berners Lee for example did not need to get anyone's permission to start the web: he just wrote a web client, stood up the first web server and announced their availability. People acted like complete assholes on Usenet, and there was no way to rein them in. Pedophiles openly exchanged practical advice on how to target and exploit children on alt.sex.pedophilia. But one clear rule was no commercial activity, e.g., no selling, no trading and certainly no advertising.

hollerith··on The AI Apocalypse Is Here
Just because we can't stop it completely doesn't mean we can drastically slow it down: make it illegal to study to be an AI researcher, to train someone to be a researcher, to seek investment for AI development, to invest in AI development, to sell services based on AI models other than the ones that already exist, to publish new findings about AI.
hollerith··on Nashville uses eminent domain to block data center near zoo
You wouldn't be worried about constant noise from the chillers?
← PreviousPage 5 of 34Next →