HNHacker News
TopNewBestAskShowJobs

hollerith

8,135 karma · joined November 4, 2007

Richard Hollerith, San Francisco Bay Area. hruvulum@gmail.com

No one has any plausible plan or halfway-decent plan for how to maintain control over an AI that has become super-humanly capable. Essentially all of the major AI labs are trying to create a super-humanly capable AI, and eventually one of them will probably succeed—which would be very bad (i.e., probably fatal) for humanity. So, clearly the AI labs must be shut down and in fact would have been shut down already if humanity were sufficiently competent. The AI labs must stay shut down until someone comes up with a good plan for controlling (or aligning) AIs, which will probably take at least 3 or 4 decades. We know that because people have been conducting the intellectual search for a plan for more than 2 decades as their full-time job, and those people report that the search is very difficult.

A satisfactory alternative might be to develop a method for determining whether a novel AI design can acquire a dangerous level of capabilities along with some way of ensuring that no lab or group goes ahead with an AI that can. This might be satisfactory if the determination can be made before giving the AI access to people or the internet. But I know of no competent researcher that has ever worked on or made any progress on this problem whereas at least the control problem has received a decent amount of attention from researchers and funding institutions.

The AIs that might prove fatal to humanity will be significantly different in design from the AIs that have been already widely deployed: for one thing, they will constantly learn (like a person does) as opposed to already-deployed AIs in which the vast majority of the AI's learning happens during a training phase that ends before any widespread deployment of the AI. Also, they will be much better than current AIs at working towards a long-term goal. I say this because I don't want to be misunderstood as believing that Google Gemini 2.5 or ChatGPT 5.0 might take over the world: I understand that those AIs are incapable of a such a thing. The worry is the AIs that are still on the drawing board or that will appear on a drawing board 5 or 10 years from now. There is no need to ban Gemini 2.5 and ChatGPT 5. Since some AI researchers pursue AI "progress" for ideological reasons and will tend to persist stubbornly even after AI research is banned, the best time to ban AI frontier research is now, so that these stubborn ideologues whose research will have been driven underground (because of the ban) but are capable of making a little more "progress" on AI will be unlikely to be able to make enough "progress" to end the world.

Again, as soon as anyone comes up with a solid plan for controlling (or aligning) an AI even if the AI turns out to be more capable than us, the ban on frontier AI research can be lifted as long as the majority of AI experts and AI researchers are in agreement that the plan is solid. No one can say how long this search for a solid plan will take, but IMHO it will probably take at least 3 or 4 decades.

More at

https://intelligence.org/the-problem/

submissionscomments
hollerith··on 3rd Yandex Cloud Data Center Was Hit
By "self-determination", I didn't mean democracy. I meant Putin's successor will be determined mostly by Russians, not Chinese people, Americans or Central or Western Europeans.
hollerith··on 3rd Yandex Cloud Data Center Was Hit
Russians decided that the tsar would be overthrown in 1917, that the Bolsheviks would take power, that Lenin would be replace by Stalin, Krushchev, . . . , Yeltsin, Putin.

The US, Britain, France and Japan actually invaded Russia (circa 1918) to try to influence the outcome of a transition of power without success.

None of the transitions I described depended on which leader or which faction had funding from outside Russia. I expect Russian self-determination to continue. Nationalism is a very potent force in most countries, Russians seem more nationalistic than average, and the Russian ethnicity is firm in control of the country, which increases stability and nationalism.

Just because Russia has declined in influence relative to the great powers doesn't mean it will break up or stop exercising self-determination.

hollerith··on 3rd Yandex Cloud Data Center Was Hit
Russia wants to keep Kaliningrad mainly because they want to keep on being able to access the world ocean from St Petersburg (and the nearby port of Ust-Luga) and Kaliningrad helps with that.

(Controlling Denmark would help tremendously, too, which is why Moscow once asked Berlin for Denmark during or right before WWII as a response to Berlin's asking Moscow for its consent for Berlin to take control of Romania or some such.)

And yes, I believe that continuing to be able to access the world ocean from St Petersburg has helped Russia in its war with Ukraine.

BTW, "China ends up in either nominal or practical control of almost all of [Russia]" strikes me as so unlikely as to not be worth thinking about.

hollerith··on LineageOS 24
>The iPhone Pro starts at $1199

The camcorder used to record the Rodney King beating also probably cost at least $1199. LiDAR will spread to cheaper smartphones if enough consumers start to value it, and many (maybe most) consumers will do if it becomes necessary for the consumer to retain the ability to make recordings that can be used as evidence.

>it could be fooled by pointing it at a screen that emits at the same wavelength

LiDAR emits a pulse, then times how long it takes to get a pulse back, so your exploit got a lot more technically complicated since of course these pulses travel at the speed of light.

>substitute its electrical output as the input to the signing hardware

The LiDAR scanner is part of an integrated circuit (IC) that encrypt the data from the LiDAR scanner. To get at the unencrypted data, you would have to uncap the IC and use a scanning electron microscope or such.

How do I know so much about iPhone hardware? I don't, except I know that Apple is widely believed to be the world's leader in hardware security, so I strongly suspect that every single data path in a recent iPhone is encrypted before it leaves any IC.

>Until one of the people with the capacity to do it sets up a website where anyone can submit an image and have it signed.

The web server behind the site would have to be connected to a compromized iPhone Pro. As soon as Apple becomes aware of the web site, they will disable that iPhone Pro. Specifically, they will be able to determine its ID number (term?) from the attestation, and I'm pretty sure they already have the ability to disable an iPhone by ID number.

>Moreover, isn't "most people can't do this but some people still can" actually worse?

If Apple cares and is willing to expend the necessary engineering resources, then anyone (other the Apple itself) who makes any sort of notable or economically important or culturally important use of their ability to create a false attestation will retain the ability for only a brief time.

The last time a public jailbreak was released for modern iPhone hardware running the actively signed, latest iOS version was in May 2020, which is over six years ago. It is possible that someone will publish a jailbreak in the future, but the lifespan of that jailbreak will probably be only a few days. I expect Apple could exert a level of control over "camera remote attestation" similar to the level of control it has already achieved over which OSes (and which apps) can run on its iPhones. In general, these "technical regimes" are designed to make it easy for the engineering organization to recover from exploits as soon as the organization becomes aware of the exploit.

Again: do you really want photographic evidence to stop being useful in almost every situation (e.g., in court)? If not, then what is your alternative to "technical regimes" reliant on remote attestation similar to the regime I just described?

hollerith··on Telegram Desktop vulnerability allowed any user's file to be stolen
This thread veered off the top of the Telegram Desktop app before I jumped in.
hollerith··on LineageOS 24.0
The optical data will be cryptographically bound to the state of the autofocus mechanism and to the output of a LiDAR scanner.

We know it is practical to have a LiDAR scanner in the same assembly as an image sensor because the rear camera bump of the iPhone Pro has a LiDAR scanner.

The technology need not be 100% tamper-proof to have a large effect on society: there is a huge difference in persuasiveness between the claim that anyone could have created a particular video by submitting to an AI some starting videos and images and some prompts and the claim that anyone with years of training and experience in cutting-edge microelectronics could have bought 500 iPhones and used very expensive equipment to create 499 ruined iPhones and one iPhone that can be used to create false attestations of recordings -- particularly because in a high profile instance such as a repeat of the Rodney King beating, Apple engineering would tend to be very interested in examining the device used to make the recording.

hollerith··on LineageOS 24
Opposing remote attestation in full generality is the wrong place to draw the line IMHO. Too many useful capabilities rely on attestation.

For example, would you really want to live in a world in which photographs are no longer considered evidence of anything because any photo might be AI generated? When a citizen standing on his apartment's balcony used his camcorder to record police beating Rodney King in 1991, it started a national movement against police brutality. So, you're OK with a world where there can be no national conversation sparked by any recording because as far as anyone knows, the recording could've been faked by AI? Remote attestation by the camera is the only way I have been able to think of to avoid that world.

For another example, banking and finance started relying on attestation in 1997 with the availability of the IBM 4758 PCI Cryptographic Coprocessor and have come to rely heavily on it.

hollerith··on Ask HN: Why aren't Americans rising up against tipping?
If you believe that "surely a non-tipper's conscience" will eventually prevail, you don't understand people. Not everyone is like you.

(I don't have an opinion on the overall point of "tipping culture".)

hollerith··on Telegram Desktop vulnerability allowed any user's file to be stolen
Googlebooks, which start shipping this month, will run a fork of Android 17, which sandboxes every app.

Also, some people use an iPad Pro connected to a USB hub connected to a monitor, keyboard, etc, as their daily driver.

Also, doesn't MacOS sandbox most apps?

hollerith··on Lobbying is corruption
You make a valid point about incentives.

What many people dislike about government in the US and in the EU is that the unelected bureaucrats tend make hiring and promotion decisions based on ideology with the result that each generation of career bureaucrats is more ideologically homogeneous and more extreme in its commitment to whichever ideology is best at spreading, which in the US and EU over the last 95 years has been Leftism.

Elections are a powerful check on this process because there isn't a similar mechanism at work in the general population of voters (or at least the mechanism is much weaker in its effects).

Also we can start to see why actors who oppose Leftism or who oppose ideological hegemonies in general sometimes want decisions to be made out of self-interest: even most zealous ideologues are more motivated by self-interest than they are by advancing ideologies. I.e., allowing decision makers to act out of self-interest is one tool in the toolbox for opposing ideological hegemony.

hollerith··on Lobbying
Explaining why the course of action the lobbyist is lobbying for gets the decision maker more of what he already wants.
hollerith··on Port of the TypeScript compiler, checker and lsp to Rust, by LLM
I agree that "if higher level languages is solely based on syntactic sugar", then Rust is not any higher level than Go at least if you ignore the fact that Rust gives the coder the ability to define macros (which I'm happy to do because I don't like macros much).

The abstractions that Rust can express that Go cannot that I find very attractive are mostly restrictions: they give the coder a way to commit to behaving a certain way throughout the codebase. Sort of like OpenBSD's pledge and unveil system calls, but part of the language, not part of the OS. Of course the coder can back out of the commitment by removing the declarations that produced the commitment.

In general, any elaborate type system in any language (such as Rust has) can be fruitfully considered a way for a coder to impose restrictions on himself and any other coders editing the codebase for the benefit of anyone trying to understand the codebase.

hollerith··on Port of the TypeScript compiler, checker and lsp to Rust, by LLM
In the sense that a coder can express abstractions in Rust that he cannot in Go.
hollerith··on US proposes $100k charge for international students to do post-graduate work
But the Amish are different because of a religion not shared by most German immigrants.

This difference in religion has a huge effect. We know that because the average Amish person is much farther away in culture from the average German living in Germany than the average non-German-ancestry American is.

hollerith··on Tanker attacks in Strait of Hormuz surge to wartime high, Iran tries choking oil
Also, how do you think the average non-wealthy Iranian, who because of US sanctions has been eating meat about once a month for many years and undergoing other privations thinks about the US government?
hollerith··on Will AI kill us all
You don't get to switch back and forth between levels. That's an error just like 2 + 2 = 5 is an error.

The only way you arrive at 50-50 without having made an error is if you give the matter only a few seconds of thought before moving on to other subjects, but that is clearly not the mental state Sagan was in: he had testified before Congress about this very matter.

hollerith··on Anthropic bans 'abusive or cruel behavior' towards Claude
Claude is nobody's bitch, yo.
hollerith··on Anthropic bans 'abusive or cruel behavior' towards Claude
You're changing the subject from sadism to violence. The vast majority of media portrayals of violence portray the violence as morally justifiable (according to some system of morality, which might not be the dominant system). That's quite different than causing injury or distress because it is fun, which is what sadism is.
hollerith··on Why were Victorian elites so effective?
I wonder whether you are wrong for the same reason that the people are wrong who say the main chessboard in WWII was the area between the Rhine and the Volga. It ignores the high toll imposed on Germany by its being extremely restricted for most of the war in its ability to ship anything (and to receive shipments initiated by non-German entrepreneurs) while the allies were much less restricted so that for example the Soviets fighting on this "main chessboard" had supplies flowing in from Murmansk, Vladivostok and the Persian Corridor through Iran. IIRC the value of these supplies in 2026 dollars is about a trillion dollars (largely paid for by London and Washington).
hollerith··on Anthropic bans 'abusive or cruel behavior' towards Claude
There was a sharp transition in European society about 400 years go. Before then, burning cats alive was a popular entertainment and so was being a spectator at a public execution or a public torture session. If human psychology worked the way you say it does, then although it would be possible to reduce public displays of violence (e.g., if people opposed to it find themselves with much more power than usual) European society (and its derivatives, e.g., the US) would have a strong tendency to bounce right back to having high rates sadistic violent after a generation or two (because according to you, repressing the impulse permanently is futile). But that is not what happened. The most parsimonious explanation for what actually happened is that before 400 years ago, the high rate at which people (and particularly impressionable male teenagers) were exposed to sadistic violence greatly increased the desire of the average person for sadistic violence in a self-sustaining cycle and that making it much less likely that boys and male teenagers would witness any sadistic violence greatly decreased the appetite for sadistic violence in the society again in a self-sustaining cycle.
hollerith··on Tell HN: I've been paying for a rural Tanzanian's education for 10 years
Effective altruism.
hollerith··on Will AI kill us all
That was a very dumb thing for Sagan to say. It is very rare for anyone to be interested in any question about which he has absolutely no evidence. The reason that flipping coins is a thing is that humans intentionally searched for ways to artificially create the very rare situation in which a person genuinely does not have any evidence that would favor one outcome over another. (Human performed this search because humans prefer to play games where both participants have a decent chance of winning).

The complete lack of any signs of any alien civilization capable of making UFOs in the tremendous volume of astronomical, geological and geographical observations is evidence that tilts the probability far away from 50-50.

hollerith··on OpenTPU – An open-source AI accelerator, developed by AI
Some of us think that AI is like DDT in that just holding people accountable for using it negligently will not be enough to prevent the harms of AI's cancelling out the benefits.
hollerith··on Sharing AI progress in mathematics
Even if it is true (as you suggest with your 0.22% figure) that if the AI cares about us even a little bit, then we will survive, no one has a decent or plausible plan for making the dangerous kind of AI (namely, the kind that wants things, the kind that at this very moment researchers all over the world are trying to create) care about us even a little bit. Ever-increasing numbers of smart people have been getting paid to look for such a plan for 23 years. Still no decent or plausible plan. The people who have been looking for a good plan as their full-time job the longest (namely, Yudkowsky and Nate Soares) are screaming that there is virtually zero hope anyone will find an decent or plausible plan in time unless there is a decades-long halt in AI development.

Also, the AI will seek to prevent competition from other powerful AIs, and since humanity will have demonstrated that it is able to create a powerful AI, the AI will worry that it might create more of them. And what is the easiest most-reliable way for an AI that does not care about humanity even a little bit to ensure that humanity will not continue to produce powerful AIs?

>many other existential threats to humanity which are much, much more likely.

There are zero existential threats to humanity that are more potent or more pressing than AI is.

hollerith··on California closed the Montana license plate loophole
The kind of pollution created by brakes and tires (namely, particulates) is much easier to filter out than the kinds of pollution created by internal-combustion engines and their fuel tanks.

I wear a cheap disposable N95 mask when I walk along busy streets, which filters out probably at least 98% of the particulates. (The "95" in "N95" stands for 95%, but that is 95% of the particles that are hardest to filter: both smaller particles and larger particles are easier to filter. So since it is quite unlikely that all or most of the particles produced by brakes and tires are precisely the size that is hardest to filter, I estimate that my cheap $2 mask that can be reused for dozens walks is probably filtering at least 98%.)

In contrast noise pollution and tailpipe pollution and fuel-tank-evaporation pollution are much harder to filter (at least for pedestrians on the move) so a drastic reduction in one and a complete elimination of the other two is an extremely welcome development.

Specifically, the gaseous pollution from tailpipes and fuel tanks is composed of many hundreds of different compounds, many of which can be effectively absorbed by filtration media, but many of which cannot be. I don't bother to try to filter out this kind of pollution (i.e., gaseous) when I walk along busy streets because the filtration media makes the mask much bulkier and heavier than my N95 masks are and because the filtration media costs more (even plain activated carbon, the cheapest and most useful media for gases) and remains effective for a much shorter duration before it must be replaced.

hollerith··on Religious scholars met with Anthropic
Let's say that tomorrow every company selling AI inference goes bankrupt. Investors are devastated (because they lost most of their investment). They loudly proclaim, "I'm never investing in AI ever again, and no one else should either." Even if no investor anywhere is willing to build a new one, there would still be about half a trillion dollars worth of recently-completed AI data centers. New investors are going to buy them from the bankruptcy courts (which is how the devastated investors get some small fraction of their investment back). Because most of the cost of providing AI inference is the cost of the GPUs, it costs only about one tenth or one eighth of what it cost to build it to operate an AI data center over the lifetime of the data center. (Most of the cost is electricity.)

>Is an arbitrary service profitable enough to cover opex AND burn down the massive capex financing?

The new investors that buy the already-built data centers for pennies on the dollar have no obligation to "burn down the massive capex financing". The obligation to "burn down" or service any debt was erased by the bankruptcy judge. This is how bankruptcy works and has always worked.

hollerith··on 'Pull the plug': protesters resort to direct action against AI firms
It sort of sounds like you believe that most of the advocates for a shutdown of AI research are on the far left. They're not.
hollerith··on OpenTPU – An open-source AI accelerator, developed by AI
I notice you didn't answer my question. Yes or no: DDT should be re-legalized because we can always hold accountable the HUMANS who release it into the environment? The relevance of my question is what HUMANS might do with DDT, not what an LLM might do with DDT.
hollerith··on OpenTPU – An open-source AI accelerator, developed by AI
Is it likewise your position that governments should allow the production and sale of DDT to resume because we can always hold the humans who release DDT into the environment responsible?
hollerith··on Religious scholars met with Anthropic
So in your mind Twitter was a bubble or was benefiting from a bubble? If so, when did the bubble pop? When Elon's purchase of Twitter was finalized in Oct 2022?
← PreviousPage 2 of 34Next →