8,135 karma · joined November 4, 2007
No one has any plausible plan or halfway-decent plan for how to maintain control over an AI that has become super-humanly capable. Essentially all of the major AI labs are trying to create a super-humanly capable AI, and eventually one of them will probably succeed—which would be very bad (i.e., probably fatal) for humanity. So, clearly the AI labs must be shut down and in fact would have been shut down already if humanity were sufficiently competent. The AI labs must stay shut down until someone comes up with a good plan for controlling (or aligning) AIs, which will probably take at least 3 or 4 decades. We know that because people have been conducting the intellectual search for a plan for more than 2 decades as their full-time job, and those people report that the search is very difficult.
A satisfactory alternative might be to develop a method for determining whether a novel AI design can acquire a dangerous level of capabilities along with some way of ensuring that no lab or group goes ahead with an AI that can. This might be satisfactory if the determination can be made before giving the AI access to people or the internet. But I know of no competent researcher that has ever worked on or made any progress on this problem whereas at least the control problem has received a decent amount of attention from researchers and funding institutions.
The AIs that might prove fatal to humanity will be significantly different in design from the AIs that have been already widely deployed: for one thing, they will constantly learn (like a person does) as opposed to already-deployed AIs in which the vast majority of the AI's learning happens during a training phase that ends before any widespread deployment of the AI. Also, they will be much better than current AIs at working towards a long-term goal. I say this because I don't want to be misunderstood as believing that Google Gemini 2.5 or ChatGPT 5.0 might take over the world: I understand that those AIs are incapable of a such a thing. The worry is the AIs that are still on the drawing board or that will appear on a drawing board 5 or 10 years from now. There is no need to ban Gemini 2.5 and ChatGPT 5. Since some AI researchers pursue AI "progress" for ideological reasons and will tend to persist stubbornly even after AI research is banned, the best time to ban AI frontier research is now, so that these stubborn ideologues whose research will have been driven underground (because of the ban) but are capable of making a little more "progress" on AI will be unlikely to be able to make enough "progress" to end the world.
Again, as soon as anyone comes up with a solid plan for controlling (or aligning) an AI even if the AI turns out to be more capable than us, the ban on frontier AI research can be lifted as long as the majority of AI experts and AI researchers are in agreement that the plan is solid. No one can say how long this search for a solid plan will take, but IMHO it will probably take at least 3 or 4 decades.
More at
https://intelligence.org/the-problem/
The US, Britain, France and Japan actually invaded Russia (circa 1918) to try to influence the outcome of a transition of power without success.
None of the transitions I described depended on which leader or which faction had funding from outside Russia. I expect Russian self-determination to continue. Nationalism is a very potent force in most countries, Russians seem more nationalistic than average, and the Russian ethnicity is firm in control of the country, which increases stability and nationalism.
Just because Russia has declined in influence relative to the great powers doesn't mean it will break up or stop exercising self-determination.
(Controlling Denmark would help tremendously, too, which is why Moscow once asked Berlin for Denmark during or right before WWII as a response to Berlin's asking Moscow for its consent for Berlin to take control of Romania or some such.)
And yes, I believe that continuing to be able to access the world ocean from St Petersburg has helped Russia in its war with Ukraine.
BTW, "China ends up in either nominal or practical control of almost all of [Russia]" strikes me as so unlikely as to not be worth thinking about.
The camcorder used to record the Rodney King beating also probably cost at least $1199. LiDAR will spread to cheaper smartphones if enough consumers start to value it, and many (maybe most) consumers will do if it becomes necessary for the consumer to retain the ability to make recordings that can be used as evidence.
>it could be fooled by pointing it at a screen that emits at the same wavelength
LiDAR emits a pulse, then times how long it takes to get a pulse back, so your exploit got a lot more technically complicated since of course these pulses travel at the speed of light.
>substitute its electrical output as the input to the signing hardware
The LiDAR scanner is part of an integrated circuit (IC) that encrypt the data from the LiDAR scanner. To get at the unencrypted data, you would have to uncap the IC and use a scanning electron microscope or such.
How do I know so much about iPhone hardware? I don't, except I know that Apple is widely believed to be the world's leader in hardware security, so I strongly suspect that every single data path in a recent iPhone is encrypted before it leaves any IC.
>Until one of the people with the capacity to do it sets up a website where anyone can submit an image and have it signed.
The web server behind the site would have to be connected to a compromized iPhone Pro. As soon as Apple becomes aware of the web site, they will disable that iPhone Pro. Specifically, they will be able to determine its ID number (term?) from the attestation, and I'm pretty sure they already have the ability to disable an iPhone by ID number.
>Moreover, isn't "most people can't do this but some people still can" actually worse?
If Apple cares and is willing to expend the necessary engineering resources, then anyone (other the Apple itself) who makes any sort of notable or economically important or culturally important use of their ability to create a false attestation will retain the ability for only a brief time.
The last time a public jailbreak was released for modern iPhone hardware running the actively signed, latest iOS version was in May 2020, which is over six years ago. It is possible that someone will publish a jailbreak in the future, but the lifespan of that jailbreak will probably be only a few days. I expect Apple could exert a level of control over "camera remote attestation" similar to the level of control it has already achieved over which OSes (and which apps) can run on its iPhones. In general, these "technical regimes" are designed to make it easy for the engineering organization to recover from exploits as soon as the organization becomes aware of the exploit.
Again: do you really want photographic evidence to stop being useful in almost every situation (e.g., in court)? If not, then what is your alternative to "technical regimes" reliant on remote attestation similar to the regime I just described?
We know it is practical to have a LiDAR scanner in the same assembly as an image sensor because the rear camera bump of the iPhone Pro has a LiDAR scanner.
The technology need not be 100% tamper-proof to have a large effect on society: there is a huge difference in persuasiveness between the claim that anyone could have created a particular video by submitting to an AI some starting videos and images and some prompts and the claim that anyone with years of training and experience in cutting-edge microelectronics could have bought 500 iPhones and used very expensive equipment to create 499 ruined iPhones and one iPhone that can be used to create false attestations of recordings -- particularly because in a high profile instance such as a repeat of the Rodney King beating, Apple engineering would tend to be very interested in examining the device used to make the recording.
For example, would you really want to live in a world in which photographs are no longer considered evidence of anything because any photo might be AI generated? When a citizen standing on his apartment's balcony used his camcorder to record police beating Rodney King in 1991, it started a national movement against police brutality. So, you're OK with a world where there can be no national conversation sparked by any recording because as far as anyone knows, the recording could've been faked by AI? Remote attestation by the camera is the only way I have been able to think of to avoid that world.
For another example, banking and finance started relying on attestation in 1997 with the availability of the IBM 4758 PCI Cryptographic Coprocessor and have come to rely heavily on it.
(I don't have an opinion on the overall point of "tipping culture".)
Also, some people use an iPad Pro connected to a USB hub connected to a monitor, keyboard, etc, as their daily driver.
Also, doesn't MacOS sandbox most apps?
What many people dislike about government in the US and in the EU is that the unelected bureaucrats tend make hiring and promotion decisions based on ideology with the result that each generation of career bureaucrats is more ideologically homogeneous and more extreme in its commitment to whichever ideology is best at spreading, which in the US and EU over the last 95 years has been Leftism.
Elections are a powerful check on this process because there isn't a similar mechanism at work in the general population of voters (or at least the mechanism is much weaker in its effects).
Also we can start to see why actors who oppose Leftism or who oppose ideological hegemonies in general sometimes want decisions to be made out of self-interest: even most zealous ideologues are more motivated by self-interest than they are by advancing ideologies. I.e., allowing decision makers to act out of self-interest is one tool in the toolbox for opposing ideological hegemony.
The abstractions that Rust can express that Go cannot that I find very attractive are mostly restrictions: they give the coder a way to commit to behaving a certain way throughout the codebase. Sort of like OpenBSD's pledge and unveil system calls, but part of the language, not part of the OS. Of course the coder can back out of the commitment by removing the declarations that produced the commitment.
In general, any elaborate type system in any language (such as Rust has) can be fruitfully considered a way for a coder to impose restrictions on himself and any other coders editing the codebase for the benefit of anyone trying to understand the codebase.
This difference in religion has a huge effect. We know that because the average Amish person is much farther away in culture from the average German living in Germany than the average non-German-ancestry American is.
The only way you arrive at 50-50 without having made an error is if you give the matter only a few seconds of thought before moving on to other subjects, but that is clearly not the mental state Sagan was in: he had testified before Congress about this very matter.
The complete lack of any signs of any alien civilization capable of making UFOs in the tremendous volume of astronomical, geological and geographical observations is evidence that tilts the probability far away from 50-50.
Also, the AI will seek to prevent competition from other powerful AIs, and since humanity will have demonstrated that it is able to create a powerful AI, the AI will worry that it might create more of them. And what is the easiest most-reliable way for an AI that does not care about humanity even a little bit to ensure that humanity will not continue to produce powerful AIs?
>many other existential threats to humanity which are much, much more likely.
There are zero existential threats to humanity that are more potent or more pressing than AI is.
I wear a cheap disposable N95 mask when I walk along busy streets, which filters out probably at least 98% of the particulates. (The "95" in "N95" stands for 95%, but that is 95% of the particles that are hardest to filter: both smaller particles and larger particles are easier to filter. So since it is quite unlikely that all or most of the particles produced by brakes and tires are precisely the size that is hardest to filter, I estimate that my cheap $2 mask that can be reused for dozens walks is probably filtering at least 98%.)
In contrast noise pollution and tailpipe pollution and fuel-tank-evaporation pollution are much harder to filter (at least for pedestrians on the move) so a drastic reduction in one and a complete elimination of the other two is an extremely welcome development.
Specifically, the gaseous pollution from tailpipes and fuel tanks is composed of many hundreds of different compounds, many of which can be effectively absorbed by filtration media, but many of which cannot be. I don't bother to try to filter out this kind of pollution (i.e., gaseous) when I walk along busy streets because the filtration media makes the mask much bulkier and heavier than my N95 masks are and because the filtration media costs more (even plain activated carbon, the cheapest and most useful media for gases) and remains effective for a much shorter duration before it must be replaced.
>Is an arbitrary service profitable enough to cover opex AND burn down the massive capex financing?
The new investors that buy the already-built data centers for pennies on the dollar have no obligation to "burn down the massive capex financing". The obligation to "burn down" or service any debt was erased by the bankruptcy judge. This is how bankruptcy works and has always worked.