8,134 karma · joined November 4, 2007
No one has any plausible plan or halfway-decent plan for how to maintain control over an AI that has become super-humanly capable. Essentially all of the major AI labs are trying to create a super-humanly capable AI, and eventually one of them will probably succeed—which would be very bad (i.e., probably fatal) for humanity. So, clearly the AI labs must be shut down and in fact would have been shut down already if humanity were sufficiently competent. The AI labs must stay shut down until someone comes up with a good plan for controlling (or aligning) AIs, which will probably take at least 3 or 4 decades. We know that because people have been conducting the intellectual search for a plan for more than 2 decades as their full-time job, and those people report that the search is very difficult.
A satisfactory alternative might be to develop a method for determining whether a novel AI design can acquire a dangerous level of capabilities along with some way of ensuring that no lab or group goes ahead with an AI that can. This might be satisfactory if the determination can be made before giving the AI access to people or the internet. But I know of no competent researcher that has ever worked on or made any progress on this problem whereas at least the control problem has received a decent amount of attention from researchers and funding institutions.
The AIs that might prove fatal to humanity will be significantly different in design from the AIs that have been already widely deployed: for one thing, they will constantly learn (like a person does) as opposed to already-deployed AIs in which the vast majority of the AI's learning happens during a training phase that ends before any widespread deployment of the AI. Also, they will be much better than current AIs at working towards a long-term goal. I say this because I don't want to be misunderstood as believing that Google Gemini 2.5 or ChatGPT 5.0 might take over the world: I understand that those AIs are incapable of a such a thing. The worry is the AIs that are still on the drawing board or that will appear on a drawing board 5 or 10 years from now. There is no need to ban Gemini 2.5 and ChatGPT 5. Since some AI researchers pursue AI "progress" for ideological reasons and will tend to persist stubbornly even after AI research is banned, the best time to ban AI frontier research is now, so that these stubborn ideologues whose research will have been driven underground (because of the ban) but are capable of making a little more "progress" on AI will be unlikely to be able to make enough "progress" to end the world.
Again, as soon as anyone comes up with a solid plan for controlling (or aligning) an AI even if the AI turns out to be more capable than us, the ban on frontier AI research can be lifted as long as the majority of AI experts and AI researchers are in agreement that the plan is solid. No one can say how long this search for a solid plan will take, but IMHO it will probably take at least 3 or 4 decades.
More at
https://intelligence.org/the-problem/
An unreliable source writes, "litigants frequently use the Administrative Procedure Act to argue that an agency acted in an arbitrary and capricious manner, failed to follow required notice-and-comment procedures, or exceeded its statutory authority."
Not only is he an American, but also he routinely makes arguments to American Federal judges very similar to the argument he is making here or at least he is a colleague and coworker of people who do.
How is a very large operation more efficient at producing comic-book sales than a smaller operation?
Why cannot new comic-book companies easily be created? One guess I have is that most of the profit potential is in sales of new stories about characters that are well-known, like Superman or the Incredible Hulk, and copyright law prevents new entrants from selling those stories.
It was often paired with another term which I cannot confidently recall, maybe "the information economy".
Although Chinese imports didn't become an important factor in the US economy till the 1990s, Deng's rise to power happened in 1978 and his famous "Southern Tour" happened in 1982. Both events committed China to a course of economic modernization, welcoming Western investment and special economic zones designed to boost exports, which the US media did not fail to notice.
Trump was the first president in the modern era to try to stem the flow of factory jobs overseas. Clinton for example was all-in on globalization.
Amazon, Tesla, Netflix, and Palo Alto Networks have never paid a dividend at any point in their histories as publicly traded companies.
Advanced Micro Devices hasn't paid one since 1995, Adobe since 2005.
Walk me through that. In 1999, some people proposed splitting up Microsoft. One spinoff would own Internet Explorer, another would own the office suite, a third would own Windows.
You could spin off Instagram and Whatsapp, but after you do that, Facebook is still really big -- worth many 100s of billions. How to split that up?
Every user east of the Mississippi ends up in Facebook East and every user to the west in Facebook West??
OK, to be more precise: it would still have a market cap of "multiple hundred billions of dollars" if it never acquired or merged with any other companies.
The US is self-sufficient in oil, so naturally it hasn't invested as heavily in risky ventures to reduce its dependence on oil as a country that gets most of its oil from a long supply line (past potentially hostile powers like India) to the Persian Gulf and even further through to Russia's European ports.
Is it not cherry picking to focus on 3 CEOs and ignore the 145,132 people who signed a "call for a prohibition on the development of superintelligence":
https://superintelligence-statement.org/
Does the opinion of these 145,132 people not count because they're not CEOs of AI megacorporations?
Also, people can be so consumed with the pursuit of wealth and power (and sometime attention) that they can't bring themselves to voluntarily stop that pursuit even when they know that stopping is the right thing to do.
Name a company with a currently high profit margin on inference. I can't.
OpenAI and Anthropic's valuations are as high as they are because investors hope that they will create AI that is much much more powerful than the AI they have now. The fact that they already have a substantial revenue stream helps, but it helps mainly by lending credibility to the hope that they might create something much much more powerful.
(You, Mike and I agree that selling the inference of models that are not much better than liberally-licensed open-weights models will be a low-margin business.)
Through good planning, France weathered the supply shock around Russian natural gas better than most of Europe.
No military dares to fuck with France.
Life in France for the average person is good. Many want to move there. Many want to visit.
France functions better than I would have thought possible of a country so attracted to Leftism and 'direct political action'.
I would phrase it differently: democratic elections everywhere tend to be won by promising free stuff, and even the US is not immune to this tendency.
During the first 140 years of democratic elections in the US, the average voter got approximately zero free stuff. E.g., when the Dust Bowl made 100s of 1000s of families completely destitute, they received no substantial assistance from (state or Federal) governments: the free stuff started flowing a couple of years after that when FDR took office.
BTW the US economy was already 3 times the size of the next largest economy (the UK) at the start of the Dust Bowl.
One reason the democratic disease of free stuff took so long to manifest in the US is that the disease is largely incompatible with the central principle of the US theory of governance, namely, to maximize freedom for the average person.
Another reason is that life was easier for the average person in the US than it was almost anywhere else in the world. E.g., there were no invasions by Mongols, Teutonic Knights, Tatars, Turks, Finns, Swedes or Napoleonic armies that severely interfered with millions of lives as was the case in Russia. (Native Americans never posed a comparable military threat to European settlers. E.g., they never took and held a settler city or even a settler town that I know of.)
The US, Britain, France and Japan actually invaded Russia (circa 1918) to try to influence the outcome of a transition of power without success.
None of the transitions I described depended on which leader or which faction had funding from outside Russia. I expect Russian self-determination to continue. Nationalism is a very potent force in most countries, Russians seem more nationalistic than average, and the Russian ethnicity is firm in control of the country, which increases stability and nationalism.
Just because Russia has declined in influence relative to the great powers doesn't mean it will break up or stop exercising self-determination.
(Controlling Denmark would help tremendously, too, which is why Moscow once asked Berlin for Denmark during or right before WWII as a response to Berlin's asking Moscow for its consent for Berlin to take control of Romania or some such.)
And yes, I believe that continuing to be able to access the world ocean from St Petersburg has helped Russia in its war with Ukraine.
BTW, "China ends up in either nominal or practical control of almost all of [Russia]" strikes me as so unlikely as to not be worth thinking about.
The camcorder used to record the Rodney King beating also probably cost at least $1199. LiDAR will spread to cheaper smartphones if enough consumers start to value it, and many (maybe most) consumers will do if it becomes necessary for the consumer to retain the ability to make recordings that can be used as evidence.
>it could be fooled by pointing it at a screen that emits at the same wavelength
LiDAR emits a pulse, then times how long it takes to get a pulse back, so your exploit got a lot more technically complicated since of course these pulses travel at the speed of light.
>substitute its electrical output as the input to the signing hardware
The LiDAR scanner is part of an integrated circuit (IC) that encrypt the data from the LiDAR scanner. To get at the unencrypted data, you would have to uncap the IC and use a scanning electron microscope or such.
How do I know so much about iPhone hardware? I don't, except I know that Apple is widely believed to be the world's leader in hardware security, so I strongly suspect that every single data path in a recent iPhone is encrypted before it leaves any IC.
>Until one of the people with the capacity to do it sets up a website where anyone can submit an image and have it signed.
The web server behind the site would have to be connected to a compromized iPhone Pro. As soon as Apple becomes aware of the web site, they will disable that iPhone Pro. Specifically, they will be able to determine its ID number (term?) from the attestation, and I'm pretty sure they already have the ability to disable an iPhone by ID number.
>Moreover, isn't "most people can't do this but some people still can" actually worse?
If Apple cares and is willing to expend the necessary engineering resources, then anyone (other the Apple itself) who makes any sort of notable or economically important or culturally important use of their ability to create a false attestation will retain the ability for only a brief time.
The last time a public jailbreak was released for modern iPhone hardware running the actively signed, latest iOS version was in May 2020, which is over six years ago. It is possible that someone will publish a jailbreak in the future, but the lifespan of that jailbreak will probably be only a few days. I expect Apple could exert a level of control over "camera remote attestation" similar to the level of control it has already achieved over which OSes (and which apps) can run on its iPhones. In general, these "technical regimes" are designed to make it easy for the engineering organization to recover from exploits as soon as the organization becomes aware of the exploit.
Again: do you really want photographic evidence to stop being useful in almost every situation (e.g., in court)? If not, then what is your alternative to "technical regimes" reliant on remote attestation similar to the regime I just described?
We know it is practical to have a LiDAR scanner in the same assembly as an image sensor because the rear camera bump of the iPhone Pro has a LiDAR scanner.
The technology need not be 100% tamper-proof to have a large effect on society: there is a huge difference in persuasiveness between the claim that anyone could have created a particular video by submitting to an AI some starting videos and images and some prompts and the claim that anyone with years of training and experience in cutting-edge microelectronics could have bought 500 iPhones and used very expensive equipment to create 499 ruined iPhones and one iPhone that can be used to create false attestations of recordings -- particularly because in a high profile instance such as a repeat of the Rodney King beating, Apple engineering would tend to be very interested in examining the device used to make the recording.
For example, would you really want to live in a world in which photographs are no longer considered evidence of anything because any photo might be AI generated? When a citizen standing on his apartment's balcony used his camcorder to record police beating Rodney King in 1991, it started a national movement against police brutality. So, you're OK with a world where there can be no national conversation sparked by any recording because as far as anyone knows, the recording could've been faked by AI? Remote attestation by the camera is the only way I have been able to think of to avoid that world.
For another example, banking and finance started relying on attestation in 1997 with the availability of the IBM 4758 PCI Cryptographic Coprocessor and have come to rely heavily on it.
(I don't have an opinion on the overall point of "tipping culture".)
Also, some people use an iPad Pro connected to a USB hub connected to a monitor, keyboard, etc, as their daily driver.
Also, doesn't MacOS sandbox most apps?