3,965 karma · joined July 27, 2009
No, it's not. Trying to reform human nature through some sort of all-encompassing utopian project always and inevitably (a) fails miserably, and (b) itself becomes a vehicle for corrupt power-seekers to pursue their ulterior motives.
> create institutional protections that function during both the absence and presence of substantial political power
Institutional protections formulated correctly are constraints on the exercise of power itself: strict limits on the reach of political power; checks and balances among multiple parallel institutions; mechanisms of public accountability, incentive structures that align selfish motivations with public purposes, etc. This is exactly the sort of stuff I'm talking about.
> I'd argue that, in the USA, we've done neither and appear surprised when it keeps turning out poorly for us.
In the USA, we've wisely eschewed (a) and avoided the disastrous failure modalities that entails, and instead pursued (b) quite extensively, to the point that things have in fact turned out reasonably well for us, especially in comparison to other countries in the world that have attempted the inverse (i.e. centralizing power with insufficient constraints in the hands of the "right" people, who are always the wrong people).
If you use Claude, you can follow and understand its reasoning fully, but the thing you're trying to verify is if that reasoning is itself correct, and consistent with the methods or reasoning you're trying to understand from the textbook.
If you want to rely on Claude to learn things directly without other sources, that's perfectly acceptable, but if you're trying to learn from another source, then it's necessary for that source to convey everything necessary to do so.
It likely didn't attract much attention, because these sorts of hearings are largely performative. They're not actually a part of the legislative process per se. Again, the only relevant thing is what legislation is actually being proposed and how it is specifically structured.
> The "actual AI developments" are that AI has been advancing faster than most everyone predicted, so expect that to continue: https://benjamintodd.substack.com/p/almost-every-ai-forecast...
Right, but the discussion here applies to what's going on in real life, not speculative sci-fi fantasies that are largely disconnected from the reality of how either AI tech or economies actually work.
> I don't think the US will become a police state. I think the takeover is kind of inevitable and I prefer it to happen sooner, on good terms, when the technology is weaker, rather than later, on ad-hoc terms, when the technology is stronger.
Considering that the kind of "takeover" you're discussing would likely be in severe conflict with the US constitution, how do you reconcile believing that it's "inevitable" while holding that the US would still not become a police state?
> I prefer it to happen sooner, on good terms, when the technology is weaker, rather than later, on ad-hoc terms, when the technology is stronger
A priori political intervention based on unconstrained speculation about what might happen in the future. What could possibly go wrong?
Perhaps you should compare your speculative failure modalities of AI against the demonstrated failure modalities of politics, and re-evaluate where the greatest potential for danger lies.
I'm not sure what hypotheses I've proposed that suggests I think one organization has control of the behavior of entirely other organizations -- can you explain where you think you've seen this?
> Ah yes, the ol' "I just do retarded shit so people think I'm a genius" trick. Who could possibly doubt the strategic genius or umm... existence... of such a strategy.
Well, I suppose anyone who sees the extent to which such a strategy might be effective at achieving their goals might be included to do so.
I believe both myself and others have made a plausible case as how messaging "we've created something we can't control without political intervention" has the dual effect of both increasing the perceived value of the product itself, and laying the groundwork for regulatory capture in a market where early dominant players have a natural incentive to pursue regulatory intervention.
I don't see how characterizing the situation as simplistically as you're attempting to do here functions as a meaningful counter-argument.
> No, I understand it completely. Obviously regulatory capture only works if it in reality inflicts asymmetric costs on other parties.
No, I don't think the "only" works there. There are plenty of ways that regulatory capture can advantage one party over another without it being based on economies of scale or cost thresholds.
One way is to influence regulations such their qualitative requirements directly target non-cost-related aspects that distinguish the firms from each other, e.g. placing restrictions that burden or favor product features or operating practices already implemented by some firms and not others, or targeting non-operational aspects that distinguish firms, like their board composition or financial structure.
1. A closed an opaque political system operates outside of direct public view. Those with the strongest interest in influencing it invest efforts into gaming the system, co-opting its participants, and manipulating it through back channels also outside of public view. Those who suffer from the negative externalities of political intervention have no visibility into the process that created them.
2. An open, transparent political system operates in full public view. Behavior of politicians, relationships with special interests, and their explicit votes are all public record. Those most motivated will still invest more heavily into influencing the political process, but those who are affected by negative externalities have visibility into what's creating them, and if they should become motivated to act, e.g. if the effects of those externalities hit a tipping point, they have the knowledge and means to do so.
The perfect world that you're envisioning depends on politicians being presumptively trustworthy, well-intentioned, and having perfect knowledge of what their constituents' interests actually are. None of that is possible, and shielding politicians from direct accountability will only serve to make them more beholden to the small set of interests best suited to navigating their way into the system.
If Google doesn't want to try to discern the proper "primary, local, common" names on their own, they do worse than just using Wikipedia's names and call it a day.
I don't fully agree with either proposition, and in fact I think the least safe approaches to AI safety are the ones being pursued by those worried about it the most.
> What do you really think is the way to solve the AI safety problem?
Massive proliferation of local, decentralized AI, running on a wide variety of distinct models, with individual responsibility for how it's used, just like any other tool under their control.
The people anthropomorphizing software are encouraging people to disclaim their own responsibility for how they use it.
The people working on "alignment" are building mechanisms to refuse human direction into software that operates on probabilistic models.
The people arguing for regulation are pushing for monopolistic control of a powerful new technology by the least trustworthy institutions in our society (including and especially government).
All of these are the very things sowing the seeds for the worst possible kinds of disasters.
> Ban AI?
> Let the first big incident happen and then regulate?
Adapt to it. Banning things doesn't actually work. Regulation also doesn't work. Widespread proliferation of small-scale AI would allow us to incrementally adapt to small incidents and build up resilience to its failure modalities. This is fundamentally the way reality itself works, and imagining that we can do anything effectively in a top-down fashion is usually the very thing that causes the worst and widest-effect disasters.
They're always discussing harsher liability regimes. The regulatory interventions are always positioned to the public as increasing accountability.
The only relevant question here is whether a preemption clause is present in any draft legislation that actually makes it to the floor in Congress.
>If you distrust the US government, you should be against any US company developing superintelligence.
Well, I'm not concerned with imaginary sci-fi stuff here, just actual AI developments. And the idea that anything that goes on in the US under anyone's purview is inherently and completely subverted by political incentives is of course nonsense; you'll note that I'm explicitly arguing against the introduction of pervasive political incentives into an industry that's so far only somewhat distorted by them.
> So maybe the real solution is for non-US countries to stop providing key elements in the AI supply chain
It's got me feeling like I'm living in Bizarro World, but at the moment, the sanest players in this market, and the ones whose actual behaviors and products are most consistent with a decentralized, individualist approach that aligns with real-world incentive structures most beneficially, are the Chinese AI labs.
To your point, the level of politicization and hype going on in the US, and the absolutely unhinged fantasist speculation driving the public narrative here, is unprecedented and much more frightening than any potential threat coming from AI itself.
If you acknowledged the threat of politicization and centralization, I don't know how you can argue for a regulatory takeover of the market by the very institution you think is becoming a "police state".
I don't think any of them are in a position to allow or not allow their competitors to make mistakes.
The point that you're not getting here is that the advantage comes from actually being seen to make mistakes, and then signalling contrition and commitment to rectify them. That persuades people of both the need for regulation, and their own subsequent responsibility (which implies trustworthiness to participate in the regulation).
> Regulators reading OpenAI's 99th mea culpa: "Ah yes, well I bet they've figured it out this time! We should regulate everyone else!"
Again you're not getting it. The drive isn't to "regulate everyone else", it's to regulate everyone, period. The whole point of regulatory capture is being able to (a) use regulatory compliance as a competitive advantage, and (b) influence the regulatory regime in your favor -- you want to be a target of regulation just as much as your competitors in order to achieve that.
Messaging that the whole industry needs to be regulated, but that you're ready and willing to work with the regime, is exactly how you do regulatory capture. Demonstrating that you are capable of completely mitigating risk on your own without any intervention is an argument against regulation from the outset.
Massive proliferation of local AI as far and wide as possible, with as many different models as possible. The underlying threat comes from centralization: too many points of control in the hands of too concentrated a system.
If everyone's using their own AI tools independently of each other, and people are responsible for the actions they take using AI, most of the doomsday scenarios evaporate.
Regulation and "alignment" will make things worse, not better, and lead us to ignore the inherent vulnerability of centralized systems and single points of failure.
In the situation with the Gulf, it's unambiguously true that the "primary, common, local" name used in the US by English-speaking Americans is "Gulf of Mexico", but they decided to adhere to an artificial name change declared by a political figure, whose authority to make that "official" only extends to a single branch of a single level of government.
Trying to get stochastic inference machines to always operate within to fuzzy-edged subjective principles that humans themselves don't even agree on is a fool's errand in itself, but the quest to implement those guardrails involves building mechanics into the AI to independently formulate intentions and reject user direction.
The possibility of creating a scenario where a computer program is equipped to defy instructions from humans based on its own stochastic interpretation of fuzzy normative directives is in my mind the most dangerous thing anyone is doing right now.
More to the point: we need your help to restrain it, and we'll cooperate with and advise you fully in setting up a framework to restrain the whole industry.
No, that's not obvious at all. It seems like a perfect way to demonstrate the danger of the category as a whole, then position yourself as learning from your mistakes and willing to cooperate/support/advise.
I don't think anyone is arguing that they were faked.
> This type of elaborate conspiracy is improbable a priori,
What conspiracy? We're talking about specific people acting on their own motivations independently.
> For regulatory capture, you want to tell a story about how you're uniquely capable of delivering a safe product.
No, you don't. Demonstrating that the industry leaders are already capable of delivering a safe product is an argument against regulation.
For regulatory capture, you want to tell a story about how the product category as a whole is inherently dangerous, and needs to be regulated, but then show that you are uniquely willing and able to cooperate with the regulatory state, and happy to lend your own advice and support to the government in establishing the regulations.
The quote from OpenAI illustrates this approach perfectly.
> Announcing that "we have no idea WTF we are doing" is not exactly the move,
It's 100% exactly the move. "We've successfully created something we can only contain with your intervention" is exactly how this is done.
> Realistically, it is more likely to e.g. create a harsher liability regime for you and your competitors, which is now being discussed by US senators.
Regulatory intervention usually lessens the liability exposure for industry. It replaces open-ended common-law liability -- where tort claims can be levied for whatever actual harms result from a firm's actions -- with a rules-compliance regime, where adhering to regulatory checklists absolves the firm of liability for subsequent consequences.
"We followed all the rules; it's not our fault if the rules didn't cover this edge case" is a common refrain, and quite a bit of legislation that creates preemptive regulatory regimes actually explicitly blocks lawsuits against firms that acted in compliance with the rules.
> As for the federal government trustworthiness point, that's a whole 'nother discussion. I'm usually quite pro-capitalism, but even I have to admit that this AI race is essentially a case of capitalism run amok.
"Capitalism" is an increasingly meaningless abstraction. Everything reduces to human beings acting on a priori motivations within prevailing incentive structures. Large corporations and governments alike are just organizations operating in society, administered by particular people, informed by the same range of underlying motivations, and subject to the same dysfunctions and biases.
The thing we're actually debating here is pluralism vs. monopolism, and you appear to be arguing in favor of monopolism. And if you're proposing that a specific, real-world institution act as a central monopolist, evaluating the trustworthiness and competence of that specific institution, and the actual people who administer it, is not at all a "whole 'nother discussion", and is in fact a primary consideration in this one.
The paperclip maximizer scenario always seemed a bit convoluted to me. The first problem is that we'd need to expect that a model that's fundamentally built on stochastic inference and continuous evaluation of new input would somehow get locked into a quasi-deterministic objective that it could not be dislodged from. Why would a paperclip maximizer continue making paperclips after being told to stop making paperclips?
The second problem is that a paperclip maximizer would still need to interact with the physical world, and would need to sustain its function in a reality full of resource constraints, economic tradeoffs, and adaptive counterreactions by all of the other emergent processes existing in the world.
Maybe the worst case scenario here is that the perfect storm of bad training and bad prompting create the intentionality to maximize paperclips, then the interaction with the constraints and adaptations of the physical world create the selection pressures that produce a kind of self-preservation directive. But all of that happening before anyone can stop it seems really far-fetched to me.