It doesn't seem possible to actually release frontier models quickly any more. Given their cyber capabilities, playing nice with the powers that be is probably required these days.
5,416 karma · joined January 21, 2012
meet.hn/city/us-New-York
Socials: - x.com/pradnelluru
---
If your employer lets you chill, their competitors will not, and win. That's how it goes.
Prompting and reading detailed AI output all day long, trying to tease apart the goods and bads - it's extremely exhausting. You don't get the mental breaks of pressing the compile or test button, or doing small refactors like pulling a logic into its own function. Coding is gone as a pleasure at work.
Still, for a large number of programs - like data-oriented applications - it’s probably a good fit. It might work ok with entity-component game systems as long as you reuse old memory…
What actually limits someone from doing this?
1) You need the hardware to play the games, still. You still need a good enough PC or a console like a PS5 to play all these emulated/cross-compiled games. Though, the cost is moderate these days: a used PS5 goes for $400-500?
2) You need to want to jailbreak your console, giving up "mainstream" features (multiplayer, warranty). This requires some interest and know-how, too. Though the jailbreak steps are easy these days.
3) If you lack a console, your PC needs to be strong enough to play WASM or emulated/cross-compiled games. I suppose for consoles like the Switch, that aren't super powerful, most PCs made the past few years could play a AAA Zelda game in the browser. But playing on PC is often not as fun as on a console, with the couch and controller and all.
I have a hard time people en masse want to give up multiplayer, by jailbreaking their consoles.
But single-player games are probably going to have a harder time.
The video game world is already super top heavy, with mass-market multi-player games (like Valorant), not to mention casino-like gacha games. So the trends may further favor this current track.
What this means is that the only hope for secure applications is security by construction, with formal methods. The race is on for constructing formally verified primitives with more and more expansive guarantees. The Anthropic/Theorem proofs show lack of memory sanitization issues. But that's just one class of problem.
Imagine a chip, a minimal boot loader, a minimal Linux, and an application library stack - all verified with high level guarantees. Then, you can plug in your application development to give you end-to-end verification guarantees. ie: no way to leak credentials at any level from the chip up to the application. No sense in your foundation being so strong if your application has easy RCEs. You need a full stack approach.
I know I'm describing a far off dream world, but it may be what's required to quell the economic "bomb" of so much dev time "wasted" in patching vulns, against actual feature dev / business value generation.
1) It's easy to generate very long docs that explore every conditional, etc. It is probably better overall, because more edge cases are being caught. But, the docs tend to be very long, and often at a lower abstraction level than appropriate for a design doc. At the limit, why not just put all the code in the design doc, too? It's rarely necessary. Though, when the thing we're focusing on is an API, it is appropriate to put the API service proto diff in the design doc...
2) They're harder to read, often for easily discernable reasons. They love doing the "bullet colon sentence" pattern, which often just means you're reading a concept twice. Sometimes, it helps. But I can't help but distrust that pattern more just because its so obviously AI generated. A more pernicious problem is the AI using phrases that are technically correct, but not part of our verbal culture. An example is using "observability workloads" instead of "probers" or "CL submission strata" instead of "CL submission order". Perhaps a Skill and an obedient AI can make these particular problems go away. Perhaps there needs to be a "preferred vocabulary" dictionary distilled from all the teams' docs...
If I generate code or text with AI, I have been adding a bit of hand-written text that tells people what exactly I verified. Like "I've read all the words in this comment", or "I cross checked this data with other similar metrics at X". The trust you get from human validation is the scarce thing now, not the output itself.
I was embarrassed to blurt out factoids based on quick AI-based analysis that turned out to be incomplete at later, deeper inspection. So I've stopped doing that. :)
You need an agent swarm to defend cyber-assets. No way we can respond at human speed.
Another consequence is that every public API with a bug bounty will be pen-tested up til the cost (in expectation) of running the AIs. We already see public AI-discovered vulns being reported to companies.
We've stumbled into general differentiable models..
The reason why things get worse is because of systemic problems: misaligned incentives, information asymmetry, diffuse responsibility, etc. It's actually hard to improve these aspects of large systems. Hence, why the world is an imperfect place.
If they weren't turned off at the billing cap, but were given some leeway instead, either that becomes the new hard limit, or GCP will have to give away the difference.
And there's no "middle ground" you could implement that makes sense either - like a "frozen" state. Preventing new writes to a GCS bucket breaks the writer app. Freezing VMs serving web traffic takes the site down.
Even if the service was shut down once the billing limit is hit, how long would GCP wait for the user to add funds or raise the limit? GCP would need to either keep the services in a hidden/frozen state or not turn them off / freeze them at all (in which case GCP would be giving away resources for free).
Maybe GCP can give users a heads-up when they're about to hit the limit? GCP already does - billing alerts do exist. It's just possible to blow past them if your usage is a massive spike.
Moreover, getting the hundreds of GCP services to implement a "frozen" state is difficult. It's hard enough getting everyone to listen to the "billing account disabled" signal, and (soft-)delete the resources (based on the resource, after some time interval). Given these billing overruns happen for smaller customers, it's not really worth solving the problem - which I don't think has a great solution to begin with.
1) Pulls you in with a catchy title, that at first glance seems like a dunk on Cal.com (whatever that is).
2) Takes the "we understand your pain" approach to empathize w/ Cal.com, so you feel like you're on the good vibes side.
3) Provides a genuine response to the actual problem Cal.com is dealing with. Something you can't dismiss out of hand.
4) But in the end of the day, the response aligns perfectly with the product they're promoting (a click away to the homepage!)
This mix of genuine ideas and marketing is quite potent. Not saying this is all bad or anything, just found it a bit funny. The mixed-up-ness is the point!
And the talk wasn't a random talk, but a memorial talk for Dijkstra: "The 2010 Edsger W. Dijkstra Memorial Lecture". I forgot this aspect as well!
a) finding a new theoretical frame that simplifies the space or solutions, helps people think through it in a principled way
b) finding ways to use existing abstractions, that others may not have been able to find
c) using non-technical levers, like working at the org/business/UX level to cut scope, simplify requirements,
The way you can make complexity work for your career, is to make it clear why the problem is complex, and then what you did to simplify the problem. If you just present a simple solution, no one will get it. It’s like “showing your work”.
In some orgs, this is hopeless, as they really do reward complexity of implementation.
What does it mean to say "we were promised flying cars", or "every city would have micro-factories, that 3D printing would decentralize production"?
The people creating these narratives may a) truly believe it and tried to make it a reality, but failed b) never believed it at all, but failed anyway, c) or be somewhere else on this quadrant of belief vs actuality.
Why not just treat it as, "a prediction that went wrong". I suppose it's because a narrative of promise feels like a promise, and people don't like being lied to.
It's a strange narrative maneuver we keep doing with tech, which is more future-facing than most fields.
Actually, no you couldn't. The subtlety of the choice of colors, their shading, and their soft shaping, and the program of their creation over many years - you couldn't do that. They're lovely and sublime, and wonderful and an abyss. If you want to throw all that away and reduce it two boxes of paint, go ahead - but you'll be wasting a lifetime's engagement, of the joy of seeing with your intellect wide open.
It bothers me, too. But, look at the history of the internet. There's no reason to expect we'll be able to fix this problem.
1. Search engines drove traffic to news/content sites, which monetized via ads. Humans barely tolerate these ad filled websites. And yet, local news went into steep decline, and the big national players got an ever-larger share of attention. The large, national sites were able to keep a subscriber-based paywall model. These were largely legacy media sites (ie: NYT).
2. News sites lost the local classifieds market, as the cost of advertising online went to zero (ie: Craigslist). This dynamic was a form of creative destruction - a better solution ate the business of an older solution.
3. Blog monetization was always tough, beyond ads. Unless you were a big blog, you couldn't make a living. What about getting a small amount of money per view from random visitors? The internet never developed a micro-payment or subscription model for the set of open sites - the blogosphere, etc. The best we got were closed platforms like Substack and Medium, which could control access via paywalls.
All this led to the internet being largely funded through the "attention economy": ads mostly, paywalls & subscriptions some.
The attention economy can't sustain itself when there are fewer eyeballs:
1. Tailwind docs have to be added just once to the training set for the AI to be proficient in that framework forever. So one HTTP request, more or less, to get the docs and docs are no longer required.
2. Tailwind does change, so an AI will want to access the docs for the version its working with. This will require access at inference time. This is more analogous to visiting a site.