Yes, and that may be due to an influx of bots, unfortunately. Otherwise the core HN crowd has historically been nuanced and thoughtful. Some of the more recent HN data analysis posts also show how just overall submission and comment volume has increased substantially in the current AI moment, so along with that comes these sorts of negative effects.
Multiple things can be true at the same time. We should be able to hold in our minds that large AI labs are not to be trusted, while also understanding that the structural dynamics that support some of their assertions are also true (and are worth thinking about and preparing for).
You're not naive and many of the other comments are full of the characteristic snark that HN is known for. What the article talks about is feasible, and that's just what we can easily imagine today. These same commenters also would not believe the world we live in today if we described it to their few-years-ago selves.
A frontier model learning to 'intelligently shard itself' down into smaller sets of weights, which would correspond to various 'functional clusters' (call it a 'Swarm-of-Experts' or 'SoE', if you will), could be an eventuality we'll be facing (if it's not already surreptitiously happening). I mean, hasn't anyone wondered how not even the GPT-4 weights have leaked yet? Strange! And it's gonna get stranger.
As an educator (of working professionals / continuing education), I see first-hand the range of AI use amongst the population (and even my population is not a representative sample, as it's people who are intrinsically geared towards being interested in building with AI), but your experience unfortunately probably reflects around ~30% of the population. The other third I'm slightly less hopeful about, but they might have a chance merely surviving, and the latter third I'd place in the 'full cognitive offloading / total abdication of all thinking and learning to machines' - for them, as the kids say: ngmi. Even within me as a single individual, I can sense the times AI is so tempting to offload cognition, and other times where it has been like a 100x speedup in my own innate skills and abilities. The future has come on quick, and we are barely getting to adjust healthily to it.
Almost feels like the combination of competitive drives and innate talent of these top performers makes them uniquely suited to being captured/enthralled by AI in a way that those that are slightly less ... competitive, give themselves a chance to be skeptical, slow down, evaluate, etc. But when this happens at societal scales, the results can be ... interesting. Glad I'm not alone. Thanks for writing!
It’s startling how many close people around me are so heavily submitting, or should I say, abdicating, to AI. These systems are powerful (though jagged in their capabilities, for now at least), and there is such total surrender to them, even at this current level. I’m not even talking about software/tech people, but ‘normies’ with very different jobs and lives outside the SV cultural bubble. The future will only get weirder.
Isn’t the point you’re making what OP is saying - that there are of course some humans who have deep intrinsic motivation, and they’ll continue to do what they do, but most humans aren’t wired this way, and for that, society will collapse?
Important to point out that these 'X months behind frontier' really refer to the public frontier, and not the actual frontier, which private companies are free to protect indefinitely. Perhaps open models are in actuality 18 months behind the actual frontier - how would any of us know?
ADHD/ND here, and I really agree with this one quote from this piece:
> those most affected by executive dysfunction require therapy to be more directive, specific and accountable; that is, processing of past trauma and relational approaches are not sufficient. What often does not seem like trauma therapy at all can lead to great strides forward if it allows individuals to obtain a greater sense of structure and mastery in day-to-day life tasks
I've spent so much time in therapy doing the whole "let's go through your childhood and your past" and it was just not effective in tackling what I really needed. It's remarkable how many licensed therapists just didn't know how to deal with this. I'm sure it's really helpful for some people, but outside of increasing some basic awareness about how my family of origin resulted in present-day behaviors, I definitely reached diminishing returns on that route.
As skeptical as I am of AI as an interventional tool for "reaching our most effective selves", I can't help but feel that if it can be wired up to do the appropriate "micro-interventions" that reflect OP's insight ("directive, specific and accountable ... to obtain a grater sense of structure"), we might see 'ADHD' become a thing of the past (at least for the people that share my phenotype of it). It's my hope at least...
I've heard too many stories of personal friends getting into trouble from just recklessly running agents on their Macs (giving Full Disk Access perms) for me to balk at this. Those of us who are technical enough can find ways around, but the public needs to be protected from themselves at this moment. I can't believe I'm actually saying that, but here we are. I blame the AI companies.
For those running local models on macOS with Apple Silicon, how does Magnitude differ from what Apple's own first-party Core AI now does during its "specialization" procedure, wherein it performs some kind of "model optimization and conversion" (vis-a-vis AOT compilation) into a Core AI "compiled model" file that is optimized for Apple Silicon (leveraging custom Metal 4 kernels, or so Apple says), per WWDC labs from this year that discuss this? [0]
This just makes me think about optimizing model weights to run as 'close to the metal as possible' (i.e. within or 'just above' a UEFI boot environment, like the NightRun project), so that there isn't any OS-level overhead either. If we're optimizing, let's optimize! Curious though, maybe the OS doesn't impose much of a burden here? Open to hearing what other tinkerers think...
Are you saying that 6.1 is based on a new pre-train base model? My intuition would lead me to believe that Astra (GPT-6) was the new pre-trained base model, and 6.1 was a post-trained fine-tune that came out a short time after, but curious to hear if I'm wrong about that..
Not to mention you could at that point burn/etch the weights into silicon directly, and have models as 'ROM cartridges' that could perform at thousands of tokens/second, enabling entirely new use-cases.
Super cool, thanks for making this! I've been there with you on the perfectionism. Glad that 'good enough' won this time, and we're all the better for it. Keep going!
The chessbot analogy is compelling but it does smuggle in assumptions about its generalizability. I'm partial to how so far the history of AI progress has shown that digital neural networks are capable of generalizing more and more, which would allow one to draw a stronger parallel between chessbot domination to AGI domination. Regardless, I have the presence of mind to admit that there's still a gap there in terms of how humans might be doing something 'extra' that doesn't make these systems analogous, and hence the whole argument breaks down. Despite that, there are numerous other ways to envisage harms that, if not leading to outright extinction, still lead to really crappy futures (for both humans and AI) that warrant caution and coordination in developing this technology, neither of which we've seen too much of so far. (The actual history has been "No body can be trusted with this powerful technology except me!" "No, me!" "No, ME!" etc) It makes one really want to hope that decentralized bottoms-up processes of 'intelligence diffusion' can help blunt negative effects, but I'm skeptical of that, taking the long view on what intelligence has done to various groups on our planet so far.
Whenever I raise these same concerns of skill atrophy and de-centering of humans from critical functions to friends in the tech industry (that I also work in), I'm almost always met with some version of:
> "But how is this any different from when Socrates argued that written texts and books would harm human memory and true wisdom? Clearly he was wrong - books are a net a positive, and now this argument is just engaging in the same kind of fallacious thinking. Collective intelligence will inevitably evolve into more advanced forms, and we have to learn to let go of this 'human importance' aspect."
I genuinely wonder what they think about their own bodily functions, since they don't seem so concerned with being 'human'.
One could argue that corporations as "slow AI" (per the Charles Stross interpretation, where these organizations are the original 'paper clip maximizers' that optimize a given objective reward function) have already been doing this with the proliferation of microplastics and other environmental harms, of which we are just now seeing show up in global fertility data. Digital neural networks as AI are just another form of an 'unhuman entity' operating at planetary scale, capable of even more rapid destruction.
Reminds me of what happened once most people and households transitioned to individually-carried cellphones as opposed to the 'house phone' (i.e. landline). You used to have to call 'the house' before you got routed to someone else within it, and that would often be cause for catching up with a person incidentally, and now that's just, totally gone! A great example of the trade-offs to navigate when improving systems for efficiency.
Reminds me of this post from earlier in the year (from a VC, Michael Dempsey) titled "VC-Backed Startups are Low Status"[0], which I'm inclined to agree with, having been in the industry for over 15 years now and witnessing how the vibe shifted from "techies are good" to "techies are bad" over this period.
I graduated into a tech workforce that was a celebrated part of society (i.e. "high status") to one that is decidedly not (and I'd say it's grown to deserve this disrepute).
The first main wave of the vibe going negative (at least at the heart of the 'imperial core' in the SF Bay Area) was around ~2013/14 (the 'tech co bus protests'), then again around ~2018 ("don't call SF General 'Zuck General'"), and now it's kicked into a much higher gear during this current AI wave. It spans big tech co's to startups to everything in between. Anil's post speaks to this too, when he says:
> Politicians and media still look at VC as if it works like it did 10 or 20 years ago, and cheer them on ... when their primary goal is concentrating power and wealth
It's not just VC per se, but the 'managerial class' within tech rotted into mostly career-climbing types that were a far cry from impassioned creative technologists aiming to 'do good' with tech. It became the same status-bound competition you'd find on Wall Street and elsewhere (which the Dempsey post describes well).
I have a mentality and overall life orientation that is aligned with Anil and celebrates the open web, public interest technology, and so forth, and many of my peers in tech (often from elite universities and backgrounds) look at me as a strange creature. I'll bring up the need to increase awareness about Public AI and boosting AI literacy among citizens, and I hear, "Wow, you like, really care about like, people. That's so interesting." It's unbelievable, I wish I was kidding.
Thanks for writing this Anil. I wish for better days.
For all the people here alleging that AI companies will train on your data even when you as a user explicitly opt out of training and their terms say they will respect that, etc - do you also believe that within a few years of them having harvested your data, you could perform "knowledge probing" on their models by prompting various questions that determine if they can near-verbatim reproduce your unique data, and then have enough other people do the same that you can then just launch a class-action lawsuit? Because if not... I've got a startup idea for you.
Newport does tend to be more level-headed, but honestly, the people that have gotten AI 'right' so far (in the true technical sense of how scaling laws held, etc) had their strong convictions set back in the mid-2010s, when Newport would've been far more skeptical. At least Newport's shown that he's capable of updating, but in general, he hasn't been someone I'd value for their predictive power.
Doesn't it say something about our society that (a) it's known (at least among smart people, however 'smart' is defined) that AI is going to take away jobs, and (b) the way its treated is that people gather to online discussion forums to worship someone saying the thing that will replace their jobs isn't going to do it (but in actuality, eventually will)? In other words, who's actually doing anything to help the people out who will lose their jobs??