bubblelicious

I think you’re welcome to that opinion and are far from alone but (1) I am very happy to pay for Claude, even $200/mo is worth it and (2) idk if people just sort of lose track or what of how far things have come in the span of literally a single year, with the knowledge that training infra is growing insanely and people are solving on fundamental problem after another.

I disagree, you have to take yourself back to when electricity was not widely available. How much labor did electricity eliminate? A LOT I imagine.

AI will make a lot of things obsolete but I think that is just the inherent nature of such a disruptive technology.

It makes labor cost way lower for many things. But how the economy reorganizes itself around it seems unclear but I don’t really share this fear of the world imploding. How could cheap labor be bad?

Robotics for physical labor lag way behind e.g. coding but only because we haven’t mastered how to figure out the data flywheel and/or transfer knowledge sufficiently and efficiently (though people are trying).

Epoch has a pretty good analysis of bottlenecks here:

https://epoch.ai/blog/can-ai-scaling-continue-through-2030

There is plenty of data left, we don’t just train with crawled text data. Power constraints may turn out to be the real bottleneck but we’re like 4 orders of magnitude away

Show me a paper, this makes no sense of course scaling laws are scaling

Sounds like someone has no knowledge of the literature, synthetic data isn’t like asking ChatGPT to give you a bunch of fake internet data.

Right let’s not have done the Industrial Revolution or the Internet or electricity

This based on any non anecdotal evidence by chance?

And you don’t think that’s already happening? Also where is your evidence for this?

Why do people think this is any different than other major economic revolutions like electricity or the Industrial Revolution? Society is not going to collapse, things will just get weirder in both unbelievably positive ways and then also unbelievably negative ways, like the internet.

Where does this view come from? I’m not aware of any real evidence for this. Also consider our data center buildouts in 26 and 27 will be absolutely extraordinary, and scaling is only at the beginning. You have a growing flywheel and plenty of synthetic data to break the data wall

Are you talking about just preferences or A/B tests on like retention and engagement? The latter I think is pretty reliable and powerful though I have never personally done them. Preferences are just as big a mess: WHO the annotators are matters, and if you are using preferences as a proxy for like correctness, you’re not really measuring correctness you’re measuring e.g. persuasion. A lot of construct validity challenges (which themselves are hard to even measure in domain).

I work on LLM benchmarks and human evals for a living in a research lab (as opposed to product). I can say: it’s pretty much the Wild West and a total disaster. No one really has a good solution, and researchers are also in a huge rush and don’t want to end up making their whole job benchmarking. Even if you could, and even if you have the right background you can do benchmarks full time and they still would be a mess.

Product testing (with traditional A/B tests) are kind of the best bet since you can measure what you care about _directly_ and at scale.

I would say there is of course “benchmarketing” but generally people do sincerely want to make good benchmarks it’s just hard or impossible. For many of these problems we’re hitting capabilities where we don’t even have a decent paradigm to use,

I’m not really sure how you envision AI use at your job but AI can be the extremely imperfect tool it is now and also be extremely useful. What part of AI use to you feels like a slot machine?

It just totally is different from my own personal experience which leads me to believe people just are lamenting poor usage of AI tools which is very understandable.

But nuanced and effective AI use, even today with current models, is incredible for productivity in my experience

I think that’s perfectly fair.

However, if you were leadership in this scenario, and you see people using various AI tools are systematically more productive then the people that aren’t, what would you do?

What do you mean?

Can you be more specific? What are the trivial cases you’re talking about? AI just doesn’t work? Coding agents are not saving anyone any time?

What’s the controversy, unless people are straw manning or pulling from some bad personal experience?

If you are not leveraging the best existing tools for your job (and understanding their limitations) then your output will be lower than it should be and company leadership should care about that.

Claude reduces my delivery time at my job like 50%, not to mention things that get done that would never have been attempted before. LLMs do an excellent job seeding literature reviews and summarizing papers. Would be a pretty bad move for someone in my position to not use AI, and would be pretty unreasonable of leadership not to recognize this.

Great take! Certainly resonates with me a lot

- this is war path funding

- this is geopolitics; and it’s arguably a rational and responsible play

- we should expect to see more nationalization

- whatever is at the other end of this seems like it will be extreme

And, the only way out is through

It’s more like people pretend it’s “obvious” that the market is going to do something or that AI developments are not going to support the spend, it’s just not obvious to me. I think people overweight probability on a bubble. Saying it’s “clearly a bubble” is hubris.

> asking where today’s AI bubble – because that’s what it clearly is – fits in a 1990s timeline

Yet evidence is:

- CAPE is high - NVIDIA has a very large market cap - enormous capital investment in AI and relatively few companies

I assume also a chorus of “AI actually doesn’t make you more productive!” And “AI capex and opex vastly outweighs realized profits!”

Seems a little less than “clearly”.

All of these are very much RISK factors, yet you need to assume the market is being irrational and assume that AI is NOT going to have the impact the market thinks it will. Personally for me I don’t understand: pretty clear trend in capabilities without a clear and insurmountable roadblock, so I totally get the “I think it’s a bubble” argument, it’s just that I think people underestimate what’s to come

I agree — probably my own selective memory and straw-manning. It just feels in my mind like the “vibe” on HN (in terms of articles that reach the front page and top rated comments) is very anti-AI. But of course even if true it is a biased picture of HN readers.

Would be interesting to see some analysis from HN data to understand just how accurate my perception is; of course doesn’t clear up the bias issue.

Yea a lot of this I understand and appreciate!

- offline and even online benchmarks are terrible unless actually a standard product experiment (a/b test etc). Evaluation science is extremely flawed.

- skepticism is healthy!

- measure on delivered value vs promised value!

- there are hard problems! Possibly ones that require paradigm shifts that need time to develop!

But

- delivered value and developments alone are extraordinary. Problems originally thought unsolvable are now completely tractable or solved even if you rightfully don’t trust eval numbers like LLMArena, market copy, and offline evals.

- output quality is seeing diminishing returns? I cannot understand this argument at all. We have scaled the first good idea with great success. People really believe this is the end of the line? We’re out of great ideas? We’ve just scratched the surface.

- even with a “feels” approach, people are unimpressed?? It’s subjective, you are welcome to be unimpressed. But I just cannot understand or fathom how

Really hard to believe articles like this and even more hard to believe this is the hive mind of hacker news today.

Work for a major research lab. So much headroom, so much left on the table with every project, so many obvious directions to go to tackle major problems. These last 3 years have been chaotic sprints. Transfusion, better compressed latent representations, better curation signals, better synthetic data, more flywheel data, insane progress in these last 3 years that somehow just gets continually denigrated by this community.

There is hype and bullshit and stupid money and annoying influencers and hyperbolic executives, but “it’s a bubble” is absurd to me.

It would be colossally stupid for these companies to not pour the money they are pouring into infrastructure buildouts and R&D. They know it’s going to be a ton of waste, nobody in these articles are surprising anyone. These articles are just not very insightful. Only silver lining to reading the comments and these articles is the hope that all of you are investing optimally for your beliefs.