keeda

These economics would choke off the pipeline that turns junior engineers into senior engineers, which would still presumably needed to supervise the AI.

I don't see the irrationality here?

And yes, bottlenecks at MSFT (or any large corporation) are unrelated to kLOC/hour, which I would argue are primarily because of human factors... which also AI threatens to obviate: https://news.ycombinator.com/item?id=50121996

There is even some evidence that suggests senior roles are more in demand (and getting correspondingly higher pay too) which gels with the common observation that senior workers are more effective at managing the AI which is doing the work of (and hence displacing) junior workers.

Asking a chatbot or Google AI overviews should surface relevant studies.

There is of course the possibility that AI gets good enough that even senior engineers won’t be needed, but at this point it seems unlikely. My reasoning is that even expert humans make mistakes, which is why we need other humans to collaborate and check each others’ work (which, if you think about it, is a source of a lot of the processes and bureaucracy at companies; trivial example: code reviews.)

So even if AI gets to the level of human experts, this dynamic will not change... unless we hit ASI, where the AI far exceeds human capabilities. At which point we’ve hit the singularity and all bets are off.

Short of that, however, human experts will be needed, which is why programs like this are needed: to future proof against the short term economic pressures to minimize junior workers which will impact the long term pipeline that produces senior experts.

However, I deeply appreciate the rich world and captivating narrative it enabled, and I think this is what Frank Herbert was going for. He wanted the intricate dynamics of the Guild and Spice and Mentats and exciting close-quarter combat for a more intriguing narrative. But AI and robots would have obviated all of those, so he made it disappear with a handwave of "because Butlerian Jihad."

This inevitably results in suboptimal dynamics like bureaucracy, empire building, territorialism and other things that get lumped under “office politics,” which drains a lot of energy away from more productive efforts.

So you can see how AI solves that problem too ;-)

(Only half joking)

https://www.sciencebuddies.org/science-fair-projects/science...

Do other researchers or research institutions “pay mathematicians (other than those in their employ) to go over these papers” or do they simply publish their work for review as part of producing “reproducible results”?

AI may be taking over Mathematicians’ jobs, along with everyone else’s, and it’s OK to hate that, but the scientific method says nothing about that, or hiring experts, or ulterior motives, or “being filthy rich.”

Specifically, a decade+ of industry experience interviewing countless candidates (hundreds, personally) for FAANG-level salaries, regardless of visa status or any other criteria, and still struggling to hire talent, which put major crimps on our roadmaps. And hearing the same from peers and leaders in the industry, in private and in public as well as right here on HN!

We were not overlooking talent "because we can", we literally could not execute on a significant portion of our roadmaps due to lack of talent despite offering FAANG salaries.

I'm also all for holding corporations accountable, and if you see my past comments, I've been calling out all the anti-worker shenanigans that started since Elon fired 80% of Twitter (e.g. https://news.ycombinator.com/item?id=47175871)

But 1) as the data suggests, H1B displacing native jobs is not really a problem, and 2) as other comments here indicate, it's not Microsoft that needs to be held accountable because, at just below FAANG-level salaries, it is one of the highest paying companies in the industry.

Like 25% - 40% lower unemployment and 60% - 100% higher wages. Even during the dot-com bust software was much better off! It's right there in the BLS data (though a bit hairy to slice and dice due to how they categorize jobs.)

The "immigrants displace native worker" narrative is too simplistic. Consider maybe that without those workers the work simply does not get done.

I mean, we've all lived through this. How many people here, for most of their careers, did NOT have mountains of backlog they would like to clear out if only they had more headcount? How many have NOT spent days on quarterly planning because of the need to be extremely judicious about what you put on your roadmap? And how many have not bemoaned, oftern right here on HN, how hard it is to find people who can actually code?

It's not just you, this dynamic plays out all over the place, and puts real constraints on the expansion potential of companies, which reduces future hiring all around. There are studies on this. Yes, there is naturally some downward wage pressure on local employees due to a larger pool, but on the whole the expansion and faster growth of companies, and hence the overall economy, more than compensates for this.

Feel free to ask your favorite AI for more... You know, the thing actually taking our jerbs! ;-)

Unfortunately, the current world is not set up for that, and until we get there, yeah things are gonna get pretty hairy. But if and when we get to the other side, I think the future will be glorious.

Whether that produced anything worthwhile can be controversial, depending on the very subjective ways people quantify these things. All I'll say is that they have 18K and 27.7K stars on GitHub and apparently a comparable number of users.

But as sibling comment indicates, even that is no longer a guarantee. There are demonstrations and anecdotal accounts of reverse engineering entire features or even small applications, including the backend code, by pointing an LLM at the web UI: https://www.thoughtworks.com/insights/blog/generative-ai/bla...

This is why I'm thinking that patents and other legal means are likely the only way forward to protect software that does not have a natural moat, like network effects or some huge data advantage. That is actually a huge amount of software.

My concern is that without proper protection software without such moats, this will create incentives for people building them to adopt less-than-ideal means of monetizing their work, e.g. the things that lead to enshittification.

Also TFA is about a much narrower ruling than it first seems:

>The 3rd Circuit distinguished Thomson Reuters' case from other AI training cases. Unlike the technology in those cases, Ross' search engine did not feature generative AI — AI that creates new content — and the appeals court said in a footnote that concerns raised by the US Department of Justice in a copyright lawsuit against OpenAI "do not apply here."

Are patents the only real IP protection left for software?

And would a patent (assuming this application had something patent-worthy to claim) help at all here? As in, in addition to sending a C&D maybe also including language about patent infringement would be more effective?

I know that patents are unpopular for many here (including you, IIRC!) but I think this question is extremely important, especially to anyone who wants to make a living purely off the software alone.

Remember: if no other moats or business models or funding models lend themselves naturally to the software in question, anything bolted on is pretty much already on the slippery slope to enshittification. A mechanism that encouraged people to compensate fairly for the value provided by software would be better for the Internet than what we've got going on today.

This was a low-key hilarious replay of George Dantzig and his homework problems: https://en.wikipedia.org/wiki/George_Dantzig

The controversy was whether they had plagiarised that other work on the sub-problem, which they categorically denied after an investigation. And yes, a few days to investigate something like this is reasonable for a company as big as OpenAI. Having seen how data infra is set up when petabytes of data are flowing about, there are thousands of entwined data pipelines to figure out. Not quite as easy as running a query on a sqlite DB!

> Believing that OpenAI's claims have any substance cannot be explained by naivety alone.

Yes, they could be explained by a GitHub repo full of proofs :-)

Or are you suggesting there were hundreds of researchers who just happened to be close to solving hundreds of these long standing open problems using Codex, and OpenAI swooped in plagiarized them all? ;-)

But that was not the original argument, which was: this incredible and rapid advance and everything we've seen since that was broadly considered decades away before ChatGPT; so it is unwise to make flat assertions that this will not lead to AGI because we have no idea where this is leading, not to mention what AGI even is.

The response to that was "In 1800 calculators would have been considered intelligent." ¯\_(ツ)_/¯

> Sorry to hear it, but some people do.

Funny, I've never heard a cogent definition anywhere so far, least of all in this thread! To reiterate my point:

The state of AI/ML pre-ChatGPT: research and industry churning away experimenting with all kinds of tweaks and tunings with all kinds of models to make narrow gains in very narrow fields. NLP, NLU, computer vision, speech recognition, text-to-speech, planning, symbolic reasoning, analytics, translation, sentiment detection, etc. were all areas of heavy research . "Code generation" that we see today was a pipe dream until 2015 or so. Ordinary users experienced AI/ML almost exclusively as a narrow feature embedded in specific products, such as recommendation systems or spam filters or ads.

Post-ChatGPT: Any researcher or any user anywhere can prompt a model in natural language, or audio and video. with any arbitrary task, and get largely useful results.

The very concept of a single model being able to even attempt an unlimited variety of tasks, doing surprisingly well on many of them and continuously getting better at everything, coming from the previous deeply fragmented state of the art, I think reasonably qualifies as "general."

If there's another take on what AGI means given the background of the field, I'd love to read it!

I keep saying this, until ChatGPT came out 4 years ago it was basically unimaginable that a single model could do any of, let alone all, the things they are doing today. Like, seriously, go take a look at the state of the art in NLP and NLU, the very first challenge in getting computers to even “understand” natural language, let alone other things like reasoning. Everything it does automatically was once a heavily experimental deep research field with long glorious careers for the researchers.

And now it’s all gone because the Bitter Lesson won again. If that’s not general enough to qualify for the G in AGI I don’t know what it is. And we’re sitting here going, “But it sometimes writes bad code though.”

And which government is going to enforce these laws here to let their geopolitical adversaries have the upper hand?

And what about the models already out there that with minor tweaks are equally capable of substantial damage?

And what do these laws do to remove the trillion-dollar incentive to produce models that will replace all knowledge work? The one thing economics teaches us is that incentives are an inexorable force, like laws of nature, and this is the biggest incentive of all. Which is why I called this a "force of economics."

If suppressed here this effort will only migrate to other countries who are desperate to get in on the AI game and are even less aligned with the US than these CEOs.

This, like nukes, is going to take international collaboration on careful regulations, else it's not going to work at all.

There's also c) use your own agent to take the vulnerability and get back to you with an assessment including whether it managed to devise a working exploit, and you go from there.

Or do you have doubts about the ability of these things to devise working exploits? ;-)

My theory is that this is due to a kind of "tunnel vision" these models have as they execute on a given task, because engineers new to a company do the same thing until they learn the "lay of the land" and figure out that similar problems have been solved elsewhere.

In a past job my team owned the internal multi-repo codesearch tool, which was by far the most popular internal tool, and later another team added a similar semantic search capability. This was very exciting, but I left before I could see how well it worked out in real-life.

Like, you'd do a keyword search and explore if you need some major piece of functionality that would require significant work, or whenever you encounter an abstraction whose code does not exist in your repo and you want to learn more about it. But when you're in the flow and inventing smaller abstractions, like a class or utility method, you don't necessarily think to search for it. Worse, even if you did, you could not search for it effectively because something similar may exist with slightly different naming or terminology or a typo that a keyword search would miss. Predictably, at scale you ended up with a dozen different implementations doing the same thing.

Now however, you could automate this with agents. I suspect these days simply prompting an agent to look for any relevant code to reuse would actually work pretty well. But they would need to store the entire codebase in their context window (if it fits at all) to refer to it all the time, which would burn a ton of tokens AND reduce performance due to a heavily polluted context. Instead, something like this would be invaluable to provide as a tool / MCP to the agent so that it could locate relevant, reusable code during its planning phase. (Or maybe a post-codegen linter-like check, which IIRC some people have tried, but why fix when you could prevent?)

I'm not sure if the exact method in TFA would work best, though; maybe a pipeline that generates comments/docs for each unit of functionality and then semantically indexes those, rather than structure-aware chunks of code itself?

Similarly for Apple’s “red herring” paper, simply adding a generic caveat to “disregard irrelevant factors” (without specifying which ones) restored performance even in the weaker local llama models back then.

The flaw was not in the reasoning; the flaw seems to be simply that the assumptions we make are often different from the assumptions it makes. I wonder if that might be a fundamental underlying cause of misalignment.

What about the next incident? Or the ones done by Chinese models, because they sure as heck aren't slowing down? And the thousands of other incidents that will happen as we deploy these things everywhere?

Because this is not happening just now in labs, it's only where they are most visible; this has been happening in the wild from the beginning, starting with the earliest AI-assisted suicides. Which is a perfect example of the problem, because not these CEOs, literally nobody in the world asked for suicide ideation machines. Or the hacks, or any of this other stuff. Yet here we are.

We have to understand: it's not these CEOs that are driving this headlong mad dash towards more powerful models. It's a force of economics. There is just too much money to be made. If we dispose of these people, there will just be somebody else doing exactly the same thing because the incentives as they exist today all force that outcome. This is why they're asking for regulation, or "urging us to urge them to stop."

Holding CEOs accountable certainly would feel good and may even be justified, but it's like putting a band-aid on a cancer; it does nothing to change the underlying cause.