It is a valid assumption but I would have expected the article to provide such details. And why would you not want to hire a socially awkward person, if I do that for my business I just want results and awkwardness doesn’t matter much in technical positions.
The article does not talk about numbers. It is hard to understand competition in the field he is applying to. Then having the opinions of his parents does not help, it’s odd I need his mom telling me the CV looks good. It would have been better to question Princeton here as it is in their interest to make sure alumni get the jobs. It would be useful to ask some recruiters about the possible red flags for this guy, just to be sure. In any case sure I understand and sympathise, anyone would be mentally broken after that many job applications.
You could ask the agent to attempt to recover using specific tools. But yes in some cases the bits could be lost without the ability to recover, there are ways to destroyy data as it is usually required for decommissioning
It all sounds good and he is on the right track but the elephant is not that. The problem is that ordinary businesses do not see ROI from dumping cash on this direction. If someone wanted to have similar access to models as internal folks in these companies, then you’d need a bit of cash to run such swarms of agents, not to mention people who would know how to do it. Another thing is that you’d need your own frontier models deployed in the place where the inputs would not be exposed to a third party, as it is important to feed the internal private info to reason about. And if you are like me then it is a constraint of capital and people who want to invest in you that is also a factor, why would they invest in you when openai/anthropic will be able to copy your ideas in a few months?
Just to sprinkle more details here about MSFT. The usage was trimmed down even more for some orgs and some specific accounts (I think those were vendors and the like). But if you are an FTE and somewhere in CoreAndAI then your limit is like 1M GitHub credits ($10k as article mentions) that could be spent on any model you have access to, e.g. Opus 5x or GPT-x. It is hard to pin down how many people use Claude vs how many just use internal harness over the Opus though. Importantly, Anthropic models are more expensive (not even talking about Fable here as it is a special case) as seen in usage metrics. Peeps can check their usage which has a good breakdown that shows how many tokens were burned on which model and how much that "costs" (the prices as you can imagine are not consumer ones), so IMO this change will drive more people to optimize based on their usage patterns if they are heavy hitters. The easiest path is to just switch to something like Sol/Astra and have similar perf for lower price.
From what I could see these numbers would usually be hit by people doing vast amounts of analysis over files, they would usually rock up tokens very fast. Then you combine this with the selection of Opus models which are much more expensive internally than GPT ones and the limit gets reached quite quickly.
I think it is important what they mentioned about using languages as well. Some smaller nations think they will culturally vanish because their “heritage” is close to nonexistent in the training data. If Mistral uses more of that language specific data it is a good thing and might help selling it to different non-english groups.
There was a section showing malware analysis which is great to have. I suspect everyone is using Chinese for this nowadays but now you can switch to this as well. Opus like GPT are restricted, it is hard to work with them doing offensive security and Fable is expensive.
Because we have access to these models and use them maybe? Because of benchmarks? It does not feel like there is much moat, look you can even use small models these days and do productive work. It is rather that private ones starting to get into diminishing return area which without moat is quite risky.
I did everything by hand for so long and miss it, now I just read thousands of lines day in day out. But there are interesting gains such as lowered barrier to quickly run adhoc perf tests, do some formal verification, you can just add more guardrails for the thing that gets shipped. The thing I hate though is when managers sell their generated code and you are left shoehorning it to production, they take away the interesting part of the job.
Misleading article, they are selling to corporations already who in turn have plans to create a few fabs in Europe. So it is maybe more about European companies who are not buying, but that rests on difficult capital availability and fragmented nature of the continent. Europe is not one country after all, someone wants a fab in Germany, but the why is it not in France or Sweeden.
> we found no immediate increase in key engineering metrics such as monthly pull requests and lines of code
> To establish a before-Copilot baseline, we used data from April, May, and June 2024. After-Copilot data was represented by the period of September, October, and November 2024
> GitHub Copilot usage varied significantly among engineers, the tool demonstrably fostered positive changes in perceived engineer value, reduced time spent on various engineering activities, and boosted motivation and perceived skills
> subsequent monitoring of PRs and LOC for participants from December 2024 to May 2025 showed no statistical improvements
> the implications of more advanced capabilities, such as retrieval-augmented generation (RAG) over enterprise codebases or deeper engineering workflow integrations, warrant separate investigation
I think it is just out of date, habits have changed as well. I have not seen much gain personally at that period except in the last 12 months. Also, models not named, token counts not shown. Not to mention it was the older autocomplete + chat that were in use, these days it is much more advanced with RAG, cli use, MCPs, etc.
Despite the article looks like it talks about Claude, in reality it describes a new type of a job - a synthesis of data science, research, comp science, plus industry specific knowledge. Another extremely important thing is to have access to all related research in some programmatic way, this is for exploration, I do not think many have such access. Finally, you need to be prepared to read all those generated results and judge them effectively to pick the strands worth pursuing further. I bet you could do it with any model and your own harness, even authors admit they use their own to manage multiple sessions which hints that claude is not enough.
Just for the context UBI kind of already exists in some countries but in a different form. If you are in Ireland and do not work you get the benefits provided you have somewhere to live and can pick them up in the post office (~200eur/week). The problem is that it solves only part of the issue because people have to live somewhere, and sure the gov will even contribute towards it (they do already) but somehow not enough is being built. The benefit is basically not enough to get the mortgage, nor is there housing available.
Not vr/ar but 3d visualisations and walkthroughs were a thing like 20 years ago. I did try to leverage it with the clients but what happened was that once people could see everything they had more opinions about how to change it. It helped me to get clients though. Sometimes it would go on for quite a ”few” iterations. I think it is more scalable not to do it, i.e. not to be on the cutting edge, unless the customer pays premium for it. I am pretty sure vr/ar has the same challenges where people are like “oh I thought the ceiling was higher, can we increase it by 2 inches and move the stair case a bit?” then you do it and something is wrong again.
It looks like the message here is “make sure to use $100k worth of Claude when doing any analysis or evaluation” and the given examples show that prior effort could have been improved or made faster. But to me 100k is an opportunity cost, and there is a possibility that these results are not reproducible, so spending it on some researcher or a grad student would buy you more in a long term. If it was 1k then sure it is worth throwing at a large problem space to find things, like using fuzzing.
Went through the comments here and there and one thing to note is that there was a question about who do you think should have won instead. This is a good question because it is possible that all submissions were like this or there were ones that looked just worse. It would be quite useful to know who came close as well in this case. If you knew which submissions were good you could have a process to revoke the prize and give it to someone else in case of fraud or negligence or similar.
Having said that it is also possible that the mistakes and claims were a human error, sure a lot gets ai generated these days but there is a chance in which case the accusation does not look so severe anymore.
It was like a bunch of mythos scans through and through which then generated the reports for everyone to implement, not sure if in all orgs though. Mythos was great as it came from the top, i.e. a clear incentive. I think bug reporting otherwise does not reach the engineers unless an incident is raised by the customer care against a responsible team.
It might make even more sense once we get to the point of a wider use of encoding the data into dna. For now we have these few commercial players in the field that cad do it (eg look up dna microfactory for storage archiving), IIRC genomika was saying they can do an MB for a 100-200eur.
I did masters in a similar way, just to get some credentials and fill in the gaps and learn something new. There is an idealistic part to it which is quite romantic as you spend the nights learning and doing the assignments. The structure of such online based learning systems is great for a determined person. However the “other” part of such courses are cheating and ai use. It is depressing to know that the specific credentials prove little because of it. So the only valid signal is: this person did not quit and they know how to write a report, use references. You’d need to test them to fully validate the credential.
The problem is that it is quite difficult to access the published papers is you are not in academia or some company that pays for the access, so AA sort of serves that niche to transfer the knowledge. Training on the other hand is a commercial activity to later rent the model, if this would be purely for open weights I suspect everyone was cool with it.
I do not trust either but you have to at least agree that having some sort of mutually recognised data privacy framework is a good idea because the courts can enforce it then. Saying everything must be from EU is also slightly silly and we should instead have something similar like certification (cyber act ?) to ensure enough competition exists to avoid service degradation. IMO cryptography could be the answer to many privacy related issues for the cross border transfers.
Also these decisions related where the data is stored and which service is used are under control of each commercial org buying them. The risks are assessed at the end of the day and in case of any issues the providers change. Why would a publicly funded org store citizen data in the US is a question regardless of privacy laws though.
Privacy laws are actually one of the very useful things that came out. It is difficult to do the same in the US because of the business lobby. It is crazy that US citizens data can be purchased in the “black” market and the used by the agencies. Leaving tech companies to self regulate is just not viable and it is proven time and time again they cannot do it.
China is building for sure, but Russia? The majority of russia does not have normal roads, full of crappy old trains, and infrastructure which was built in the soviet times. Russia is a joke of a country, oligarchy and kleptocracy rules.
This example is a bit over the top and is more of an edge case, subagents of the same session can use the same VM because what is the point to isolate among them? If at least one subagent is trying to hack you then I would consider the whole session was compromised anyway as you cannot guarantee the agents leaking this among themselves.