agentcoops

Not an answer to your original question, but I think you’d be surprised how much high quality historical linguistic content was hiding in the dusty old corners of the internet. I’ve been doing some work recently with LLMs on historical languages (various forms of Latin, Ancient Greek and medieval European languages) and the out-of-the-box performance of state of the art LLMs is shockingly good. It isn’t that surprising when you remember all these archive digitization projects that took place in the early 00s, but ended up either as stale links, preserved only by archive.org, or stored in arcane CRMs essentially unusable by humans. I assume the same is especially true for various historical Yiddish corpora.

I ran some tests and, without fine-tuning, GPT can translate medieval German, for example, considerably better than well-known scholars today.

GraphRAG does full entity extraction across the entire data set, then looks at every relation between those entities in the documents, then looks at every “community” of relations and generates narratives/descriptions for everything at all of those levels. That is… not linear scaling in relation to your data to say the least — and because questions will be answered on the basis of this preprocessing you don’t want to just use the stupidest/cheapest LLM available. It adds up pretty quickly — and most of the preprocessing turns out to be useless for questions you’ll ask. The OP’s approach is more expensive per query, but you’re more likely to get good results for that particular question.

Honestly I don’t know the best answer, but my sense is there’s something important in the direction the OP is going: I.e moving away from vector search or preprocessing towards dynamic exploration of the document space by an agent. Ultimately, if the content in one’s corpus develops in a linear manner (things build one after another), no vector search will ever work on its own, since you just get a however exhaustive list of every passage directly relevant to the question — but not how those relate to all the text before or after.

GraphRAG gets around this by preprocessing these “narrative” summaries of pretty much every combination of topics in a document: vector search then returns a combination of individual topic descriptions, relations between topic descriptions, raw excerpts from the data, and then such overarching “narratives.” This definitely works pretty well in general, but a lot of the narratives turn out to be pretty useless for the important questions and it’s expensive for preprocessing etc.

I think the area that hasn’t been explored enough is generating these narratives dynamically, ie more or less as the OP does having the agent simulate reading through every document with a question in mind and a log of possibly relevant issues. Obviously that’s expensive per query, but if you can get the right answer to an important question for less than the cost of a human’s time it’s worth it. GraphRAG preprocessing costs a lot (exponentially scales with data) and that cost doesn’t guarantee a good answer to any particular question.

I’ve been working on RAG systems a lot this year and I think one thing people miss is that often for internal RAG efficiency/latency is not the main concern. You want predictable, linear pricing of course, but sometimes you want to simply be able to get a predictably better response by throwing a bit more money/compute time at it.

It’s really hard to get to such a place with standard vector-based systems, even GraphRag. Because it relies on summaries of topic clusters that are pre-computed, if one of those summaries is inaccurate or none of the summaries deal with your exact question, that will never change during query processing. Moreover, GraphRag preprocessing is insanely expensive and precisely does not scale linearly with your dataset.

TLDR all the trade-offs in RAG system design are still being explored, but in practice I’ve found the main desired property to be “predictably better answer with predictably scaling cost” and I can see how similar concerns got OP to this design.

I guess I haven't been an OSS maintainer in too long and might be a bit naive, but I still feel like completely incompetent "vibe-coded PRs" can't be that common... I understand why unqualified people try to spin-up X vibe-coded app that turns out to be a complete nightmare, but not why they would flood github with random PRs: there just doesn't seem to be all that much or even any incentive to push out PRs for free, especially if you more-or-less need a pro Claude subscription.

Oh for sure, I'm not too stressed by it -- but I think the ship has sailed on the chance for mainstream Scala adoption. Perhaps it was always delusional, but there was a period when it really seemed like Scala had somewhat of a chance to be the Ruby replacement and become one of the main backend languages (after the Twitter rewrite to Scala, when Foursquare, Meetup and various other startups of that generation were all in on the language); then there was a generation where at least it was the defacto language for data infra. Now, I'm not even sure if many major companies are even using it for the latter case.

Mainstream adoption isn't everything and I still mostly use Scala for personal projects, but it's such a different world working in a language where the major open source projects have industry backing. The Scala community, meanwhile, seems mostly stuck starting entirely new FP frameworks every other week. Nothing against that, but I don't see that much advantage to choosing Scala over OCaml at this point (if you don't need JVM integration).

Momentum appears to be behind Rust now, of course, but I've yet to be convinced. If it had a better GPU story and could replace C/C++ entirely I'd be on board, but otherwise I want my everyday language to be a bit closer to Python/Ruby on the scale against C/C++.

I’m honestly still stunned at the self-implosion of the Scala community… can’t think of any other language that threw away such quite unexpected success at an industry level. Apart from the toxic community, not trying to challenge Python for supremacy in the scientific computing/data analysis space seems like the major mistake, given it was for a time the lingua franca of data infra (Spark, Scalding etc).

It's a really difficult problem. I read a comment on here the other day about the increased burden on project maintainers that I sympathized with, but I wonder if the solution isn't actually just more emphasis on reputation tools for individual committers. It seems like the metric shouldn't just be "uses AI assistance" vs "doesn't", which as you note just leads to people hiding their workflow, but something more tied to "average quality of PR." I worked in finance briefly and was always really intrigued by the way responsibility worked for the bankers themselves: they could use any tools they wanted to produce results, but it had to be transparent and if someone was wrong the pretty strict burden fell on that IC personally.

The worst case for AI and OSS is a flood of vibe-coded PRs that increase bugs/burden on project maintainers; the best case is that talented but time-starved engineers are more likely to send the occasional high-quality PR as the time investment per PR decreases.

As someone who has always despised front-end programming, I would be more than satisfied if LLMs turned out to only be particularly good at that. Given the number of pages we've all seen where the content as well as the front-end are obviously 90% AI-generated, I understand the knee-jerk reaction against sites clearly designed by AI, but it's unfortunate that leads to a negative reaction against (or even just a difficulty distinguishing) sites where people share a tool/knowledge of their own that they never would have shared if it would have taken even just a few days of futzing around with CSS/figuring out the latest JS framework...

I think people under-estimate how much of an effect the unreasonable complexity of modern front-end programming has had, especially outside of tech. My wife works in the non-profit world and I have been completely blown away by how much "designers" barely able to tweak a wordpress plugin get away with charging... She's an "AI skeptic" herself, but almost cried with joy when I showed her V0 [0].

[0] Not as good at coding as others, but probably the single most impressive AI product I've played with from a UX perspective.

More or less! Just like when writing a typical program or indeed mathematical proof, you have a lot of freedom in how you break up the problem into subproblems or theorems (and rely on “libraries” as results already proven). Traditionally the problem with these theorem-proving environments was that the bulk of even relatively basic theorems, let alone the state-of-the-art, had yet to be formalized, so you weren’t quite starting from the axioms of set theory every time but close enough. Lean has finally seen enough traction that formalizing highly non-trivial results starts to be possible.

I worked a long time ago on tools for algorithmically pruning proofs in another theorem-proving environment, Isabelle. That project was largely just using pretty straightforward graph algorithms, but I’m sure the state-of-the-art is much more advanced today (not least it’s an area where I assume LLMs would be helpful conjoined with formal methods).

I fell into a rabbit hole when I was looking for coverage of a recent relatively technical biological result and the most prominent Google results were from 'evolutionnews.org' aka 'scienceandculture.com', run by the 'Center for Science and Culture.' Imagine my surprise to find this is all run by a 'Discovery Institute' with tens of millions in funding dedicated to pushing 'intelligent design' and behind the campaign to have anti-evolution views taught in public high school science courses.

Not of course to say these outfits are behind the decline in science reporting, but it's a real tragedy how difficult it is to find actual and competent scientific journalism today, a tragedy that makes the job of such charlatans all the easier. I'm glad that Quanta Magazine seems to be doing well enough, which certainly isn't perfect but I've read some good articles from them.

I share your concern for the livelihood of authors (and your skepticism regarding the naiveté that often surrounds pro-piracy rhetoric), but I don't think that's fair to the question here. Unlike in the case of music or film, most users are not just trying to get the latest NY Times best-selling novel. The percent of books made accessible through these services that are tied to an author's income through consumer sales is negligible. Most specialist literature, whether in the natural sciences or the humanities, is priced under the assumption that university libraries are the ones making the purchase, often more or less automatically. Yet even and perhaps especially in the US (I know nothing of the library culture in certain sanctioned countries), it's increasingly rare that university libraries have open stacks for non-students and there are incredibly few public libraries that actually provide access to scholarly works, past or present -- New York Public Library and the Library of Congress in DC are the ones I've used personally, but I'm sure there are a handful of others.

Moreover, however many countless AI companies now buying and pulping copies of every book in existence seems to be really changing the used book market. Prices are going up dramatically and before this year it was very rare to not find a single copy in the world of whatever old book one desired.

As someone who spends a disproportionate amount on books and shares your concern for not making life even more difficult for authors, these services going away would be a tremendous regression.

You don't need full reasoning to get accurate results, so even with GPT5 it's still pretty cheap for a one-time job and easy to reason about costs. It's certainly cheaper if you have data where reliability is key and classical OCR will undoubtedly require some manual data cleaning...

I can recommend the Mistral OCR API [1] if you have large jobs and don't want to think about it too much.

[1] https://mistral.ai/solutions/document-ai

100%. My sense is that many in this thread have never gone through the misery of trying to use classical OCR for non-English documents or where you can't control scan quality. I did a test recently with 18th-century German documents, written in a well-known and standardized but archaic script. The accuracy of classical models specifically trained on this corpus was an order of magnitude lower than GPT5. I haven't experimented personally or professionally with smaller models, but your experience makes me hopeful that we might even get this accurate OCR on phones sooner rather than later...

I've worked extensively with Tesseract, ABBYY, etc in a personal and professional context. Of course they work well for English-language documents without any complexity of layout that are scanned without the slightest defect. At this point, based on extensive testing for work, state of the art LLMs simply have better accuracy -- and an order of magnitude so if you have non-English documents with complex layouts and less than ideal scans. I'll give you speed, but the accuracy is so much greater (and the need for human intervention so much less) that in my experience it's a worthwhile trade-off.

I'm not saying this applies to you, but my sense from this thread is that many are comparing the results of tossing an image into a free ChatGPT session with an "OCR this document" prompt to a competent Tesseract-based tool... LLMs certainly don't solve any and every problem, but this should be based on real experiments. In fact, OCR is probably the main area where I've found them to simply be the best solution for a professional system.

Have you evaluated this lately? Last year or even just earlier this year I would have mostly agreed with you. At this point, however, with at least the documents I have been working on, OCR reliability with GPT5 or Mistral OCR [1] has been much better than even domain-trained classical OCR. If the documents have even slightly complex layout (to say nothing of page numbers or page headings or an uncommon font), the accuracy of state of the art LLMs has been in my work an order of magnitude greater. The ability to have the LLM tentatively combine trailing sentences across pages, which is especially useful if you have to work with documents in German say, is invaluable.

[1] https://mistral.ai/news/mistral-ocr

I'm not sure your argument applies only to AI. Intelligence is certainly not knowing through, say, divine inspiration what another person wants you to do. This bottleneck of "describing the problem" is the same bottleneck faced when working with junior (or senior) engineers, especially in a team. One need only consider the classic of our field, Mythical Man-Month, which is really dedicated to this precise and, in some sense, irresolvable problem -- often it's best to just have one person who understands and ideally first posed the problem do the work, rather than introduce this bottleneck of communication.

It's a difficult and crucial problem, we all agree, but it's a stretch to define intelligence as such to be "describing the problem." Choosing the right problem in the first place (i.e. not just telling person B to do X but selecting the X that in fact is worth pursuing), perhaps, but I don't think that's right either as a definition of intelligence. Indeed, even the best scientists often speak of an "intuition" that drives their choice of problems.

More classical definitions place intelligence in the domain of "means-ends rationality", i.e. given an end to pursue being capable of determining the correct way to do so and carrying it out until completion. A calculator like a hammer is certainly not intelligent in that sense, but I would struggle to see how even an AI skeptic could maintain that state-of-the-art LLMs today are not a qualitative step above calculators according to this measure.

I'm biased, but very fond of the open-access introductory textbooks used where I studied. The department was very much pure maths, but the intro classes were accessible to general liberal arts students. I think the texts are relatively unique in that they're very proof oriented, yet with a pedagogical style that doesn't assume the reader is a future graduate student.

For calculus, three options to see if you like a particular author's style of explanation: http://people.reed.edu/~mayer/math111.html/math111.pdf - the most pedagogical http://people.reed.edu/~mayer/math111.html/math111.pdf - the most beautiful https://people.reed.edu/~jerry/111/calc.pdf - the most technical

For introduction to mathematical analysis and proof: http://people.reed.edu/~mayer/math112.html/math112.pdf

For multivariable calculus: https://www.stat.rice.edu/~dobelman/notes_papers/math/calcul... [1]

For linear algebra, we used the relatively standard Friedberg Insel and Spence [2], but I hear good things about https://hefferon.net/linearalgebra/.

[1] Link is off university domain, since apparently it was at some point turned into a bit more hardcore textbook oriented towards those going onto graduate studies in mathematics. If curious: https://www.amazon.com/Calculus-Analysis-Euclidean-Undergrad...

[2] https://www.amazon.com/Linear-Algebra-4th-Stephen-Friedberg/...

While I would agree with you generally, in this case we’re talking about decisions that are precisely not desired by either business or investors. Which is to say, I think the American people and institutions are in a much worse place than at the whims of business alone: caught between an evangelical cultural coup and unrestrained capitalism. It’s actually surprising to me that the evangelicals appear to be dominating the concerns of business and investment — I don’t have a great explanation of why it has turned out like this. It leads to such a manifestly contradictory situation in which the government is betting the future on US AI global dominance and yet gutting the institutions that would enable it.

I moved to the UK with my family just before the Brexit vote and left last year. I love the country, but the changes I saw over that time period were so stark -- and, similarly, so many of the friends I made in that time had already left the country.

That I could have multiple negative NHS experiences relating to missed cancer diagnoses of friends in that relatively short span of time is suggestive of a real problem. The institution seemed to have less of an issue with elder care (in the US, the phantom menace posed by Obamacare or any governmental involvement in healthcare was meant to be "death panels" deciding the fate of grandparents) than with avoiding at all costs detecting potential long-term problems in the young. It's a 'rational' fear in the sense, as you note, that such cases put tremendous pressure on services, but there's no world where the best health outcome is refusing to screen your working age population.

Apologies if this comes across as too abstract, but I think your comment raises really important questions.

(1) While studying the properties of the mathematical objects produced is important, I don't think we should understand the situation you describe as a problem to be solved. In old supervised machine learning methods, human beings were tasked with defining the rather crude 'features' of relevance in a data/object domain, so each dimension had some intuitive significance (often binary 'is tall', 'is blue' etc). The question now is really about learning the objective geometry of meaning, so the dimensions of the resultant vector don't exactly have to be 'meaningful' in the same way -- and, counter-intuitive as it may seem, this is progress. Now the question is of the necessary dimensionality of the mathematical space in which semantic relations can be preserved -- and meaning /is/ in some fundamental sense the resultant geometry.

(2) This is where the 'Platonic hypothesis' research [1] is so fascinating: empirically we have found that the learned structures from text and image converge. This isn't saying we don't need images and sensor robots, but it appears we get the best results when training across modalities (language and image, for example). This is really fascinating for how we understand language. While any particular text might get things wrong, the language that human beings have developed over however many thousands of years really does seem to do a good job of breaking out the relevant possible 'features' of experience. The convergence of models trained from language and image suggests a certain convergence between what is learnable from sensory experience of the world and the relations that human beings have slowly come to know through the relations between words.

[1] https://phillipi.github.io/prh/ and https://arxiv.org/pdf/2405.07987

I used to work quite extensively with Isabelle and as a developer on Sledgehammer [1]. There are well-known results, most obviously the halting problem, that mean fully-automated logical methods applied to a formalism with any expressive capability, i.e. that can be used to formalize non-trivial problems, simply can never fulfill the role you seem to be suggesting. The proofs that are actually generated in that way are, anyway, horrendous -- in fact, the problem I used to work on was using graph algorithms to try and simplify computer-generated proofs for human comprehension. That's the very reason that all the serious work has previously been on proof /assistants/ and formal validation.

LLMs, especially in /conjunction/ with Lean for formal validation, are really an exciting new frontier in mathematics and it's a mistake to see that as just "unreliable" versus "reliable" symbolic AI etc. The OP Terence Tao has been pushing the edge here since day one and providing, I think, the most unbiased perspective on where things stand today, strengths as much as limitations.

[1] https://isabelle.in.tum.de/website-Isabelle2009-1/sledgehamm...

For the record, I wasn't justifying the world that has created these incentives. In order to understand (maybe even someday change) the rottenness of the world, it's important to avoid the sort of reasoning that the OP critiques, which sees everything as just a sort of arbitrary maliciousness, rather than understanding the very concrete institutional mechanisms by which it has perversely become 'rational' to view everything outside a few profit centers as a 'liquid asset.' If a firm doesn't generate free-cash-flow, which is very very difficult outside the software industry, it will not receive significant capital investment and will be dependent on debt/profits. This impacts not just the life of employees, but limits what endeavors are funded at all, i.e. in part why it is that "software is eating the world." How to change those incentives is a much more difficult, but real problem.

I agree with your point, but I think it's worth noting that there's a real problem of language today both in popular and scientific communication. On the one hand, in popular understanding, there's the importance of clearly separating the era of "machine learning" as let's say Netflix recommendations from the qualitative leap of modern AI, most obviously LLMs. This article clearly draws on the latter association and really leads to confusion, most glaringly in the remark you note that the AI probably took up some forgotten Russian text etc.

However, scientifically, I think there's a real challenge to clearly delineate from the standpoint of 2025 what all should fall under the concept of AI -- we really lose something if "AI" comes to mean only LLMs. Everyone can agree that numeric methods in general should not be classed as AI, but it's also true that the scientific-intellectual lineage that leads to modern AI is for many decades indistinguishable from what would appear to be simply optimization problems or the history of statistics (see especially the early work of Paul Werbos where backpropagation is developed almost directly from Bellman's Dynamic Programming [1]). The classical definition would be that AI pursues goals under uncertainty with at least some learned or search‑based policy (paradigmatically but not exclusively gradient-descent of loss function), which is correct but perhaps fails to register the qualitative leap achieved in recent years.

Regardless -- and while still affirming that the OP itself makes serious errors -- I think it's hard to find a definition of AI that is not simply "LLMs" under which the methods of the actual paper cited [2] would not fall.

[1] His dissertation was re-published as The Roots of Backpropagation. Especially in the Soviet Union, important not least for Kolmogorov and Vapnik, AI was indistinguishable from an approach to optimization problems. It was only in the west where "AI" was taken to be a question of symbolic reasoning etc, which turned out to have been an unsuccessful research trajectory (cf the "AI winter").

[2] https://arxiv.org/pdf/2312.04258

They bury the important point halfway through the article: Amazon is a free-cash-flow machine and it has almost always been so. The distinction is subtle [1], but consistent (and ideally ever-growing) free-cash-flow is what really matters in valuing a highly capitalized company, even if the firm has always re-invested it each quarter to date. The important point to investors is that the firm's re-investment is a decision -- the company is not brittle to economic downturn and any quarter could decide to payout investors through share buyback etc.

Copious free-cash-flow every quarter is why software companies generally have higher valuations than traditional industries and why it was novel that Amazon, which is not obviously a software company, behaves as one financially.

[1] https://www.investopedia.com/terms/f/freecashflow.asp

The timeless dilemma! I don't know how old you are, but just don't let the experience break your internal motivation as it can be hard to recover -- and remember however much 100k is and however much you save, it's still many years at that salary to genuine retirement. I don't know what equity packages are like these days at OpenAI/Anthropic, but especially if you're interested in the topic and have strong beliefs about how AI should play out in the world it's worth considering rolling the dice on a 2-4 year sprint. I imagine SA type positions at either of those companies are some of the most interesting roles, especially working in legacy industries/government [1], since you'd get to see firsthand how/where it's most effective (to say nothing of honing your language skills). Good luck regardless!

[1] https://openai.com/careers/solutions-architect-public-sector... for example - listed salary is 2x your current in the US, not sure what the salary is like in the EU.

Yeah, I've had similar results. Even with GPT-o1, I find almost all errors at this point come from the web search functionality and the model taking X random source as an authority. It's interesting that I find my human intelligence in the process is most useful for hand-collecting the sources and data to analyze -- and, of course, for directing the process across multiple LLM queries.

On the second point, it’s worth noting how many of the most vocal and well-positioned critics of LLMs (Marcus/Pinker in particular) represent the still academically dominant but now known to be losing side of the debate over connectionism. The anthology from the 90s Talking Nets is phenomenal to see how institutionally marginalized figures like Hinton were until very recently.

Off-topic, but I couldn’t find your contact info and just saw your now closed polyglot submission from last year. Look into technical sales/solution architecture roles at high growth US startups expanding into the EU. Often these companies hire one or two non-technical native speakers per EU country/region, but only have a handful of SAs from a hub office so language skills are of much more use. Given your interest in the topic, check out OpenAI and Anthropic in particular.

Charitably, I don’t understand what those like you mean by the “whole facade” and why you use these old machine learning metrics like “accuracy rate” to assess what’s going on. Facade implies that the unprecedented and still exponential organic uptake of GPT (again see the actual data I linked earlier from Mary Meeker) is just a hype-generated fad, rather than people finding it actually useful to whatever end. Indeed, the main issue with the “facade” argument is that it’s actually what dominates the media (Marcus et al) much more than any hyperbolic pro-AI “hype.”

This “80-20” framing, moreover, implies we’re just trying to asymptotically optimize a classification model or some information retrieval system… If you’ve worked with LLMs daily on hard problems (non-trivial programming and scholarly research, for example), the progress over even just the last year is phenomenal — and even with the presently existing models I find most problems arise from failures of context management and the integration of LLMs with IR systems.