AI migrated legacy COBOL programs to Java, bugs included | Hacker News Reader

AI migrated legacy COBOL programs to Java, bugs included

arxiv.org

100 pointsfelineflock99 comments

Before, only the senior cobol programmers at the company understood and knew the codebase.

Now, no one does.

I can see the allure of moving away from a legacy cobol system. But an AI rewrite doesn't actually solve any of the issues with having a legacy cobol codebase. You just have a new system no one knows or understands.

> Now, no one does.

I got a chuckle over this, but in my experience working on legacy systems, this is the case already. Using AI to translate/explain the code brings extra understanding.

Whilst true, it's not without its gotchas, and I saw a video the other day that elegantly highlights this: https://www.youtube.com/watch?v=t6MAZW-joCo. A false sense of confidence is bad; one that is offloaded onto somebody else is another.

Can it really explain the code if the information is not there anymore ? It was in a binder that got shredded 40 years ago by accident.

More like it will hallucinate some explanation & cause even more confusion.

If what you want is the "Why", you're right it probably won't get it right. It'll just give you baseless speculation. But if what you want is the "What" or the "How", I think it can be useful.

The code is the information. If you need binders of info then it wasn’t codified

Yes, it would take a long time for humans to remake a documentation from the codebase itself, but AI could do that much quicker and with close to perfect accuracy, Code is superior to documentation as it's the direct intended behavior (including bugs), in a sense, we won't even need documentation in the future, except maybe to lower AI costs.

Having worked in COBOL extensivly in the 80s and 90s, as well as working for a software house that did migrations (Craziest was Plan assembler to Dec Vax C).

I do wonder how an ambitious AI would handle the maths. COBOL is renowed for its fixed no rounding maths, utterly rigid and defined in the code, Java on the other hand is not (e.g., 0.1 + 0.2 becomes 0.30000000000000004), unless they are extensivly using BigDecimal, then its already a broken migration.

I've seen projects proclaim in the past to migrate COBOL systems to Java, and fail expicly when the solution was a dtaawarehouse system and a seperate Java system using the legacy backend to do all the fancy marketing reports or whatever other department was pushing for changes they wanted without a grasp of what it truly entailed and blinded by some sales pitch.

It's not just the code that needs migrating, its the data as well, then you want solid robust infrastrucure, which is why IBM still sell those mainframes to run all that well battle tested COBOL code.

Is it impossible to migrate to Java or any othger language - no, anything can be done, but the level of devil in the details and not just software, the data and a robust system to run it upon. It is very easy for even the best to overlook a gotcha.

We have all used an AI where it has gone down one path and you then point out something and it corrects itself, without the knowledge and a load of old grey beads(or a team) to act as devil advocate and try to pull the output apart, YOU can end up with something that works for everyones satisfaction, but is hiding a few gotchas down the line. AI coding needs a full debate team with one puishing it and one countering it almost I'd say.

If it works, why change it is always a good approach and why we mostly have in certain systems a COBOL backend with APIs or datawarehousing to expose for fancy java or other language added value/functionality.

I can only image generations of directors pushing cutting edge to replace legacy systems, and learning that cutting edge can cut you. My sympathy for the IT departments who have to deal with that as it has always been a bain of IT departments who are then forced to justify why it's fooling in a constructive way and explain it to non IT people in upper managment why the case.

As for migrating COBOL to anything, I'd not do it myself, id suggest a rewrite/redesign of a new system from scratch and then run both in parallel for a long time and make sure they actually do the same job and results. Even hand migrating back in the day, would produce a lot of messy code that refactoring would make sence and yet, not what the client would pay for as that would be a rewrite.

You can't use BigDecimal either. There are libraries whose whole purpose is just to exactly reproduce COBOL arithmetic, eg. the IBM Decimal Arithmetic Library.

Wow, their Java code example [0] makes COBOL look elegant and terse by comparison.

[0] https://community.ibm.com/community/user/blogs/gregory-cerne...

I know it's not the only problem (but it is IMO the main one), but this is why I'm a proponent of allowing operator overloading in languages.

Yes, you can't tell at a glance if `+` really still does what it should, but I find that problem no different from a library mis-naming a function and/or said function having strange side-effects.

`+` should do addition, period. No, not even concatenation if possible (I do find the lack of a separate concatenation operator to be a language flaw --- that said, I understand the use of `+` for concat in libraries targeting languages that offer no alternative).

I think the abuse (especially in C++ ... seriously, allowing overloading of the comma operator? --- and <iostream> working via bit-shifting streams by ${some_string} bits) gave everyone a bad taste and they kind of threw out the baby with the bathwater.

php has a separate `.` operator that concatenate and you get an error if you try to do it with a `+`. That's indeed better but most of the time you concatenate with `sprintf` anyway

Yes, and it's a surprising good part about PHP (a language that otherwise has an enormous collection of warts, to put it mildly).

I feel like the more dynamic and/or weakly-typed the language is, the more important it is to have a separate concat operator.

The worst ones being weakly-typed and dynamic. Consider `'5' + '10'` in JavaScript vs `'5' - '10'`. I also very much dislike Python's `+` for concat, but at least it's more strongly typed.

PHP, Lua, and Erlang are three examples of dynamically-typed languages with separate concat operators (`.`, `..`, `++`; respectively), and they all are better for it.

That said, even statically-typed languages benefit greatly from it. Maybe if C++ had a concat operator, it'd never end up abusing `<<` for writing to a stream (yes, there's no opposite for reading, but maybe they wouldn't end up doing that in this case).

For example, D uses `~` in infix for concat. Even UnrealScript, a 1990s game-engine-specific language (which tend to be a mess in general) did this right (it uses `$` or `@` --- with `a @ b` being equivalent to `a $ " " $ b`, IIRC).

It seems like Java is thinking about it now that Valhalla is landing and they’re expecting more user defined mathematical types.

Aehm we were writing proper math code in Java 15 years ago already.

Your 0.1 + 0.2 example wasn't an issue than and is not an issue today.

"Now, no one does."

well, we just need to learn the rewrite then ???

Yeah, "We are looking for a developer specialized in critical Java code following idiomatic COBOL which hasn't been tested in production yet and which nobody is sure about whether the exact behavior matches what's been running in production. You will enjoy meaningful work in an environment which has been considering engaging in environmental concerns for decades and which caries the well-being and social condition of fellow human beings as a core values, directly addressing your middle age existential crisis. Extra bonus points if you like using LLMs. The pay is good, but remember you are a Java developer, not a COBOL developer, and also remember we are already paying for tokens to do your job. Those are also possibly getting more and more expensive so we'll see about the raises. Job starts as soon as possible."

The problem isn't as much understanding what the code does (that's hard but not impossible), but understanding why that duplicate INSERT into the database is needed.

The guy who no longer works at the company knew it was because of a bug in the DB that wasn't fixed by the vendor after the company cancelled the maintenance contract due to financial difficulties during the great recession.

AI doesn't and can't know that. So AI proposes to "fix the bug as we port it" and remove the double INSERT. The CTO sends an email saying that the new AI fixed 174 bugs during the porting process and now the company has a much better and resilient system.

BREAKING NEWS: thousands of travelers can't get go back home after thanksgiving due to a computer glitch that accidentally cancelled their return tickets. Stay tuned to our live coverage!

nah you just make it hyperbole

Start extracting specs from the codebase.

Prompt the core architecutre well suited for it.

Let it write tests.

Let it vibe migrate from left to write.

Now you at least have a chance of a team being able to work on that code base and actually clean it up and make it good.

Cobold is hard, hard legacy with bad tooling.

Wasn't COBOL invented to read like business prose and be easily understood by programmers and non-programmers alike? If developers are able to write COBOL code no one understands than it failed its main objective.

IMHO every large software system should be developed on the basis of clear and concise functional specifications so everyone can understand what the software does and how it does so at a higher functional level.

Cobol code isn't esoteric, but it is designed in a way that is unambiguous and clear about its control flow, but very verbose and tedious to read/write. Translating from Cobol to a modern programming language is trivial for LLMs for that reason because the lack of any hidden state functionalities like generics for example.

Modern Cobol has most of the goodies, it is much more feature rich than Go.

Your last paragraph is getting at the core problem: COBOL _is_ easy to understand and write at the language level but what keeps people on these systems is decades of accumulated business rules, and it’s almost certain that there’s no accurate spec for the whole system. You might have some overall documentation but I’d expect to find cases where someone added a special case back in 1987 to handle an edge case and everyone who works in shipping knows there’s something special to do if this usually numeric field starts with a $ sign etc. but it’s not described in some kind of spec.

> Now, no one does.

Not true, AI does. It's AI job security...

If you can migrate using a strangle pattern, I can see it working pretty well and fast.

It seems to me that some companies are looking to move off COBOL and mainframes to say Linux and Java, because that is what someone was employed to do, not because it's necessarily a good idea.

I think COBOL systems would be plenty maintainable if the owners of the systems invested in maintaining them. If companies just let people retire, attempt to hire replacements with unremarkable salaries, then maintenance is a problem.

If they paid salaries that reflected how critical this work is, I don't think it would be that big a problem to find and keep good developers.

We have critical economic infrastructure being treated as annoyance and a cost centre.

I firmly believe that any thoughtful rewrite is good. Most companies just don't have resources for that. At least, it serves as a security review, but also refreshes team knowledge of the system. They usually also improve documentation if it was wrong.

Java should have significantly better tools for trying to understand a program than cobol.

It has had for plenty of years now, however many think vi and emacs is enough.

The biggest problem is not that bugs are migrated with COBOL, but that lots of new bugs are going to be introduced. AI is not deterministic, it will be making tons of mistakes. The only realistic low-error approach is incremental step-by-step migration using Cursor or similar tools. However, it requires much more time as each step must be prompted, tested and committed manually. Any hope that one-shot migraton of a large code base will not introduce enormous number of bugs is very naive. LLM is very bad on handling long context - it is their nature unfortunately. There is no answer to this problem yet.

Does it? I found llms to be great for straight conversions

At least if tescoverage is good, but well... That's something llms can also be used for

> AI is not deterministic, it will be making tons of mistakes.

From the paper:

> The COBOL source is passed through an internal deterministic Migrator to produce a generated Java target.

Also, humans are not deterministic either. Give the same COBOL -> Java translation to multiple developers and each will come up with a different solution. Heck, even the same developer will produce a different output for the same task, depending on the day of the week.

> Also, humans are not deterministic either.

Indeed, and that's why so few dare to migrate them, and so many who do fail or blow the budget many times over.

I think that was the implicit point of the comment: don't expect that with AI, suddenly we can convert all those COBOL apps with a single prompt.

But it says that the agentic authoring step patches the migrator when things get stuck. So while the execution of this stuff is deterministic, its actual content is not.

> Also, humans are not deterministic either.

And that's why it's usually a stupid idea to nilly-willy migrate large code bases to different languages (also I'm getting really tired of the "but humans aren't either" trope).

Which is why N-version programming is often used for systems that need to work (like plane software) and even then, as we know today, it can not be perfect. Google "plane software bugs history" for an insight into that.

So they solved the "make no mistakes" problem? They deserve a nobel prize.

Deterministic != Correct

> AI is not deterministic, it will be making tons of mistakes.

Just set the sampling temperature to zero and remove any unintended non-determinism during the parallel computation of the token probability distribution. The problem is solved? Of course, not. Non-determinism has little to do with LLMs' mistakes.

You don’t need the entire codebase in context in every moment to migrate it. Also AI being non deterministic does not prevent it from one-shotting perfect solutions 100% of the time for simple enough problems. And every model generation brings this bar higher. So that’s really not a fundamental problem. And we can also implement llm inference deterministically if we want, it’s just that it’s not worth the loss in performance to do it.

> And we can also implement llm inference deterministically if we want, it’s just that it’s not worth the loss in performance to do it.

My understanding is thats not possible (different from being practical), wonder if you have any literature, research to back up that claim?

This one from thinking machines: https://thinkingmachines.ai/blog/defeating-nondeterminism-in... Was discussed a lot also here on hn

Thanks for sharing, it was really good read. The authors in the article have done a good job for sure to validate their hypothesis and make it work for a particular subset of problem.

1. Local hardware, no networking or HTTP requests 2. No other parallel requests 3. 1000 runs bounded by 1000 tokens

I have worked a little bit in academia and I am not a big fan of the way favorable samples for the hypothesis are kept and unfavorable ones are thrown away. I might be wrong here, however its highly unlikely that the team would have just worked with one query. Chances are that a lot of different prompts with varying number of runs would have been tried to see what works and supports the hypothesis.

> The only realistic low-error approach is incremental step-by-step migration using Cursor or similar tools.

Correction: AI is not deterministic, the only realistic low-error solution is not a more complex use of non-deterministic AI, but deterministic transpilation.

The problem is that this results in COBOL-in-Java which runs correctly but it is a nightmare to maintain.

AI (LLMs) is typically not run deterministically for performance reasons, but can be turned deterministic if you want it to be. The output will still be probabilistic, however, so you will be no further ahead. Determinism isn't helpful.

You are, of course, correct about determinism. In the grandparent post I was accepting the previous poster’s somewhat sloppy use of “nondeterministic” because I was focussed on other aspects of the discussion.

If a considerable size of test data is available for any system, rewriting a well-understood one is much easier today than rewriting by hand. The issue is mostly with the “well-understood” part, as over the years, none cared about understanding and it became a working blackbox that is responsible for a critical part of large system and none wants to take the risk.

AI doesn't need to be deterministic.

But, hear me out, a COBOL to Java migration absolutely needs to be deterministic. Surviving COBOL programs run a lot of sensitive operations.

As a side note, I can't fathom why they chose Java. It's easier to teach COBOL to a competent engineer than to rewrite everything in a language that encourages onion architectures. Why not Go, for example?

Java as a programming language doesn't enforce any architecture. There is no difference to go. Its a system of folders.

The tests need to be green, the math needs to math, but the code can look a little bit different here or there.

I don't think its easy at all to educate someone on COBOL. Its carrier suicide if you want to ever do anything else again after that gig. I was writing java for years, switched for 2 years to php for another company and had a few companies complaining about these 2 years...

As someone that has worked at a bank maintaining RPG/COBOL and Java, the latter definitely encourages programmers to wrap things in yet another class. They think they're simplifying, but in reality, they're piling abstraction layers that have to be peeled off one by one when, for example, you're tring to find that EBCDIC-to-UTF8 bug.

A good Java architecture is possible, though, but it requires planning, discipline, and restraint; and yet the ever-changing business needs can mar your beautiful castle given enough time. COBOL is simpler and more straightforward, you have to go out of your way to mess it as badly as Java allows.

> AI is not deterministic, it will be making tons of mistakes.

Most compilers are also not deterministic, at least by default, but they don't usually make mistakes. Determinism isn't an important quality here. And if it were, AI can be deterministic, it just isn't normally for much the same reason compilers typically aren't (hint: performance).

I find it bizarre that I keep reading this here. Just one of those things that keeps getting blindly repeated without receiving any thought?

They cannot be made entirely deterministic. You have cases where the next set of token choices have the same probability, and most people also are using cloud provider served models which can be served across different hardware implementations which can modify the math used in running the model.

can you get some arbitrary number of 9s for consistency? Yea. You can’t get 100% deterministic though.

There is nothing about LLMs that prevent them from being 100% deterministic, unless you are really stretching the definition. There are implementation shortcuts often used in practice, just as is the case with compilers, that lead to non-determinism but there is no reason you have to rely on those implementation details. Not even if cloud providers do. LLMs can be run in environments under your control.

Computers are deterministic. When we talk about non-determinism we're just talking about where there are hidden inputs, but the fundamentals of computing means that all hidden inputs can become visible if you want them to be (although possibly at the cost of things like performance), so, yes, you can reach 100% determinism just fine if you wish to. It's just not worth the tradeoffs in most cases. But if a deterministic solution solved a problem here it would be worth it.

However, a deterministic LLM doesn't help here. LLMs don't "make mistakes" because they are typically non-deterministic. They would equally "make mistakes" when deterministic.

The largest test case was 4kloc.

There are hundreds of billions of lines of cobol in production.

The IRS alone has approx 160 cobol programs, averaging 230kloc each.

This sounds like what the company Mechanical Orchard offers: https://www.mechanical-orchard.com/

Their pitch is like, "we take your COBOL code, and all the real-world data you can give us. Then we model your COBOL program as a graph, where each node has inputs and outputs. Then we use AI to port each node, making sure that it has the same (input => output) mapping for all the test data you gave us."

> for all the test data you gave us

This is the secret sauce. You'll be responsible for any defects if you didn't prepare your test data to act like a massive unit test. Wonder how hard is to prepare the ideal test data vs writing the unit tests themselves in COBOL and verifying their translation. I guess if you throw enough data at it, the effort is minimal while the coverage becomes good enough to fix any bugs manually.

I remember a similar story shared this year at JAX2026 from the Sparkasse Group, they said they were using AI to migrate from COBOL, but they still were in the middle of the migration. Maybe they faced the same issues / problems? It seemed pretty zealous to me, that everything was working smoothly, but this article highlights the limitations

some years ago I talked to a software engineer at a bank, he said it would be too risky for them to move away from the mainframe what with regulations and all, I'm not sure if AI could make the banks more risk friendly, so if Sparkasse does that already, I, for one, am eager to learn the result (and am happy to have no account there)

Stuff like this are already a commercial product, it isn't only Zig to Rust rewrites going around.

https://www.pega.com/insights/resources/break-free-legacy-ma...

https://www.ibm.com/products/watsonx-code-assistant-z

https://global.fujitsu/en-global/pr/news/2026/03/30-01

https://www.rocketsoftware.com/en-us/insights/ai-powered-cob...

Why should I convert COBOL to Java? LLM can write COBOL just as fine.

Because humans don't want to spend time to learn how to use "dead" languages and humans still play a role in programming. Having LLMs churn out more COBOL instead of Java means more of the code base becomes 'terra incognita' for the involved humans. Learning COBOL isn't that hard - which was one of its stated purposes - but it is still seen as decidedly 'uncool' and 'legacy'. Maybe the increased role of LLMs in coding will change this and make coding in 'legacy' languages 'cool' again just like e.g. the availability of accurate time sources like mobile devices made mechanical watches trendy again or the direct access to music through streaming services made vinyl and now CDs (i.e. 'physical media') regain market share.

Maybe.

Still not something you want to risk your business on.

The irony is that talking to AI could not be any closer than COBOL's original goal.

yes i was wondering the same... what's wrong with COBOL? especially in an era of LLM... also, i find COBOL to be way more readable and intuitive than Java.

While not everything can be easily converted (IMS, CICS, reports, batch processing etc.), there are many situations where automated tooling can be helpful in migrations.

Related to this, I created a tool for situations where you want to compare COBOL code with Java code. It includes a preprocessing step where IMS etc. calls are converted to mocks that return JSON (from file), and also use JSON for input/output, and GnuCOBOL to run the program. More a proof-of-concept than production, but here is a link if somebody finds it helpful.

https://github.com/mikko-ahonen/coboltwin

How does ai implement all the not Cobol parts a Cobol program rely on? Job Contol, CICS, sort processors? Cobol and mainframe technologies are non existant in java on any modern machine

Batch job control is handled by alternative systems (already often triggering the work on mainframes anyway) like Control-M

CICS is the big issue but AFAIK there were attempts. Everything else, including sort and VSAM, has various options provided usually by COBOL compiler vendors.

For the article they just mocked it out for unit tests. But in reality you really can't - that’s a massive pain in the ass. Rewriting pure math is easy, but mocking CICS transaction isolation in java means spinning up these monstrous adapter frameworks that just tank performance

If the systems are to stay on mainframe/micro land, all IBM, Fujitsu, Unisys platforms support Java.

This is one set of ecosystems, where .NET going cross platform still misses out to Java, and neither of those companies cares about supporting it on their platforms.

There are some half way supported Python, Go and Rust versions as well, although I think they need the "WSL" of the respective mainframe/micro OS, aka UNIX services.

As I understand it, the translation isn't done by an LLM but by a deterministic AST-based migrator. Also, carrying over the bugs is the stated goal.

I just asked Google AI how LLMs translate COBOL code containing GOTOs into Java, a language that doesn't have GOTO. It gave several examples, one of them being this:

COBOL code:

  PROCESS-DATA.
      ADD 1 TO COUNTER.
      DISPLAY COUNTER.
      IF COUNTER < 10 GO TO PROCESS-DATA.

Java translation:

  while (counter < 10) {
    counter++;
    System.out.println(counter);
  }

If "counter" is 10 on entry, the COBOL code prints 11 while the Java code prints nothing. So not only keeping old bugs, but apparently introducing new ones too!

I wrote COBOL code for a few years at a job when I was a teenager. What makes legacy COBOL code difficult IMO is it can sometimes be very hard to maintain a mental execution state when examining the code, for several reasons:

1. all variables are global, aka, WORKING-STORAGE. You list all the variables used in the program and they are accessible to the entire program.

2. programs are divided into paragraphs. Control normally flows sequentially top to bottom through paragraphs, one executing after another. Except that the PERFORM statement can drastically alter this normal flow control, and you can't tell by looking at a paragraph how it will be executed. To do that, you have to look at all PERFORM statements that mention this paragraph or any paragraph physically before it, because in COBOL you can say PERFORM PARA1 THROUGH PARA27. If PARA13 is physically between PARA1 and PARA27, it's potentially going to get executed.

3. In true legacy COBOL, before structured COBOL was a thing (circa 1985), the main control flow statement in addition to PERFORM was GOTO. Lots of flag setting, and lots of GOTOs. So in the previous example, you can't tell if PARA13 is going to get executed because any prior statement might be a GOTO PARA14, skipping execution of PARA13. But even worse, you are still under the influence of the PERFORM THRU, so after PARA27 is executed, control returns to the statement following the PERFORM THRU, wherever that was. But if you GOTO PARA27, without being under a PERFORM THRU, then PARA27 is executed followed by the next sequential paragraph. Trying to figure this out statically by looking at the program can be very difficult, especially considering PERFORMs that are nested at runtime but may not be anywhere near each other in a code listing.

When they migrate to Java how are they migrating things like the messaging (send, receive) functionality?

I have been translating Motorola 6809 assembly into C. It is not that bad. I don’t find AI saves me much time writing the code because understanding it is still the bottleneck, but man it is good at explaining stuff.

COBOL is not about the language, it's about the whole environment in the mainframe. LLM fanboys won't understand this. You need something like a mainframe with a 99.99% uptime no matter what happens in hardware, with live CPU swapping and such.

Yes and that mainframe runs several other technologies besides COBOL, and more importantly, that don't depend on it (COBOL fanboys won't understand this ;) )

Given the option, I would go with DB 2 on z/OS instead of COBOL, for example.

Erlang/Elixir/OTP has a resilient environment with hot swapping. Is that truly problematic?

Can you explain why you need one machine never falling over ever instead of a cluster of machines never falling over at once collectively?

You don't. Either would work. Getting rid of a system running cobol on a mainframe is perhaps the hardest work possible in software engineering.

Yeah nah maybe fix the bugs before swapping the average COBOL dev for the average Java dev.

sometimes if the bug exists for enough time it is intended behavior

Indeed, there may even be a whole lot of code that depends on it.

Yeah. Bug-for-bug migration is a real thing in large code-bases in the industry. You want to replicate all behavior of the code regardless whether the behavior is a feature or a bug.

See Hyrum's Law: https://www.hyrumslaw.com/

Yep, and when it’s about money you don’t want any unexpected.

Absolutely, and you'll have all kinds of fixes for the symptoms throughout the project.

COBOL isn't hard, the tooling around it on old systems are a pain though.

Anything running on COBOL to day is a large enterprise system. You'll have reports running in other systems in subsidiary companies that rely on bugs in the upstream cobol code.

One of the hinges on our front screen door was coming off, the screws were coming loose from the wooden door jam. So I screwed it back on.

My wife and son then told me I "broke" the front door. I was confused, I thought I had fixed it

Turns out, with the door coming off the hinges, it wasn't closing properly, which meant it was ajar most of the time, so our dog could push it open with her body.

Now I'd fixed it, it closed properly all the time, so our dog started whining for someone to open the front screen door for her

So I loosened the screws, and hence "fixed" it by breaking it again

So a form of https://xkcd.com/1172/

Ah, the classic load bearing bug.

hyrum's law applies here

COBOL will never die. Whatever this is will only result in more COBOL being written.

I can see it.

"Let's just do this in COBOL for now to get it working. Add a ticket to use AI to migrate it to Java in the backlog."

"Oh, you mean we don't have to learn java now? Sweet! Let's keep writing in COBOL!"

To IBM

As specified, please find 99997 correct parts and the 3 defects (do not use)

300 to 4000 lines of "production like" (whatever that is) cobol code which is easily ported to a non-mainframe env. Our's sometimes uses assembler in its innards, so good luck with real legacy code spanning a dozen files and 50k loc...

I recently threw in (want to check those intelligence metrics!) some real production code into a non-agentic system (just to get a feel how things perform without a custom harness) and results where ... interesting. The particular program uses some preprocessor no LLM we have access to (newest was GPT 5.5) has any clue about - so they confabulate what it could do (Gemini 2.5 didn't even notice there was a preprocessor...). This is expected of course but it somehow seems the problem of this technology that unless you feed it masses of data or mechanically break up the tasks in rote subunits, it just doesn't do anything sensible still...

Yeah that’s why they delegated code gen to deterministic tooling and saved the model for input fuzzing - let it fight the data instead of legacy syntax

there was some London company migrating Cobol to Rust - with 'a.i' assistance.

I don't remember their name, hopefully they stayed in business. since probably convincing financial firms that using Rust is better than Java is a mammoth task.

All this ranting about COBOL. Makes me fairly sure AI hasn't tackled RPG3 yet :-D

Reply on news.ycombinator.com