The 80% problem: how banks modernize legacy COBOL with AI

The shape of the problem

COBOL didn't go away — it went underneath. By figures widely cited from Reuters and Communications of the ACM, COBOL still underpins more than 65% of enterprise software and around 70% of business transaction processing, including roughly 95% of ATM swipes — on the order of $3 trillion in daily commerce. The people who wrote it are retiring faster than banks can replace them, and the systems are largely undocumented.

That's the squeeze: the payoff for modernizing is enormous, and the cost of getting it wrong is existential — a mistranslation here doesn't ship a buggy feature, it miscomputes money at the center of the bank. AI arrived promising to collapse the timeline. The interesting question isn't whether it can write Java. It's which part of the job AI actually makes cheaper, and which part it makes more dangerous.

What the system does

Credible deployments don't treat this as one magic translation step. They run a five-stage pipeline, and AI plays a different role at each stage — comprehension first, generation last.

COBOL / PL-I / JCL / copybooks
        │  static + semantic analysis
        ▼
   dependency graph → application blueprint        (UNDERSTAND)
        │  RAG-grounded, plain-language explanation
        ▼
   legacy logic restated in domain terms            (EXPLAIN)
        │  domain decomposition, dead-code flagging
        ▼
   bounded business-domain modules                  (DECOMPOSE)
        │  LLM translation, grounded by static analysis
        ▼
   Java / .NET / PostgreSQL                         (TRANSFORM)
        │  parallel-run vs. original + human sign-off
        ▼
   validated, mergeable code                        (VALIDATE)

Where AI actually earns its keep

The productivity numbers are real, and they cluster on the comprehension end of that pipeline, not the generation end.

These are bank-sourced, and they describe AI assisting engineers — explaining, reviewing, documenting — far more than autonomously rewriting cores. Citi's 740,000+ automated code reviews and roughly 100,000 engineering hours a week freed (bank-reported, 2025 earnings via American Banker) also come with ~2,000 legacy applications retired. Bank of America's April 2025 disclosures put developer efficiency gains around 20%, with some test cycles streamlined by up to 90%.

Goldman's Devin deployment is the most-cited autonomous case, framed by the bank itself as a "hybrid workforce" under continuous human supervision, deployed org-wide for legacy refactoring and debugging. Note the framing carefully: CIO Marco Argenti's "3–4x more productive" figure is an expectation, not an audited result (CNBC, Jul 2025). It belongs in the "watch this" column, not the "proven" one.

The comprehension bottleneck

The step everyone under-resources is Explain — and it's the one that decides whether the whole program holds. Getting an accurate read on what a legacy process does requires the people who built it: who know why it was designed that way, what regulation it quietly honors, what edge case it silently absorbs. An engineer fluent in prompting but ignorant of the banking domain cannot supply that. AI accelerates a domain expert; it does not replace one. Skip this step and you faithfully migrate logic you never understood — bugs and all — into a modern language, where it becomes your team's permanent liability.

This is why the honest framing is that AI leverages the work while process and people carry it: business analyst → architect → QA, every link understanding the system. Modernization is a domain project with AI acceleration, not an AI project with a domain footnote.

Quality gates, not blind trust in the model

The failures here are not the loud kind you catch in a demo. They compile. They pass shallow tests. They're wrong anyway.

Silent semantic drift is invisible on any one transaction — it surfaces weeks later as a reconciliation nightmare across millions of records, with no crash to trace it to. Hallucinated copybook logic happens when the model can't fully resolve a shared data definition (field sizes, REDEFINES overlaps) and confidently guesses a layout that doesn't match reality; documented cases have corrupted a database column off exactly this. Lost-in-the-middle context loss — LLMs reliably under-weight the middle of a long input — is often the root cause of the two failures above, since the one definition that mattered got dropped from the model's attention.

The gate that catches them is not model confidence — it's parallel-run validation: run the migrated code and the original against the same inputs and diff the outputs, with human sign-off before merge. Vendor tooling reinforces this pattern; IBM's watsonx Code Assistant for Z, for instance, uses RAG so the model answers only from ingested content, specifically to suppress hallucination — a vendor claim, and a sound control regardless.

The governance gate

Because these models inform business decisions, they sit inside the model-risk perimeter. SR 11-7 (Fed/OCC, technology-neutral since 2011, updated by SR 26-2 in April 2026 for the AI era) applies to any model that informs a decision — now including the LLM generating production code — for banks above $250M in assets. The EU AI Act separately classes credit scoring as high-risk. In practice that means non-negotiable controls: immutable audit logging, role-based access control, human-in-the-loop sign-off, and drift detection with rollback. Letting an agent modify a core system unsupervised isn't speed — it's an unbounded risk with a named accountable owner on the hook.

The verdict

Deploy now for AI code comprehension, explanation, documentation, and review — proven, bounded, auditable, and where the bank-verified numbers actually live. Pilot first for autonomous COBOL-to-Java migration of critical systems: instrument a bounded, non-critical module with mandatory parallel-run validation, and only scale once defect and PR-acceptance rates hold across a full audit cycle.

And the reframe worth keeping: modernization makes a system composable, and composable is the precondition for every agentic use case on the roadmap. You cannot layer agents on an opaque monolith. This isn't a cost-cutting side quest — it's the foundational work that makes the rest of the AI strategy physically possible. The reframe moves the business case from "reduce legacy spend" to "buy the right to build the next decade of the bank."

The result

The banks pulling ahead aren't the ones that bought the best tool. They're the ones that understood their own system first — then used AI to compress the timeline around that understanding. AI didn't replace the engineers. It let them read, review, and reason across decades of code at a speed the retiring experts never could.