---
title: "Opinion: Legacy code may be where AI coding helps most, and open tests will build the trust it needs"
url: https://anawords.com/story/opinion-ai-legacy-code-cobol-open-tests-build-trust
published: 2026-10-20T14:01:16+05:30
updated: 2026-10-20T14:01:16+05:30
section: Opinion
author: Mayank Sahu
checked: sources checked by AI, approved by editor
version: 1
---

# Opinion: Legacy code may be where AI coding helps most, and open tests will build the trust it needs

OpenAI’s COBOL results are encouraging for banks and governments running old systems. Publishing MultiLangBench was a good step; independent checks are the next one.

**In short:** We think legacy systems are one of the most useful places for AI coding tools. OpenAI’s reported jump in COBOL migration accuracy, from 5.4% to 86.6% at low effort in its own test, is encouraging. Releasing MultiLangBench publicly was a welcome step, and independent results will help organisations trust these tools with critical code.

## Why legacy code matters

Many banks, insurers and government departments still run core systems in COBOL and Fortran. The people who wrote that code are retiring, and moving it to newer languages is slow and risky. That is why OpenAI’s new results caught our attention.

In OpenAI’s own tests, GPT-6.1 Sol migrated COBOL programs correctly 86.6% of the time at low reasoning effort and 97.9% at higher effort, up from 5.4% for GPT-5.6 Sol.[^1][^2] GPT-6 Astra averaged 87.4% on a 20-language test of building a database engine from scratch.[^1]

## Credit where it is due

Two things stand out positively. First, OpenAI chose a strict scoring rule for migrations: a translated program only counts if every hidden test passes.[^1][^2] That is the right bar for financial code. Second, OpenAI released MultiLangBench’s setup and results on GitHub, so others can check them.[^1] Open tests help everyone, including other labs.

OpenAI also says real migrations still need expert review.[^2] We agree, and it is good to see that said clearly.

## What we would like to see next

- **Independent results.** Universities, auditors or other labs running MultiLangBench would make the numbers more useful to buyers.
- **A public migration test.** The COBOL migration test has not been released.[^2] Even a smaller public version would help.
- **More model comparisons.** Results from several AI providers on the same open test would help teams choose.

## How teams can try this safely

Start with code that already has strong tests. Migrate small pieces, compare outputs exactly, and keep a person who knows the business rules in the loop. A rounding change that looks harmless can change financial results.[^2]

## Our view

AI will not replace the experts who understand these systems. It may help them move much faster. Open, independent testing is how that promise becomes something organisations can rely on.

## Fact ledger

- [Company claim] OpenAI reports GPT-6.1 Sol migrated COBOL correctly 86.6% (low effort) and 97.9% (higher effort). (source 1,2)
- [Confirmed] A migration counts only if every hidden test passes. (source 1)
- [Confirmed] MultiLangBench is released on GitHub. (source 1)
- [Unconfirmed] The COBOL migration test is not public. (source 2)

## Sources

1. [OpenAI: Advancing coding in low-resource programming languages](https://openai.com/index/low-resource-programming-languages/) — official
2. [CellCog blog: OpenAI’s COBOL test, 5.4% to 86.6% in one model jump](https://cellcog.ai/blog/openai-cobol-low-resource-languages/)