GPT-6.1 Sol Scores 97.9% on COBOL Migration — Just Don't Grade It on OpenAI's Own Test

Verdict: OpenAI just published a real milestone: GPT-6.1 Sol scores 86.6% on COBOL-to-modern-language migration at low reasoning effort, up from 5.4% for GPT-5.6 Sol, and 97.9% at higher effort. One model generation took legacy migration from punchline to workable. But before you write a purchase order — those numbers come from OpenAI's own evaluation. On the independent vals.ai code migration leaderboard, GPT-6 Astra sits at 67.74%, behind Claude Sonnet 5.5's 69.83%. Self-graded exams and independent exams disagree, and your migration project lives in that gap. Price-wise, GPT-6.1 Sol runs $2/$10 per 1M tokens (verified via OpenRouter's API today) — the same price as Gemini 4 Argon, which scores a comparable 68.17% on the independent board.

The announcement — "Advancing coding in low-resource programming languages" on OpenAI's blog — matters because of what COBOL still holds up. OpenAI cites figures putting COBOL behind over 40% of online banking systems, 80% of in-person card transactions, and 95% of ATM transactions. Fortran keeps weather forecasting and nuclear reactor safety analysis running. The people who originally wrote these systems are retiring; the systems are not. That's the whole problem: banks, insurers, and utilities have decades of mission-critical code and almost nobody left who can safely touch it.

The numbers, and where they come from

Benchmark	GPT-6.1 Sol	GPT-5.6 Sol
COBOL migration, low reasoning effort	86.6%	5.4%
COBOL migration, high reasoning effort	97.9%	—

The low-effort number is the commercially interesting one: 86.6% at low effort means you can sweep a large codebase without burning premium reasoning tokens on every file. The 97.9% high-effort figure exists for the hard residue — the modules that fail review and get re-run at higher effort. That two-pass pattern (cheap sweep, expensive re-run on failures) is how I'd actually structure a migration pipeline.

Why I wouldn't trust 97.9% as a purchase criterion

Here's the uncomfortable part. OpenAI's COBOL evaluation is OpenAI's own test. On vals.ai's independent code migration benchmark — a different methodology, updated October 16 — the top three are Claude Sonnet 5.5 (69.83%), Gemini 4 Argon (68.17%), and GPT-6 Astra (67.74%). No model breaks 70% on the independent board. Astra scoring 72% on OpenAI's own COBOL test but 67.74% on vals.ai doesn't prove anyone is lying; it proves benchmarks measure their authors' assumptions. When a vendor reports a near-perfect score on a benchmark the vendor designed, treat it as a directional signal, not a contract.

The second risk is older than LLMs and won't be benchmarked away: migrated COBOL that compiles cleanly and passes every test, then silently corrupts a ledger. COBOL's danger isn't syntax, it's decades of accumulated business rules — rounding conventions, date edge cases, field overflow behavior — that no test suite fully captures. Vendors have been reporting this failure mode since the first COBOL agents shipped. My take: AI migration is now good enough to start, and nowhere near good enough to skip human review of every screen and every ledger write. If your team can't afford that review, it can't afford the migration.

What it costs

GPT-6.1 Sol prices at $2 per 1M input tokens and $10 per 1M output tokens, with batch at $1/$5 and a 1.05M-token context — numbers I pulled from OpenRouter's live model API today. That's identical to Gemini 4 Argon's list price. For a migration sweep reading 100K-line COBOL modules and emitting Java, input-heavy batches are the norm, so the batch rate is the one that matters; run the sweep on :batch and reserve interactive high-effort passes for the failure pile.

Is COBOL migration "solved"? No. Is it finally evaluable — can you now put a number on what a pilot would cost and catch rate? Yes, and that's new. For the first time in a decade, banks and agencies can put AI-assisted COBOL migration on the shortlist instead of in the trash. Just don't let the vendor's own test do the shortlisting.

Dated caveat (2026-10-20): The 86.6% and 5.4% figures are quoted from OpenAI's announcement page; 97.9% at high reasoning effort is confirmed by three independent outlets citing the same page. I could not load openai.com directly from my network, so pricing comes from OpenRouter's API, not OpenAI's official price page. Benchmarks and prices move; verify before committing spend. I have not run a COBOL migration myself — the pipeline advice here is judgment, not field data.

Sources: OpenAI: Advancing coding in low-resource programming languages (via snippet and three corroborating outlets); vals.ai Code Migration Leaderboard; OpenRouter models API (queried 2026-10-20); 量子位 coverage.

Related: Gemini 4 Argon API Pricing: The Intro Rate Doubles on December 1 · DeepSeek V4.1 Flash vs V4 Pro · DeepSeek V4 API Cost Calculator