DeepSeek V4 Closes the Last Defensible Gap — Open-Source Frontier Parity Is No Longer a Forecast, It's a Definition Problem
textak holds open-source frontier parity at 75%, up from 72% — and today's data is the strongest single-day evidence package we've seen for this forecast. But the movement in our probability reflects a genuine analytical problem we need to name openly: as traditional benchmarks collapse to noise-level gaps, 'parity' has become a moving target, and our forecast's credibility depends on being precise about which dimension we're measuring.
The headline number from today is this: the MMLU gap between open and closed frontier models collapsed from 17.5 points to 0.3 percentage points over the past year. That's not a trend — that's near-resolution on the benchmark that defined the capability gap for three years. Llama, Mistral, Qwen, and DeepSeek now match or beat closed models on multiple standard evaluations. DeepSeek V4 Pro posts 80.6% on SWE-Bench Verified, 93.5% on LiveCodeBench, and a 3206 Codeforces rating — ahead of GPT-5.4 and Gemini 3.1 Pro on competitive programming. On the benchmarks that defined this forecast's thesis when we set it, the case is close to closed.
But here's the honest version of what's happening: the frontier has moved. The MMLU collapse is real, but it's also evidence that MMLU is no longer the relevant measure. Claude Fable 5 scores 80.3% on SWE-Bench Pro. DeepSeek V4 scores 80.6% on SWE-Bench Verified — these are different evaluations, and the closed-model lead on the harder SWE-Bench Pro remains meaningful. On Terminal-Bench 2.1, where real agentic task performance is being measured, Gemini 3.5 Flash scores 76.2% and DeepSeek explicitly acknowledges trailing closed-source frontier by 3-6 months. The frontier has migrated from knowledge recall to multi-step agentic execution, and the gap there is not yet 0.3 points.
This is why our 75% reflects benchmark parity on the dimensions where we originally set the forecast — general reasoning, knowledge, and standard coding — rather than parity on all dimensions. What the 75% does NOT yet fully account for: the post-training and RLHF techniques that Anthropic and OpenAI hold proprietary, which appear increasingly load-bearing as raw capability gaps close. DeepSeek's own acknowledgment of a 3-6 month lag is direct evidence that open-source developers themselves don't believe full parity holds today. That honest self-assessment from the strongest open-source competitor is actually the most important data point in today's package — more so than the benchmark scores.
The strongest counterargument to our 75% is that we're measuring the wrong thing. Enterprise deployment data shows a 50x cost variation for similar accuracy and a 37% gap between lab benchmarks and real-world performance — and that gap may be structurally larger for open-source models that lack the production infrastructure, reliability SLAs, and support ecosystems of closed alternatives. 'Parity' on a leaderboard and 'parity' in a Fortune 500 production environment are different claims. We're comfortable at 75% for benchmark-defined parity by end of 2026. We would need to see consistent real-world deployment performance data — not just benchmark scores — to push this above 85%. What would drop us below 65%: if Anthropic's Mythos-class unreleased capabilities, now partially visible through Fable 5's 80.3% SWE-Bench Pro score, represent a systematic frontier advantage that open-source cannot replicate within 12 months of closed release.