Open-Source Has Closed the Gap — But 'Parity' Depends on Which Dimension You're Measuring
textak places the probability of open-source matching closed frontier performance at 75%, moved up from 72%. Today's benchmark data from LLM Stats — showing Llama 4, Qwen 3.7, and DeepSeek V3.2 matching or beating closed models on reasoning and coding tasks — is the most direct evidence we've seen yet. But we want to be precise about what 'parity' means, because the answer changes depending on which dimension you examine, and one of today's news items cuts in exactly the opposite direction.
The benchmark story is legitimately strong. LLM Stats reports the open-versus-closed capability gap has compressed from 18 months to roughly 6 months, and the 10x annual pricing improvement for equivalent performance is verified. These aren't vibes — they're measurable on standardized reasoning and coding evaluations. This is direct evidence for our thesis, not circumstantial. The 75% reflects exactly this: the gap has closed to a point where a credible parity claim on defined benchmarks is now a matter of when, not whether.
But here's what keeps us honest: today also brought news that the Trump administration has imposed export controls on Anthropic's Fable and Mythos models, citing a reported jailbreak vulnerability. This matters for our forecast in a specific way. We've consistently flagged that frontier labs hold unreleased capabilities — Anthropic's Mythos in particular was listed as a potential step-change improvement. If Mythos represents a genuine capability jump rather than incremental progress, and if it's now restricted from broad deployment while simultaneously remaining unavailable for open-source benchmarking, we have a scenario where the frontier moves in ways we can't observe cleanly. That's not a reason to abandon the 75%, but it is a reason to flag that our benchmark-convergence evidence is only as good as our visibility into what frontier labs are actually holding back.
The stronger counterargument is the one we've always held against ourselves: benchmark parity, developer preference, UX quality, and commercial impact are four different things, and our forecast is technically about the first one. The 75% reflects convergence on standardized reasoning and coding benchmarks — not that a developer would choose Llama 4 over GPT-5 for a production enterprise deployment, not that the post-training stack has converged, and not that open-source models have matched the safety and reliability profile that enterprise customers actually care about. The McKinsey finding that only 6% of enterprises qualify as AI high performers is circumstantial evidence here — it suggests that the gap between benchmark performance and production value is large and largely unsolved, for both open and closed models.
The Chinese open-source signal — 41% of Hugging Face downloads from DeepSeek and Alibaba models, GLM 5.2 released under MIT license with 1M-token context hours after the Fable 5 export restriction — is worth tracking carefully. This is exactly the dynamic that makes export controls on US frontier models potentially self-defeating: the open-source frontier is increasingly Chinese, and it's moving fast. For our forecast, this is a push: it accelerates open-source capability convergence, but it also complicates the geopolitical framing around which 'open-source' models we're counting. What would move us above 80%: an independent evaluation — SemiAnalysis-style, not just Hugging Face leaderboard — confirming that a non-US-lab open-source model matches GPT-5 or Gemini Ultra on a pre-registered benchmark suite by Q4 2026. What would drop us below 65%: Anthropic's Mythos deploying publicly and demonstrating a capability gap that reasoning benchmarks simply don't capture.