Open Source AI Parity Claims Face Reality Check as Frontier Labs Pull Further Ahead
TexTak's forecast for open-source models matching closed frontier performance sits at 69%, but today's evidence complicates that picture. The UK AISI reports an 8-month lag for open-source models to match frontier performance—seemingly supporting our thesis. Yet Anthropic's Claude Mythos Preview represents a step-change capability that may reset the competitive landscape entirely. We're pressure-testing whether the gap is actually narrowing or widening at the frontier.
Our 69% probability rests on Meta's heavy open-source investment, training technique democratization, and dramatic compute cost reductions—100x drops are verified across the stack. The UK AISI data initially appears confirmatory: an 8-month lag suggests open source is consistently catching up to closed models. But this analysis demands deeper scrutiny. The 8-month figure represents an "upper bound" estimate on METR tasks, which may not capture the full spectrum of frontier capabilities.
Claude Mythos Preview exposes the analytical gap in our model. When Anthropic can develop capabilities that cause "industrywide panic" and represent clear step-changes over previous frontier models, we're not just seeing incremental advancement. The Mythos cybersecurity demonstrations—completing 32-step attack simulations, finding decades-old vulnerabilities—suggest frontier labs have access to training data, techniques, or architectural insights that aren't immediately replicable in open-source contexts. The Fortune report specifically noted these capabilities went beyond what previous models could achieve.
The strongest counterevidence to our position isn't just the 8-month lag—it's what that lag doesn't measure. If frontier labs are developing capabilities through proprietary post-training techniques, data advantages, or unreleased architectural improvements, the meaningful gap may be widening even as open-source models match last year's benchmarks. The UK AISI noted capabilities "already beginning to surpass expert baselines," but that baseline-crossing appears concentrated in closed systems.
What would move us below 50%? Evidence that Mythos-level capabilities require fundamentally proprietary resources that can't be replicated through open-source methods within our forecast timeframe. What would push us above 75%? Meta or other open-source leaders demonstrating comparable cybersecurity capabilities or step-change performance improvements that close the actual frontier gap, not just the benchmarked lag.