Open-Source Has Won the Coding Benchmark. The Enterprise Is a Different Fight.

We need to be precise about what this forecast is actually predicting, because the original phrasing — 'open-source model matches closed frontier performance' — is vague enough that a reasonable person could argue it resolved YES today. We're not going to hide behind ambiguity. So let us state the operative definition clearly: we define 'matches closed frontier performance' as benchmark parity across a composite of coding, reasoning, and graduate-level science tasks, plus demonstrated production deployment at Fortune 500 scale. Under that definition, coding benchmark parity is confirmed. The SWE-Bench numbers are direct evidence of benchmark score parity — same test, same score, different license. That is not circumstantial. But benchmark score parity is necessary, not sufficient, for what we're actually watching. SWE-Bench Verified measures task completion on a curated problem set under controlled conditions. It does not measure latency under sustained agentic load, context window reliability on real enterprise codebases that span millions of tokens, or tool-use fidelity across the kind of multi-step workflows Virgin Voyages is now running at 1,500-agent scale. The gap between 'passed the test' and 'running in production reliably' is where the forecast lives.

The reasoning and science benchmarks tell a more complicated story. Claude Mythos Preview at 94.6% on GPQA Diamond and GPT-5 at 100% on AIME 2026 are genuinely significant — and we need to be transparent about their epistemic status. These figures come from LLM Stats reporting on preview releases and benchmark leaderboard data, not independently audited submissions. We're treating them as credible provisional data, not confirmed fact, and we'll revise that assessment if they get independently verified or contested. What they suggest is that the closed frontier has not stood still while open-source closed the coding gap. Frontier labs appear to have a meaningful lead on the hardest graduate-level science reasoning tasks, which is exactly where you'd expect proprietary training data and post-training technique advantages to compound.

Here's the part of our thesis that keeps us up at night, and we want to name it explicitly because it's the strongest version of the counterargument: the open-weight models driving today's evidence — DeepSeek, MiniMax, GLM, Qwen, Kimi — are predominantly Chinese lab output. And US Fortune 500 enterprise procurement decisions are increasingly subject to a barrier that cost curves alone cannot resolve: geopolitical and data-sovereignty risk. This is distinct from the 'compliance documentation is slow' friction we've previously cited. Several US agencies and major financial institutions have active internal policies restricting or prohibiting data routing through Chinese-developed model infrastructure, regardless of self-hosting arrangements. If the enterprise deployment leg of our forecast definition requires Fortune 500 adoption, and if the open-weight frontier is disproportionately Chinese-lab output, then we may be forecasting something that faces a structural US enterprise procurement barrier that 10x cost advantages cannot fully overcome. We haven't fully priced this in. It's a genuine gap in our model.

So why are we at 72% rather than lower? Three reasons. First, Meta's Llama lineage and the broader Western open-source ecosystem are not standing still — they benefit from the same training technique advances and cost dynamics without the geopolitical barrier. Second, the enterprise deployment evidence (Virgin Voyages scaling from 50 to 1,500 agents in four months on Google Cloud infrastructure) shows that agentic deployment at scale is happening right now, which conditions the market for open-weight alternatives. Third, the cost differential — sub-$1 per million tokens for GPT-4-level capability, near-zero marginal cost for self-hosted open weights — is not a temporary arbitrage; it's a structural shift that enterprise procurement committees will eventually be forced to reckon with regardless of geopolitical preferences. What would move us above 80%: a major Western-origin open-weight model (Llama 5 or equivalent) achieving composite benchmark parity across coding, reasoning, and graduate science, combined with a single confirmed Fortune 500 production deployment case study at scale, within the next two quarters. What would drop us below 55%: Mythos or a comparable proprietary release independently verifying a sustained 10+ percentage point lead across the composite benchmark set, or a formal US government procurement restriction on open-weight Chinese-origin models that meaningfully constrains enterprise adoption channels.