Kimi K3 and Inkling Set Open-Source Benchmark Highs

Moonshot AI released Kimi K3 on Thursday — a 2.8-trillion-parameter open-source model that the Beijing-based company says outperforms or matches leading proprietary systems from Anthropic and OpenAI on key benchmarks. The same week, Thinking Machines launched Inkling, a 975-billion-parameter multimodal open-weights model under Apache 2.0, scoring 77.6% on SWE-bench Verified. Together, the two releases mark a notable week for open-source AI benchmark performance.

Kimi K3: 2.8 Trillion Parameters, Open Weights Incoming

Moonshot AI’s Kimi K3 is now the largest open-source AI model by parameter count, at 2.8 trillion total parameters — roughly 75% larger than DeepSeek’s V4 Pro, which Moonshot’s own technical documentation places at approximately 1.6 trillion parameters. Benchmark results shared at launch show the model performing neck-and-neck with top closed-source systems, a claim that, if borne out by independent testing, would represent a significant shift in the open-versus-proprietary capability gap.

The model features a 1-million-token context window and native visual understanding. According to VentureBeat’s coverage, full model weights are scheduled for release on July 27, based on details shared by researchers who reviewed the company’s technical documentation. The model is currently accessible via kimi.com without a credit card.

The timing of the release — just ahead of the 2026 World Artificial Intelligence Conference in Shanghai — is notable. Moonshot AI had seen its market position erode over the prior 18 months following DeepSeek’s rapid rise, according to VentureBeat, making Kimi K3 a significant competitive reentry.

Inkling: Thinking Machines’ First Open Model Targets Enterprises

Thinking Machines — the American AI startup founded by former OpenAI CTO Mira Murati — released Inkling on the same day under an Apache 2.0 license, making it freely usable for commercial deployment. The model scored 77.6% on SWE-bench Verified, beating NVIDIA’s Nemotron 3 (71.9%) on software engineering tasks, and 91.4% on VoiceBench, compared to 94.4% for Gemini 3.1 Pro on high reasoning effort — placing it below the state-of-the-art but above most open-weights peers.

Inkling is a natively multimodal Mixture-of-Experts system capable of reasoning across text, images, and audio. A key architectural feature is what Thinking Machines calls “controllable thinking effort” — a mechanism that lets operators adjust compute cost per inference, rather than running at fixed reasoning depth. Weights are available immediately on Hugging Face and through the company’s own Tinker API.

Thinking Machines also announced a preview of Inkling-Small, a 276-billion-parameter variant optimized for cost-sensitive workloads.

Censorship Resistance as a Differentiator

One explicit design choice sets Inkling apart from most enterprise models: Thinking Machines states the model was built “to answer directly on topics that may be subject to censorship.” The company positions this as a trust feature for enterprises that need factual outputs regardless of topic sensitivity — a direct contrast to the content filtering common in hosted API models.

Benchmark Methodology Under Scrutiny

As new records accumulate, a parallel research thread is questioning whether leaderboard scores are as reliable as they appear. A paper published this week on arXiv introduces two new metrics — the Format Sensitivity Index (FSI) and the Parseability Sensitivity Index (PSI) — designed to measure how much a model’s benchmark score shifts based solely on how a prompt is formatted.

Across 140,000 generations spanning 7 QA tasks, 5 prompt-wrapper families, and 4 instruction-tuned models ranging from 7B to 72B parameters, the researchers found that mean FSI varies by over 30x across models. The core finding: accuracy differences between models on standard benchmarks can be largely explained by compliance failures — cases where a model doesn’t parse the output format correctly — rather than genuine reasoning differences. The authors argue that “reporting accuracy without wrapper variance and compliance is statistically fragile.”

This finding is directly relevant to both the Kimi K3 and Inkling releases, where benchmark comparisons are being used as primary evidence of competitive performance.

Singularity Gate: A Benchmark for Post-Cutoff Scientific Prediction

A smaller but methodologically distinct benchmark surfaced this week on Reddit’s r/singularity: the Singularity Gate, which tests whether frontier models can predict paradigm-breaking scientific discoveries published after their training cutoff — a direct probe of generalization rather than memorization.

According to the post, Claude Fable 5 currently leads the benchmark, though with a notable caveat: the original Fable 5 responded to 45% of benchmark tasks, while the latest version responded to only 39%, with a slight performance degradation also observed. The scores reported compare only tasks where both versions responded. GPT-5.6 Sol was described as a noticeable improvement over GPT-5.5 and placed second on the leaderboard.

The benchmark remains community-run and has not been independently verified. Its methodology — testing predictions about post-cutoff discoveries — is conceptually different from standard QA or coding benchmarks and has not yet been peer-reviewed.

What This Means

The Kimi K3 and Inkling releases, taken together, compress the performance gap between open-weights and proprietary models on standard benchmarks — but the arXiv FSI paper is a timely reminder that those benchmarks may be measuring format compliance as much as capability. Moonshot AI’s 2.8-trillion-parameter model is the most direct challenge yet to the assumption that frontier performance requires closed weights, while Thinking Machines’ Apache 2.0 licensing and censorship-resistance positioning target a specific enterprise segment that closed APIs cannot serve.

The Singularity Gate benchmark, while unverified, points toward a more demanding class of evaluation: not what models have memorized, but whether they can extrapolate beyond their training data. If that class of benchmark matures, it could substantially reorder current leaderboard standings.

For enterprises making deployment decisions now, the practical takeaway is straightforward: open-weights options at near-frontier performance now exist at a scale and licensing terms that were unavailable six months ago. The reliability of the benchmark numbers used to support those claims, however, deserves scrutiny proportional to the stakes.

FAQ

What is Kimi K3 and how does it compare to other open-source models?

Kimi K3 is a 2.8-trillion-parameter large language model released by Moonshot AI, which the company describes as the largest open-source AI model ever built. According to Moonshot’s technical documentation, it is approximately 75% larger than DeepSeek V4 Pro and benchmarks competitively with leading proprietary systems from Anthropic and OpenAI.

What makes Inkling different from other open-weights models?

Inkling, released by Thinking Machines under Apache 2.0, combines multimodal reasoning (text, images, audio) with a “controllable thinking effort” mechanism that lets operators trade compute cost for reasoning depth. It also explicitly targets topics that may be subject to censorship, positioning it as a trust-focused option for enterprise deployments, according to the company’s announcement.

Why do benchmark scores sometimes disagree between sources?

A July 2025 arXiv paper found that prompt formatting alone can shift a model’s benchmark accuracy by a range that varies over 30x across models — a metric the authors call the Format Sensitivity Index. The study argues that benchmark scores reported without wrapper variance and output parseability data are statistically unreliable, meaning two sources testing the same model with different prompt formats can reach meaningfully different conclusions.