Inkling Is Not Trying to Win: How to Measure an Open-Weights Fine-Tuning Base

On July 15, 2026, Thinking Machines Lab released its first model, Inkling, and did something labs almost never do on launch day: it told you the model does not win. Inkling entered the Artificial Analysis Intelligence Index at 41, well behind Moonshot's Kimi K3 at 57, and the release notes say plainly that it is "not the strongest overall model available today, open or closed." A lab founded by a former OpenAI CTO, sitting on a reported multi-billion-dollar raise, shipped a 975B-parameter open-weights model into fourth place among open models and led with the caveat rather than burying it.

That is either a strange thing to be proud of or a signal that the scoreboard is wrong. Inkling's pitch is that it is a good open-weights fine-tuning base: something you adapt to your task through Thinking Machines' Tinker service, not something you deploy off the shelf and rank. If that is the claim, then the Intelligence Index measures the wrong thing, because a base model's job is not to answer your benchmark questions well. Its job is to move the furthest, in the right direction, when you point your own data at it.

The problem is that "a good base for customization" is a marketing sentence until someone names the measurement. This piece names four of them, says how you would run each one, and then grades Inkling against all four. The honest answer is that Inkling clears the two measurements that are cheap to check and leaves the two that actually decide the question open, which is exactly the position a careful buyer should expect to be in on launch day.

Why the leaderboard is the wrong axis for a base

A leaderboard rank is a scalar summary of a fixed checkpoint on a fixed distribution of questions. It answers one question: if I send this model prompts that look like the benchmark, how often is the ungraded output correct? That is the right question for a model you will deploy as-is, an assistant behind an API where the model is a component and its general competence is the product.

It is the wrong question for a base you intend to fine-tune, for a mechanical reason. Fine-tuning moves the model. The number you care about is not where the base sits today but where your adapted model lands after you have spent your data and compute budget on it, and the two are only loosely correlated. A base that scores two points higher on an aggregate index can leave less room to improve on your task, retain less of its general ability after you specialize it, or cost more to serve the moment you deploy the result. None of that is visible in the rank. As our evaluation-crisis piece argues, a single benchmark number next to a model name is a hypothesis about behavior, not a result, and the hypothesis it supports here ("higher index means better base") is one nobody has actually tested.

The tell that Thinking Machines means this is what it shipped alongside the weights. Tinker is a first-party fine-tuning service, and Inkling is tunable through it on day zero rather than after the community reverse-engineers a training recipe. A lab that wanted a leaderboard number would have spent its launch budget on post-training the shipped checkpoint harder. A lab that wanted a base spends it on the customization path, which is what a co-shipped fine-tuning service is. The positioning is coherent even before you check whether the model backs it up: the release is optimized for the second use, adaptation, not the first, deployment as-is.

So the base-model question has to be re-posed as something you can measure on your own task. Four measurements cover it: how much headroom the adapter has on your data, how much general ability survives the specialization, whether you can actually serve the result on day zero, and whether the license and checkpoint format let you own the pipeline at all. Take them in turn.

What Inkling actually is

Before grading, the specs, because the architecture is the evidence for and against each measurement. All of the following is confirmed against Thinking Machines' model card, Sebastian Raschka's architecture notes, and the vLLM and SGLang release stacks.

Inkling is a 975B-parameter sparse mixture-of-experts model with 41B parameters active per token, released under Apache 2.0. The backbone is a 66-layer decoder that uses regular attention, not a Mamba-Transformer hybrid: 55 layers run local attention over a 512-token window and 11 run global attention with a learned bias over the preceding 1,024 tokens. Each MoE layer holds 256 routed experts plus 2 shared experts, and the router sends every token to 6 of the routed experts while the 2 shared experts stay active on every token. Routing uses a sigmoid gate with an auxiliary-loss-free load-balancing bias, the same recipe DeepSeek-V3 popularized. If you want the mechanics of why that routing choice matters for cost and capacity, our mixture-of-experts explainer covers the sparse-forward-pass tradeoff in full.

Two details are worth flagging because they shape the fine-tuning story. First, Inkling activates about 4.2% of its parameters per token (41B of 975B), noticeably denser than Kimi K2.5's roughly 3.2% (32B of 1T). Raschka reads this as a deliberate bet on quality-per-token over maximum sparsity, and for a fine-tuning base that bet is defensible: a denser active path gives an adapter more live capacity to move. Second, the architectural oddity is a set of short kernel-4 convolutions applied after the key and value projections and on the attention and MLP branch outputs, plus an extra RMSNorm right after the token embedding and a learned input-dependent relative-position bias in place of RoPE. These are cheap local-mixing and stability choices, not exotic ones, but they mean Inkling is not a bit-for-bit standard decoder, which matters when your serving stack has to implement it.

The context window is 1M tokens on the open weights (256K through the Tinker API). Pretraining ran on 45T multimodal tokens spanning text, images, audio, and video. Thinking Machines ships three checkpoints: BF16, MXFP8, and an NVFP4 deployment variant for Blackwell.

On benchmarks, Inkling's vendor-reported scores are 97.1% on AIME 2026, 77.6% on SWE-bench Verified, 79.8% on IFBench, and 91.4% on VoiceBench, with 87.2% on GPQA Diamond and 73.5% on MMMU-Pro alongside. Every one of those is a number the vendor measured on its own harness, and they are useful only as a rough ceiling on what the base already knows. The SWE-bench Verified figure carries a heavier discount than the rest: the benchmark's largest users have publicly documented that a large fraction of "Verified" tasks carry flawed tests, that scores swing 30-plus points depending on scaffold, and that training-data overlap shows up in every frontier model. A coding score reported on SWE-bench Verified in mid-2026 is a hypothesis with an active-adversary asterisk, not a fact, and it should not carry weight in a base-selection decision.

Inkling-Small: comparable index at a quarter of the size

Thinking Machines followed the flagship with a second open-weights checkpoint, Inkling-Small, and it is the cleanest illustration of why the index is the wrong axis for a base. Inkling-Small is a 276B-parameter mixture-of-experts model with 12B active per token, roughly a quarter of the flagship's size, released under the same Apache 2.0 license with the same family recipe: 42 layers, 256 routed experts with 6 active per token plus 2 shared, hybrid local and global attention, a 1M-token context, the controllable thinking-effort dial from minimal to xhigh, and BF16 plus NVFP4 checkpoints. It is multimodal over text, image, and audio, trained by distillation followed by a short reinforcement-learning stage, and fine-tunable on Tinker on day zero.

The headline is that it lands at Artificial Analysis index 40.0, essentially tied with the 975B flagship's 41, at a quarter of the active and total parameters. On its vendor-reported benchmarks it holds that ground: 95.5% AIME 2026, 80.2% SWE-bench Verified, 82.2% IFBench, and 90.1% VoiceBench, above the flagship on SWE-bench Verified and IFBench and within about two points on AIME and VoiceBench. The same discounts apply, the SWE-bench Verified figure most of all, so read these as a ceiling that happens to sit where the larger model's does rather than as proof the small model is better.

For a fine-tuning base this is not a footnote, it is the argument in miniature. A 12B-active model is far cheaper to run headroom and retention experiments against, and far cheaper to serve, than a 41B-active one, and it starts from the same index. If rank were the axis you would call a quarter-size model at the same index a lateral move. Because rank is not the axis, Inkling-Small may be the better base for your task precisely because it costs roughly a quarter as much to move with your data and to deploy. The correct response is to run the nine-step protocol below against both checkpoints and let cost-per-solved-task decide, rather than assuming the larger model is the stronger base.

Four measurements that decide a fine-tuning base

1. Adapter headroom on your own task

Headroom is the gap between the base model's zero-shot accuracy on your task and the accuracy your adapted model reaches after fine-tuning. It is the measurement the leaderboard most directly fails to capture, because a high base score can mean less room to climb and a mediocre base score can mean a steep, cheap improvement curve. What you want is not a high starting point; it is a high finishing point for a fixed budget.

You measure it the way you measure any intervention: pick a held-out slice of your real task, record the base model's score, run a controlled fine-tune, and record the score again. The delta is the headroom, and the only honest version reports it against a baseline. Baseline vanilla LoRA at rank 16 to 32 first, as our LoRA fine-tuning tutorial argues at length, then try anything fancier only if it beats that baseline on identical settings and the same eval. The trap here is measuring headroom in isolation and calling a two-point gain a win when the base you did not test would have given you eight.

The mechanism behind the architecture argument is worth stating, because it is the one place the active-parameter number stops being trivia and starts predicting behavior. When you train a low-rank adapter, the gradient can only route through the experts a token actually activates. A model that fires 6 routed experts plus 2 shared on every token, at 4.2% of 975B, gives your adapter a wider live path to push against than a sparser model that fires a thinner slice of a larger total. Denser activation is not free, it costs decode bandwidth, but for a base that will be adapted rather than run at maximum sparsity, spending that bandwidth on live capacity is the right side of the trade. This is a hypothesis about headroom, not a measurement of it, but it is a hypothesis with a mechanism, which is more than the index offers.

Does Inkling clear it? Two structural facts point the right way. The 4.2% active-parameter path is denser than most frontier open MoEs, which gives an adapter more live capacity to modify per token, and the 45T-token multimodal pretraining base is broad, which usually correlates with cleaner transfer. But headroom is task-specific by definition, so the honest grade is: promising on architecture, unproven until you run your own before-and-after. Nobody, including Thinking Machines, has published your task's number, and the general index cannot stand in for it.

2. Quality retention after fine-tuning

Fine-tuning that lifts your task can quietly wreck everything else. The failure mode is well known: specialize hard on a narrow distribution and the model forgets general instruction-following, reasoning, and safety behavior it had before. For a base you plan to adapt and then serve on real, varied traffic, retention is not optional. A model that scores beautifully on your task and has forgotten how to follow a system prompt is a regression wearing a benchmark medal.

You measure retention by keeping a fixed general-capability probe outside your fine-tuning data and running it before and after: instruction-following, a reasoning set, and whatever safety or format checks your deployment depends on. The number that matters is how much of the base's general score survives your specialization, reported next to your task gain, so you can see the trade instead of only the win. This is the eval-honesty discipline applied to your own training run rather than to a vendor's claim: report the distribution, not the single point that flatters you.

Does Inkling clear it? The architecture gives one genuine reason for cautious optimism and one caveat. The 2 always-on shared experts carry general behavior that per-token routing does not disturb, and a shared-expert design tends to localize task-specific adaptation in the routed path while the shared path holds a stable general prior, which is structurally friendly to retention. The caveat is that Inkling exposes a controllable thinking-effort dial, and reasoning-heavy fine-tuning is exactly the regime where long-range and instruction behavior tends to degrade, so retention has to be measured with the effort setting held fixed across the before and after or the comparison is meaningless. Grade: architecturally favorable, but it lives or dies on your probe, and the probe has to control for the effort dial.

3. Day-0 serving-stack parity

A base you cannot serve is not a base; it is a research artifact. The measurement is blunt: on the day you finish fine-tuning, can your production stack run the adapted model, with your adapters, at your context length, with the throughput features you depend on? Custom architectural pieces are where this breaks. Inkling's kernel-4 convolutions, learned position bias, and local/global attention split all need explicit implementation in whatever engine you serve on, and "the weights are open" says nothing about whether your engine has that code.

You measure parity by checking, against the actual release notes of the engine you run, whether the model is supported, whether multi-adapter serving works, whether the long-context path is implemented, and whether the quantized checkpoint you plan to deploy is a first-class citizen rather than a stub. Then you benchmark it at your real concurrency, because a day-0 support checkbox is not the same as day-0 performance.

Does Inkling clear it? This is the measurement Inkling clears most cleanly, and it is not an accident: Thinking Machines shipped the serving stack alongside the weights. vLLM v0.26.0 landed day-0 Inkling support with full feature parity, including LoRA, tensor/data/expert/pipeline parallelism, prefix caching, and disaggregated serving, and reports up to 380 tokens per second per user on 4x GB200 with MTP speculative decoding. SGLang shipped day-0 support in the same window, reporting up to 71.7k tokens per second of input throughput and 171 tokens per second per-user decode on Blackwell, with multi-LoRA serving and both BF16 and NVFP4 checkpoints supported. Multi-adapter serving on day zero is the specific capability a fine-tuning base needs, and both major open stacks have it. If you are weighing the hardware bill behind those throughput numbers, our self-hosting a frontier open MoE breakdown covers why the 975B total, not the 41B active count, sets the VRAM you have to buy. Grade: cleared, and cleared better than most open releases manage.

4. License and checkpoint format

The last measurement is the one people skip because it has no benchmark, and it is the one that can veto everything above. If the license restricts commercial use or derivative distribution, or if the only checkpoint that ships is a format your training and serving stack cannot ingest, then adapter headroom and retention are moot. You cannot build a customization pipeline on a base you are not allowed to customize, or one you cannot load in the precision you fine-tune and serve at.

You measure this by reading the license against your actual use, commercial and derivative, and by checking which checkpoints exist: is there a full-precision checkpoint to fine-tune from, and a deployment-precision checkpoint to serve, and do they match your pipeline? A model that ships only a quantization-aware 4-bit checkpoint, as Kimi K3 does with its MXFP4-only release, is a different and harder fine-tuning problem than one that gives you a BF16 base to adapt and a separate low-precision checkpoint to deploy, because 4-bit-native weights leave less obvious headroom for the standard QLoRA-over-BF16 recipe our fine-tuning coverage assumes.

Does Inkling clear it? Yes, on both counts, and this is where the "built to be fine-tuned" claim is most substantiated. Apache 2.0 is a genuine commercial-and-derivative license with no field-of-use or scale restrictions, the most permissive tier a frontier-scale open release comes in. And Inkling ships the full stack of checkpoints: BF16 to fine-tune from, MXFP8, and a separate NVFP4 deployment variant, so you can train the adapter in full precision and serve the merged result at 4-bit without owning the quantization step yourself. Compared against the mid-2026 open field, where the strongest models often ship one restrictive license or one take-it-or-leave-it precision, Inkling's license-and-format story is its most decisive advantage. For where this sits in the broader open-weights shift, our open-source LLM power shift piece traces why permissive licensing became the axis that actually moved adoption.

The protocol: is this a good base for my task?

The four measurements collapse into a procedure you can run in a week, before you commit a training budget or a serving contract. Run it against Inkling and against at least one alternative base, because a measurement without a baseline is half a result.

1. Assemble a private eval slice. Pull 200 to 500 real examples from your task, held out from any fine-tuning data. This is your instrument; keep it off the internet so it stays uncontaminated.
2. Record the base line. Score every candidate base zero-shot on the slice, with the thinking-effort setting fixed and logged. This is a ceiling-on-current-knowledge number, not a decision number.
3. Assemble a general-capability probe. Add a fixed, separate set of instruction-following, reasoning, and safety or format checks that your deployment depends on. Score every base on it now, before any training.
4. Run a controlled fine-tune. Baseline vanilla LoRA at rank 16 to 32, identical settings across every candidate, identical data. Do not tune the recipe per model; you are comparing bases, not adapters.
5. Measure headroom. Re-score the task slice. The gain over step 2 is the headroom. A base that starts lower and finishes higher wins over one that starts higher and barely moves.
6. Measure retention. Re-score the general probe from step 3 with the effort setting still fixed. Report the retained fraction next to the task gain. Reject any candidate whose general score collapses, whatever its task gain.
7. Confirm serving parity. Verify, against your engine's release notes, that the adapted model serves with your adapters, at your context length, at your quantization, and then benchmark tokens per second at your real concurrency, not at batch size one.
8. Confirm license and format. Read the license against your commercial and derivative use, and confirm a fine-tunable full-precision checkpoint and a servable deployment checkpoint both exist and fit your pipeline.
9. Decide on cost-per-solved-task. Combine the headroom, the retention, and the measured serving cost into one number: what it costs to get one correct answer on your task at your quality bar. Rank bases on that, never on the index.

The output is a per-base cost-per-solved-task with a retention floor and a serving-parity gate, which is a decision. The Intelligence Index is not.

So does Inkling clear it?

Inkling clears the two measurements you can check on launch day and leaves the two that need your data open, which is the honest and expected shape of a base-model verdict one week after release.

The license and checkpoint measurement is cleared decisively: Apache 2.0 with a BF16 base to fine-tune, an MXFP8 intermediate, and an NVFP4 checkpoint to serve is close to the ideal a customization pipeline wants, and it is strictly better than a restrictive license or a 4-bit-only release. Serving parity is cleared cleanly: day-0 support in both vLLM and SGLang with multi-LoRA, long context, and quantized checkpoints all present, plus Tinker as a first-party fine-tuning path, is more than most open releases deliver in their first month. Those two are real, they are verifiable today, and together they substantiate the narrow, defensible version of the "built to be fine-tuned" claim: the plumbing to fine-tune and deploy Inkling actually exists on day zero.

Adapter headroom and quality retention are the two that decide whether Inkling is a good base for your task specifically, and both are architecturally favorable and empirically unproven. The denser 4.2% active path and the broad 45T-token pretraining argue for headroom; the always-on shared experts argue for retention; neither is a substitute for running steps 5 and 6 of the protocol on your own eval. Anyone who tells you Inkling is or is not a good base without those two numbers is quoting the leaderboard back at you, which is the one axis the model was explicitly not built to win.

That is the useful lesson underneath a single release. Thinking Machines chose to be graded on customizability rather than rank, and the right response is not to check whether it beat Kimi K3 (it did not, and it said so) but to hold it to the scoreboard it chose. On that scoreboard, license and serving are settled, headroom and retention are your homework, and the vendor benchmarks, the SWE-bench Verified figure most of all, are a hypothesis you have no reason to trust until your own before-and-after says otherwise.

Key Takeaways

1. For a model built to be fine-tuned, leaderboard rank is the wrong axis. Inkling shipped at Artificial Analysis index 41, behind Kimi K3's 57, and Thinking Machines said so on purpose. The number you care about is where your adapted model lands, not where the base sits.
2. "A good base for customization" is a marketing sentence until you name the measurement. Four decide it: adapter headroom on your task, quality retention after fine-tuning, day-0 serving-stack parity, and license plus checkpoint format.
3. Inkling clears license and serving decisively. Apache 2.0 with BF16, MXFP8, and NVFP4 checkpoints, plus day-0 vLLM v0.26.0 and SGLang support with multi-LoRA and 1M context, is the most substantiated part of the "built to be fine-tuned" pitch.
4. Headroom and retention are architecturally favorable but unproven. The denser 4.2% active path (vs Kimi K2.5's 3.2%) argues for headroom and the 2 always-on shared experts argue for retention, but both are task-specific and only your own before-and-after eval settles them.
5. Every vendor benchmark is a ceiling, not a fact. The 97.1% AIME, 77.6% SWE-bench Verified, 79.8% IFBench, and 91.4% VoiceBench figures are vendor-measured. The SWE-bench Verified number carries the heaviest discount: flawed tests, scaffold-dependent swings of 30-plus points, and contamination.
6. Measure retention with the thinking-effort dial fixed. Inkling exposes a controllable effort setting, and reasoning-heavy fine-tuning degrades instruction and long-range behavior, so a before-and-after that does not hold effort constant is measuring noise.
7. Run the nine-step protocol against a baseline. Private eval slice, base line, general probe, controlled vanilla-LoRA fine-tune, headroom, retention, serving parity, license and format, then rank on cost-per-solved-task, never on the index.