Multimodal LLMs in Production: What Native Vision Actually Costs

Putting multimodal LLMs in production is now a cost and latency problem, not a capability problem, and most teams discover that in the wrong order. The frontier models (Gemini 3.5, GPT-5.x, Claude Opus 5, and on the open side Qwen3-VL, now folded into the unified Qwen3.5 line) all read images, charts, and documents well enough that the question "can it see" is settled. What is not settled, and what the benchmark leaderboards never show, is what a single image does to your token bill and your time-to-first-token, why two models with nearly identical MMMU-Pro scores behave completely differently on your actual documents, and which modality-specific gap is going to break your use case. This piece is about the part of multimodal that ships, the part after the demo where the image becomes a cost center.

The trap is treating vision as a free upgrade to a text model. It is not free; it is paid for in tokens, and image tokens follow rules that text tokens do not. Understanding that pricing is the difference between a multimodal feature that scales and one whose unit economics quietly fall apart at volume, which is the same inference-cost reality every production LLM system runs into, sharpened by the fact that images are large.

An image is a pile of tokens, and the pile is bigger than you think

The single most important fact about multimodal in production: images are converted to tokens, and a high-resolution image can cost as many tokens as several pages of text. A vision model splits an image into patches, encodes each into the token stream, and the count scales with resolution. A small thumbnail might be a few hundred tokens; a full-page screenshot or a detailed photo at high resolution can run to one or two thousand. Send a handful of those per request and the image content dominates the prompt, dwarfing the text instructions next to it.

This has three consequences that the per-token price tag hides. Cost scales with resolution, not with how much you needed the detail. If your task only needs to read a headline, sending the full-resolution page pays for every patch anyway. Most multimodal APIs expose a detail or resolution setting; using the lowest one that still solves the task is often the largest single cost lever you have. Image tokens eat the context window. A 256K-context model sounds roomy until a few high-resolution pages consume tens of thousands of tokens each, and the effective context problem compounds it, because the usable span is shorter than the advertised one and images fill it fast. Prefill latency scales with image tokens. Time-to-first-token is dominated by processing the prompt, and a prompt thick with image patches is slow to prefill, so the first token on an image-heavy request lands well after a text-only one would. The accuracy benchmarks never measure this, and it is frequently the constraint that decides whether the feature is usable.

The practical discipline is to treat image resolution as a budget knob, downscale aggressively to the lowest resolution that preserves the detail the task needs, and measure tokens-per-request on real inputs before you price the feature. A multimodal feature that pencils out at demo volume can lose money at scale purely because nobody counted the image tokens.

Do the token arithmetic before you price anything

The counting rules are model-specific and worth pinning down for whatever you ship, because the vendors do not tokenize images the same way. Claude uses an area-based rule: an image costs roughly (width x height) / 750 tokens, capped at a per-model visual-token budget that recent releases raised substantially (from around 1,568 to roughly 4,784 tokens) precisely to support higher-resolution reads. OpenAI and Gemini use tile-based schemes: the image is cut into fixed-size patches and each patch is a flat number of tokens, with a base cost on top and a small floor for tiny images. The formulas differ, but the shape is identical, cost climbing with pixels.

Run the arithmetic on a concrete case. A 1024x1024 screenshot on the area-based rule is about 1,048,576 / 750, roughly 1,400 tokens, already the size of a long instruction prompt. A full-page document scan at 1500x2000 is about 3,000,000 / 750, roughly 4,000 tokens, which slams into the visual-token cap and means the model is reading a downscaled version of your page whether you asked for that or not. Send four such pages in one request and you have spent on the order of 16,000 tokens before a single word of your prompt, on every call, forever. That is the line item that turns a demo into a loss at volume, and it never appears on an accuracy leaderboard.

There is a second lever most teams miss: prompt caching. When the same image or document is referenced across many requests (a fixed reference chart, a system diagram, a page a user asks several questions about), caching the prefilled image context amortizes both its token cost and its prefill latency across the whole session instead of paying full price each turn. The inverse is the warning: image-heavy prompts that change every request get no cache benefit and pay the full prefill every time, so a workload's cache hit rate on image context is worth measuring before you commit to an architecture.

The benchmarks saturated, so they stopped discriminating

The headline multimodal benchmark is MMMU and its harder variant MMMU-Pro, and here is the eval-honesty problem with reading the leaderboard: the top models now cluster. The leading frontier multimodal systems all clear the low-to-mid 80s on MMMU-Pro, separated by a couple of points rather than the ten-plus-point gaps of two years ago. When the spread between models is inside the benchmark's own noise, the ranking stops telling you which model is better for your task; it tells you the test saturated. This is the evaluation crisis in its multimodal form: a single aggregate score with a model name next to it is a hypothesis, and on saturated benchmarks it is a weak one.

The differentiation moved off the aggregate and onto the axes, and the axes are where production lives. The current frontier separates not on "multimodal ability" as one number but on specific modalities: video understanding (long-video temporal reasoning remains genuinely hard and is where the gaps are widest), long-document OCR and dense-text extraction, chart and figure reasoning, GUI and screen understanding for agents, and reasoning that combines an image with a long text context. A model can sit at the top of MMMU-Pro and still be middle-of-the-pack on the dense-document OCR your use case actually depends on. The leaderboard rank predicts almost nothing about which of these axes a given model is strong on.

Two axes deserve special attention because they break in opposite ways. Video is where the token bill and the capability gap collide. A model does not watch a video; it samples frames and treats each as an image, so a one-minute clip sampled at even one frame per second is sixty images, and the token math from the previous section multiplies by sixty. That is before the genuinely hard part: long-video temporal reasoning, tracking an entity or a causal chain across minutes, is the least saturated axis in all of multimodal, the one place where models still fail plainly rather than subtly. If your product reasons over video, you are paying the highest token cost on the weakest capability, and the leaderboard number, computed on short clips or single frames, will have told you nothing about either.

Dense-document OCR breaks the other way, on the cost-accuracy tradeoff directly. Reading small text, dense tables, and multi-column layouts is exactly the task that needs high resolution to be reliable, which collides head-on with the pressure to downscale for cost. A model that transcribes a clean invoice perfectly can drop or hallucinate digits on a dense financial statement at the same token budget, and the failure is silent: you get plausible numbers, not an error. For any OCR-heavy pipeline, the resolution setting is not a cost knob you can turn freely; it is a correctness parameter you have to tune against your own worst-case documents.

So the selection method is not "pick the top of the leaderboard." It is to identify the one or two modalities your product actually exercises, build a small eval set from your own data on exactly those, and measure the candidate models on it. A hundred representative examples from your real inputs, scored on the dimension you ship, beat any public benchmark for this decision, because they measure the thing you are buying instead of a saturated average that hides it.

Concretely, that eval is a short loop, not a research project. Pull a hundred real inputs from your actual traffic or a faithful proxy, weighted toward the hard cases you already know exist (the dense pages, the low-light photos, the edge-case charts). Define the score as the decision you ship, exact-match on extracted fields for a document pipeline, or a pass or fail on the specific question for a visual-QA feature, not a vague quality rating. Run every candidate model at the resolution and detail setting you would actually deploy, since a score measured at full resolution is a lie if you plan to downscale in production. Read the failures, not just the aggregate, because the pattern of what each model gets wrong on your data is the thing that decides the choice. The whole loop is a day of work and it replaces a month of leaderboard-watching that would not have answered the question anyway. This is the same evaluation discipline text-only systems need, with resolution added as a variable you control.

Open or API, the multimodal version

The open-weights-versus-API question takes a specific shape for vision, and the economics tilt it. The strongest open vision line, Qwen3-VL, ships in mixture-of-experts configurations (a large 235B-A22B variant and a smaller 30B-A3B) that activate a fraction of their parameters per token, the MoE efficiency that makes self-hosting a capable vision model realistic on owned hardware. The newer Qwen3.5 line folds vision into a single early-fusion backbone rather than shipping a separate VL model, and reports matching or beating the older Qwen3-VL-235B at a fraction of the active parameters, which only sharpens the self-hosting case. Either way, open vision weights are reported as multiple times cheaper per call than the closed frontier via API, and effectively free at the margin once the hardware is paid for.

The call follows the same system-dependent logic as text, with image volume as the new variable. When you process images at steady, high volume (document pipelines, screenshot analysis at scale, anything where the image throughput is the workload), self-hosting an open vision model is often the economical answer, because the per-image API cost that looks small in testing becomes the dominant line item at volume, and owning the model caps it. When vision is an occasional component of a broader product whose value is the UX, the API is the right answer and self-hosting the model is overkill. The deciding factor is image volume and how central it is, not a preference for open or closed, the same pragmatism that governs the text-only decision.

What still breaks

Naming the failure boundary matters more than listing capabilities, because the capabilities demo well and the failures show up in production. Three are worth planning around. Counting and precise spatial reasoning remain weak, even at the frontier; models that ace complex visual question answering still miscount objects and misjudge precise positions, so any task that hinges on an exact count or a precise layout needs verification, not trust. Dense small text is unreliable at low resolution, which collides directly with the cost pressure to downscale, so OCR-heavy tasks force a real tradeoff between token cost and accuracy that you have to measure rather than assume. Hallucination on images is confident and fluent, the same way it is on text: a model will describe a detail that is not in the image with the same tone it uses for one that is, and there is no native confidence signal that separates the two. For anything consequential, the image output needs the same verification discipline as any other model output, because the fluency of a wrong answer is identical to the fluency of a right one.

One more failure is worth planning around because it sits at the seam where most multimodal features actually ship: structured extraction from an image is less reliable than the free-text description that precedes it. Asking a model to "read this receipt" produces fluent prose; asking it to return a strict JSON object with the vendor, date, line items, and total is where the errors concentrate, because now every hallucinated digit and every misread field has to land in a typed slot with no room to hedge. The mitigation is the same one text pipelines use, constrained decoding to force schema-valid output, but constraint only guarantees the shape, not the truth: a schema-valid object with a wrong total is still wrong, and it now looks authoritative. Validate the extracted fields against the image, not just against the schema, for anything that feeds a downstream system.

Multimodal in production is a solved capability wrapped around an unsolved cost-and-reliability engineering problem. The models can see. Whether your feature works at scale depends on counting the image tokens, picking the model on the modality you actually ship rather than the saturated average, choosing open or API by your image volume, and verifying the outputs that matter, because the leaderboard measured none of those and they are the entire job.

Key Takeaways

1. Multimodal is a cost and latency problem, not a capability one. The frontier models all see well. What ships or fails is the token bill, the prefill latency, and the modality-specific gaps the leaderboard hides.
2. An image is a large pile of tokens. A high-resolution image can cost as many tokens as several pages of text, and the count scales with resolution. Image tokens dominate cost, eat the context window, and slow time-to-first-token.
3. Resolution is your biggest cost knob. Downscale to the lowest resolution that preserves the needed detail. Sending full resolution to read a headline pays for every patch anyway. Measure tokens-per-request on real inputs before pricing the feature.
4. MMMU-Pro saturated. The top models cluster within a couple of points, inside the benchmark's noise. The aggregate rank no longer discriminates; build a small eval on your own data, on the modality you ship.
5. Differentiation moved to the axes. Video understanding, dense-document OCR, chart reasoning, and GUI understanding separate the models. A leaderboard leader can be middling on the exact axis your product needs.
6. Open vs API turns on image volume. Qwen3-VL's MoE configs make self-hosting a capable vision model realistic. Steady high image throughput favors self-hosting; occasional vision in a UX product favors the API.
7. Counting, dense small text, and hallucination still break. Models miscount, misread small text at low resolution (colliding with the downscaling cost pressure), and hallucinate image details fluently. Verify any consequential visual output.

The Acing AI newsletter covers multimodal the way this piece does: the token bill and the failure modes, not the benchmark. Subscribe for the production read.