AI Cost-of-Service Model

What it actually costs to serve a token — six cost layers from silicon to R&D — and how that compares to the price vendors charge. A first-principles estimate, every input a slider.

Impact Calculator · Handbook · ← mmurr.ai

What it costs a vendor to serve one token — built up from silicon to R&D — compared against what they charge. This is the fully-loaded cost underneath the API price on the Impact Calculator: physical serving here is the same quantity as its "token cost". Every number is an editable estimate, not a disclosure. Data anchors as of —.

Model economics — cost to serve

No lab discloses this — it swings the $ total ~10×. For scale: OpenAI's reported ~1B messages/day × ~1,500 tokens/message, over an ~18-month model life, lands almost exactly on the default (8×10¹⁴) — that's where it comes from. A mid-size product copilot (1M users × 20 msgs/day, 1yr) lands nearer 10¹³, the low end of the slider.

Also undisclosed. Epoch AI's cost-split finding (R&D staff = 29–49% of total dev spend) implies the shipped run is roughly a third of total spend — R&D ≈2× the run itself, the default. A lean lab with few failed attempts sits nearer 1×; a frontier lab running several large parallel attempts, nearer 4×.

Advanced inputs
Physical serving—
Soft (training + R&D)—
Fully-loaded / 1M tok—
API list price / 1M tok—
Gross margin—
Cost logic

Cost over time — the efficiency curve

Fully-loaded and physical-only cost per 1M tokens across the year baselines, at the current slider settings. The unit cost falls several-fold as throughput, batching and silicon improve — directionally tracking the experience curve.

Fully-loaded $/1M Physical serving $/1M

Whole company · per year

Top-down from a fleet's average IT power (MW): annualised cost of running the fleet that exists, plus its CO₂e and water. Pick a lab:

Estimates, not disclosures. The MW and R&D figures are order-of-magnitude estimates back-solved to be plausible against disclosed capex — no lab publishes a served-token denominator or a clean location-based fleet average. And cash ≠ cost of service: the total below is the annualised accrual cost of the existing fleet, not the disclosed cash capex (mostly new capacity — the buildout gap).

Methods & sources

The six cost layers & the physical chain

Root everything in accelerator-seconds; the rest is unit conversions + accounting. Per 1M tokens:

FLOPs/token = 2 · N (N = ACTIVE params, MoE) GPU-hours/1M = g = 1e6 / (r · 3600) (r folds in batching / MFU / bandwidth) Energy/1M = g · P_g/1000 · (1+host) · PUE (kWh — power-in = heat-out) Carbon/1M = Energy · grid Water/1M = Energy · (w_on + w_off) silicon = g · cap / (life · 8760 · util) facility = g · dcW·P_fac / (12 · 8760 · util) energy = Energy · $/kWh network = 0.10 · (silicon+facility+energy) training = C_tr / T_s · 1e6 rnd = training · m TOTAL = silicon + facility + energy + network + training + rnd

Physical serving = silicon + facility + energy + network — the same quantity as the "token cost" on the Impact Calculator. Soft = training + R&D, dominated by the two undisclosed inputs. Onsite-water ceiling is latent heat: rejecting 1 kWh by evaporation needs ≈1.48 L (hfg ≈ 2.44 MJ/kg); real WUE 0.2–1.9 reflects how much heat is rejected evaporatively.

Physical chain is robust; soft-cost and per-company inputs are explicitly estimates. Every value is an editable anchor in js/cost-model.js. Compiled from public sources + first-principles derivation — verify before load-bearing use. Runs entirely in your browser; no network calls.