Run It Locally

Planning estimates for the hardware requirements and cost of running open-weight artificial intelligence models on locally owned equipment.

Impact Calculator · Cost-of-Service Model · ← mmurr.ai

This page addresses three questions, in order: first, whether a given model can be run on available hardware at all (its minimum hardware requirement); second, the cost of doing so, accounting for regional variation in hardware and electricity prices; and third, the practical operational considerations that planning guides frequently omit. The figures presented are planning estimates, derived from published architecture specifications and community-sourced hardware guides. They have not been independently measured by mmurr.ai. Data verified as of —.

Glossary of terms used on this page

GPU (graphics processing unit): A specialised computer chip, originally designed for rendering images, that is also well suited to the mathematical operations artificial intelligence models require. Running a model locally requires one or more GPUs.
VRAM (video random-access memory): The memory built into a graphics card. A model's parameters must fit within the combined VRAM of the GPU or GPUs used to run it, or the model cannot be loaded.
Parameter: One of the numerical values that define a model's behaviour. Larger models — those with more parameters — are generally more capable, but require more memory and computation to run.
Quantisation and "Q4": A technique that reduces the numerical precision used to store each parameter, in order to reduce the memory required to load and run a model, at a small and generally acceptable cost to output quality. Parameters are ordinarily stored at 16-bit precision; "Q4" denotes 4-bit quantisation, which reduces the memory footprint by roughly a factor of four. Every figure on this page assumes Q4 quantisation.
Mixture-of-Experts (MoE) and active parameters: An architecture in which a model is divided into specialised sub-networks ("experts"), only some of which are used to process any given request. The "active parameter" count is the portion actually used per request, and is what determines hardware requirements — the total parameter count, on its own, overstates the hardware a Mixture-of-Experts model needs.
Capital cost (capex) and operating cost (opex): Capital cost is the upfront cost of purchasing hardware. Operating cost is the ongoing cost of running it, principally electricity. This page reports both separately, since they behave differently.
API (application programming interface): The mechanism by which software requests a response from a provider's hosted service over the internet, as an alternative to running a model on local hardware.

Select a Model

Open-weight model

For models using a Mixture-of-Experts architecture, the total parameter count and the active parameter count differ; hardware requirements are determined by the active count (see Glossary, above). The Q4 weight size shown is the amount of storage and graphics memory (VRAM) required to load the model at 4-bit quantisation — the reduced-precision format assumed throughout this page. Quantisation reduces the memory required to run a model, typically by a factor of roughly four relative to its unquantised form, at a small and generally acceptable cost to output quality.

Hardware Requirements

The diagram below indicates the minimum tier of hardware required to load and run the selected model. A minimum hardware requirement means the model will load and produce output — it does not mean the hardware will do so quickly, or that it can serve more than one person at a time. See "Operational Considerations", below.

Tier	Representative hardware	Physical space required	Typical power draw
A	None — the request is sent to a cloud-hosted provider over the internet	None	None
B	One consumer graphics card, such as an NVIDIA GeForce RTX 4090 or RTX 5090 (24–32GB of VRAM)	Fits inside a standard desktop computer	350–575 W — a standard household power socket is normally sufficient
C	Two to four data-centre-grade accelerators, such as the NVIDIA A100 or H100 (80GB of VRAM each)	A dedicated server chassis, typically 2–4U of rack space (1 rack unit, "U", equals 44.45mm)	2,000–4,000 W — a dedicated electrical circuit and active cooling are normally required
D	Four to eight NVIDIA H100 (80GB) accelerators	Half of a server rack to a full server rack	4,000–8,000 W — a three-phase electrical supply and computer-room-grade cooling are normally required
E	More than eight accelerators, connected across multiple physical servers	Multiple server racks in a dedicated server room or data centre	Above 8,000 W, with industrial cooling

Cost by Region

Compare against

Select a specific hosted model — for example, Anthropic Opus or OpenAI GPT-5.5 — to compare its cost against self-hosting, at the same monthly token volume. The default option is a generic estimate of hosted-service cost, drawn from the Cost-of-Service Model.

Utilisation — hours per day under load

Prices are sourced by hand and dated, not retrieved live — see Methods and Sources, below.

Operational considerations

Serving software. Running a model requires dedicated serving software, such as vLLM or SGLang (open-source programs designed to run large language models efficiently). The maximum context length — the amount of text the model can consider at once — is limited by available memory, not merely by a software setting. For example, Kimi K3's advertised 1,000,000-token context window is, in practice, limited to approximately 128,000–256,000 tokens on typical local hardware, unless the intermediate calculation data (known as the "key-value cache", or "KV cache") is distributed across multiple graphics cards.
Concurrent users. The ability to load a model does not imply the ability to serve multiple people simultaneously. Tier B hardware comfortably supports a single user. Supporting several people at once without requests queuing typically requires Tier D or Tier E hardware.
Storage requirements. The model weights alone require substantial storage — for example, approximately 594 gigabytes for Kimi K3 at Q4 precision. Additional storage should be allowed for checkpoints (saved copies of the model) and for experimenting with alternative quantisation levels.
Ongoing maintenance. Running a model locally requires periodic maintenance, including updates to graphics-card drivers and to CUDA (NVIDIA's software platform for running computations on its GPUs), and occasionally re-processing ("re-quantising") the model as improved community-produced versions become available. GGUF is a common file format used for these community-produced, quantised models.
Licensing. The terms under which a model may be used vary by provider and should be read directly from the official source repository. The licences referenced on this page — including Modified MIT, Apache 2.0, MIT, and Meta's custom Llama licence — differ materially, particularly for commercial use at scale.
Hybrid deployment. Many organisations run sensitive workloads on local hardware while using a hosted API for occasional tasks that require the largest models or the longest context windows. This hybrid approach is frequently the most practical outcome.

Methods and Sources

Sourcing hierarchy: the provider's release announcement or model card is used for parameters and release dates; Hugging Face's config.json is used to confirm architecture; community hardware guides are used for real-world VRAM requirements; national energy statistics are used for electricity prices. Every model links to its Hugging Face source. Enterprise-tier GPU price and power draw are read from the Cost-of-Service Model's figures rather than sourced a second time.

Every figure is an editable value in js/data/local-models.js. Data anchors as of —. These are planning estimates and should be verified independently before being relied upon. This page performs all calculations in the browser; no data is transmitted over the network.