API or Local Models: Where the Cost Break-Even Sits

Compare the monthly cost of the API with the full monthly cost of self-hosting, which includes hardware spread over its life, power and hosting, and, in many cases, the engineering time to run, update, monitor and secure it. Because most self-hosting cost is fixed, self-hosting only wins above a certain volume, provided the local model is good enough and the hardware can serve your peak load. Then check the factors cost cannot capture: quality, privacy, latency, reliability and the effort of staying current. Use your own measured numbers; every figure in this post is an illustrative assumption.

Both options have costs beyond the headline price. Listing them side by side stops the comparison being decided by whichever number is easiest to find.

The numbers below are assumptions chosen to make the arithmetic easy to follow. They are not real prices and not a benchmark. Replace every one with your own measured values, and use the site's API vs Local calculator, which models hardware write-off, operations and power, to do it with your own inputs.

With those assumptions, self-hosting costs about $333 (hardware) plus $60 (power and hosting) plus $1,000 (people), roughly $1,393 a month, regardless of volume. The API costs $2 for every million tokens. Setting them equal:

fixed_local_per_month = 12_000 / 36 + 60 + 1_000      # hardware + power + people
api_cost_per_million_tokens = 2.0                      # assumed

break_even_millions = fixed_local_per_month / api_cost_per_million_tokens
print(f"Local = ${fixed_local_per_month:,.0f}/month fixed")
print(f"Break-even at about {break_even_millions:,.0f} million tokens per month")
# Local = $1,393/month fixed
# Break-even at about 697 million tokens per month

Under these assumed numbers, self-hosting only pays off above roughly 700 million tokens a month. Below that, the API is cheaper even though the hardware is "free". Change one input and the answer moves a lot: halve the engineering time and the break-even falls, double the hardware cost and it rises. That sensitivity is the point of doing the calculation yourself.

A break-even volume is a necessary check, not a sufficient one.

The quality row is the one teams most often skip. If the local model is worse on your task, the savings can be wiped out by wrong answers, rework and lost trust. Measure it with the same evaluation set you use for the API model, as described in LLM evaluation fundamentals.

Some reasons to run locally have little to do with price.

Some teams end up with both: an API for hard or rare requests and a local model for high-volume, well-understood ones, with routing between them. The model router pattern and how to reduce LLM costs describe that approach, and an LLM gateway is where the routing usually lives.

If you do go local, the model has to fit the memory you have. Open-weight models differ widely in size, and techniques such as quantisation can shrink them, sometimes at a cost to quality. The Local Model Advisor lists open-weight models that fit a given GPU or unified memory, with parameter counts sourced from Hugging Face, and the local and open-source models chapter covers the options. Check each model's licence and the quality on your own task before committing.

Is running a local model cheaper than using an API? Only above a break-even volume. Self-hosting has mostly fixed costs (hardware, power, and above all engineering time), while an API charges per token. Below the break-even volume the API is cheaper, even if you already own the hardware.

What are the hidden costs of self-hosting a model? Engineering time to deploy, update, monitor and secure it, plus power and hosting, capacity headroom for peak load, and the cost of errors if the local model is weaker than the API model on your task.

How do I calculate the break-even point? Add up the monthly fixed cost of running locally (hardware spread over its life, power, hosting and people time) and divide by the API cost per million tokens. The result is the monthly volume above which local is cheaper, assuming equal quality and enough capacity.

Should I use the same evaluation set for both? Yes. Run the same task-based evaluation on the API and the local model so you compare quality as well as cost. A cheaper model that is not good enough is not a saving.

When should I run a model locally for non-cost reasons? When data cannot leave your network, when you need offline or very low-latency operation, or when you want control over the model version. These can justify local hosting even when the cost is higher.

Can I use both an API and a local model? Yes, and some teams do. Route hard or rare requests to a hosted model and high-volume, well-understood ones to a local model, usually through a gateway so applications do not need to know which served the request.

How do I pick a local model for my hardware? Check how much GPU or unified memory you have, choose models that fit it, consider quantisation and test quality on your own task. The Local Model Advisor on this site filters open-weight models by the memory you have.

• How to reduce LLM costs by 90 percent without hurting quality
• LLM gateways: what they do and how to choose one
• What an AI platform engineer actually owns
• AI platform engineering topic hub
• AI Platform Engineer roadmap