2026-10-06 · 8 min read
LLM Cost & ROI: How to Estimate Before You Build
A simple method to estimate the running cost of an LLM feature, the levers that cut it, and how to calculate payback against the work it replaces.
Model pricing pages make LLM costs look trivial or terrifying depending on how you read them. A back-of-envelope estimate takes ten minutes and is usually accurate to within a factor of two — good enough to decide whether a use case is worth piloting.
Step 1: Estimate tokens per task
Input tokens: instructions + retrieved context + the user's input. Long documents dominate this.
Output tokens: the length of the answer or extracted data. Output usually costs several times more per token than input.
Calls per task: agents and multi-step pipelines make several model calls per task — count them all.
Step 2: Multiply out
Monthly model cost = tasks per month × calls per task × (input tokens × input price + output tokens × output price). Use the provider's current price sheet for the models you are considering, and add 20–30% for retries and growth.
Step 3: Add the rest of the running cost
Hosting for the application, vector store and pipelines.
Observability and evaluation tooling.
Human review time for the cases the system escalates.
Maintenance: prompt and model updates, new edge cases, integration upkeep.
The levers that cut cost
Model routing: send simple tasks to a small, fast model and reserve frontier models for hard cases.
Prompt caching: cache long, repeated instructions or reference documents to reduce input cost and latency.
Better retrieval: send the five best passages, not fifty mediocre ones.
Batch processing: use batch APIs for work that does not need an instant answer.
Shorter outputs: ask for structured fields instead of prose when a system is the reader.
Step 4: Calculate ROI
Value per month = tasks per month × time saved per task × fully loaded hourly cost — plus any measurable gains in revenue, error reduction or speed. Payback period = one-off build cost ÷ (monthly value − monthly running cost). Use conservative numbers: assume a share of tasks still need human handling, and measure the real figure during the pilot.
Rules of thumb
For most business workflows, model cost is small compared with the human time saved; integration and change management are the real costs.
If the estimate only works with optimistic accuracy assumptions, pilot first and measure.
Re-check pricing yearly: model prices have generally fallen over time, so a use case that is marginal today may not be next year.