The formula
Inference cost per player-hour comes down to three numbers multiplied together:
cost/hour = model calls per hour × tokens per call × price per token
Lifetime cost per player is that times the hours they play. Simple, but every term hides decisions:
- Model calls per hour isn't just dialogue turns. It includes every call your systems make: intent parsing, memory summarisation, NPC-to-NPC simulation, director decisions, safety checks.
- Tokens per call is dominated by input, not output. A character's system prompt, personality, relevant memories and world state can easily reach 2,000–6,000 tokens before the player has said anything.
- Price per token varies by more than 100× between a small self-hosted model and a frontier API, and differs between input, cached input and output tokens.
Blended price assumes cached prompt tokens are billed lower than fresh ones. Self-hosted models convert GPU-hours to an equivalent per-token rate.
With the defaults above (120 calls an hour, 2,500 tokens each, a blended $0.60 per million tokens), a player costs about 18 cents an hour. Twenty-five hours of play is $4.50 per player, fine for a $25 premium game and a serious problem for a free-to-play one.
Realistic ranges by game type
These are rough ranges from titles we've worked on or reviewed, after reasonable optimisation. Your design will differ; use them as a sanity check, not a target.
| Game type | Calls / hour | Tokens / call | Typical cost / player-hour |
|---|---|---|---|
| Conversation-light RPG (occasional NPC chats) | 20–60 | 1,500–3,000 | $0.01–$0.05 |
| Dialogue-centric mystery or narrative | 100–250 | 2,000–5,000 | $0.08–$0.40 |
| Multi-agent strategy (off-screen simulation) | 300–2,000 | 800–2,500 | $0.15–$1.50 |
| Companion / social games | 150–400 | 3,000–8,000 | $0.20–$1.20 |
| On-device small model | any | any | $0 marginal (device cost) |
Multi-agent simulations are where budgets explode. A hundred agents each "thinking" once a minute is 6,000 calls an hour, per player, before you've optimised anything.
Matching cost to business model
Inference cost only matters relative to what a player pays. A useful rule from our live titles: keep lifetime inference cost under 15% of revenue per player for premium games, and under 25% of average revenue per paying user for free-to-play, factoring in that most free players never pay.
- Premium ($15–$40): the risk is your most engaged players. Someone who plays 200 hours at $0.30/hour costs $60, more than they paid. Design the late game to need fewer calls, or cap expensive systems.
- Free-to-play: you're paying for every non-paying player. Start new players on cheaper models and unlock richer characters with engagement or purchase.
- Subscription: the cleanest fit for heavy AI use, because revenue scales with time played.
Seven ways to cut inference cost
Between vertical slice and launch, our median title cuts inference cost by 71%. These are the techniques that do most of the work, roughly in order of impact:
- Model routing. Most turns don't need your biggest model. Route greetings, small talk and simple acknowledgements to a small model; escalate to a larger one for plot-critical conversations or confusing input. A classifier that costs fractions of a cent decides.
- Prompt caching. Put the stable parts of the prompt (character sheet, world rules, tone bible) first and keep them byte-identical across calls. Cached input tokens are typically billed at a fraction of fresh ones, and self-hosted servers can reuse the computed prefix.
- Memory compression. Don't stuff entire conversation histories into context. Summarise older exchanges into short episodic memories and retrieve only the relevant ones.
- Distillation and fine-tuning. Once a character's voice is stable, fine-tune a smaller model on the large model's best outputs. A fine-tuned 8B model often matches a general 70B model inside one character's narrow domain.
- Batching off-screen simulation. Agents the player can't see don't need real-time responses. Batch their decisions, run them less often, and simulate only what could affect the player soon.
- Response caching. Many player inputs are near-duplicates ("hello", "who are you?"). Cache good responses keyed by character, state and normalised input, and vary them lightly.
- On-device models. Small models now run on modern PCs, consoles and phones. Use them for frequent cheap calls and as an offline fallback.
A worked example
Here's a simplified version of a cost model from a dialogue-driven mystery game, from vertical slice to launch.
At vertical slice, every guest response went to a large frontier model. Each call carried a 3,800-token prompt (character sheet, case facts, the last 20 exchanges) and produced about 120 tokens. Players made roughly 140 calls an hour, including the systems' own checks. At a blended rate around $3 per million tokens, that's about $1.65 per player-hour. With a 12-hour average playtime, lifetime cost was nearly $20 per player, for a game planned at $19.99.
The changes, in the order they were made:
| Change | Effect | Cost / player-hour |
|---|---|---|
| Starting point | Frontier model for everything | $1.65 |
| Reorder prompt for caching | ~70% of input tokens cached | $0.78 |
| Summarise history into memories | Prompt down to ~2,200 tokens | $0.49 |
| Route small talk to a small model | ~60% of calls to cheaper model | $0.24 |
| Fine-tuned mid-size model for guests | Frontier model only for accusations | $0.09 |
Lifetime cost fell from about $20 to about $1.10 per player, and playtest ratings for "the guests feel real" didn't change. The biggest single gain came from the cheapest change: putting stable text first so it could be cached.
Latency is a cost too
Players notice a pause after about 400–700 milliseconds in conversation, and earlier in action games. Some of the cheapest optimisations also improve latency (smaller models, caching); others make it worse (cross-region calls to a cheaper provider). Budget for both:
- Stream responses so the first words appear quickly, and drive animation or barks while the rest arrives.
- Serve from regions near your players. A 150ms round trip to another continent eats a third of your budget before the model starts.
- Pre-generate likely next lines while the player reads, and discard what isn't used. It costs tokens but buys perceived speed where it matters most.
Putting it in your pitch
A publisher will want to see one slide on inference economics. Include: your current cost per player-hour, the assumptions behind it, the expected playtime, the resulting lifetime cost as a percentage of price, and your optimisation plan with target numbers. It instantly separates you from pitches that hand-wave this, which, in our experience, is most of them.
At Ammighty we host inference for published titles on Ammighty Runtime and cover the cost for 24 months after launch, outside recoup. It doesn't remove the need for good unit economics, but it gives you room to find the right balance with real players. More in our deal terms guide.
Frequently asked questions
How much does an LLM NPC cost per player?
Typically between $0.01 and $1.50 per player-hour depending on how often the model is called, how much context each call carries, and which model you use. Multiply by expected hours played for lifetime cost.
Are on-device models good enough for game NPCs?
For narrow, well-defined characters, often yes, especially after fine-tuning. They're a strong choice for frequent cheap calls and offline play, with a server model for complex moments.
What is prompt caching and why does it matter for games?
Prompt caching reuses the processed form of a prompt prefix that doesn't change between calls, like a character sheet. Cached input tokens are much cheaper and faster, and character prompts are highly cacheable.
Should inference costs be included in my development budget?
No. Inference is a live operating cost that scales with players. Model it separately and agree with your publisher who pays it after launch.
How do I lower AI NPC latency?
Stream responses, use smaller models for simple turns, cache stable prompts, serve from nearby regions, and cover the gap with animation or authored barks.