Model the Unit Economics of an AI Feature

Software instincts do not transfer here. Traditional software has near-zero marginal cost, so pricing can ignore usage and margin improves automatically with scale. An AI feature has real marginal cost per use, which means a flat-priced product can lose money on its most engaged customers and lose more of it the more successful it becomes.

The fix is to model the cost per completed task early, while the design is still changeable.

Cost the task, not the request

A single user-visible outcome usually consumes several model calls, and the components are easy to miss.

Input tokens include the system prompt, retrieved context, conversation history, and the user's text. Retrieved context is frequently the largest single item and the least examined — teams retrieve ten documents because ten was the default.

Output tokens cost more than input on most providers, and reasoning-style models can generate a great deal of output that the user never sees.

Retries and failures are real cost. A task that succeeds on the second attempt costs twice. Whatever your success rate is, divide by it.

Multi-step flows multiply. A plan-act-verify loop is at least three calls, and agent-style flows with tool use can be dozens.

Supporting services — embeddings, vector search, reranking, moderation, transcription — are individually small and collectively significant.

Human review, where a workflow includes it, usually dominates everything above. A minute of a specialist's time can exceed the entire model cost by an order of magnitude, which is worth knowing before optimising tokens.

Measure this from real traffic rather than estimating it. Instrument token counts per completed task and look at the distribution, not the mean — the tail is where the money goes, and the mean hides it entirely.

The levers that actually move the number

Ranked by impact in practice:

Routing by difficulty. Most requests do not need your most capable model. Sending the easy majority to a smaller model and escalating the rest is typically the largest available saving, and it usually costs little quality if the routing is measured.

Trimming context. Retrieving three well-chosen documents instead of ten cuts input tokens substantially and frequently improves quality, because irrelevant context degrades output as well as costing money.

Caching. Repeated system prompts and shared context can often be cached at the provider level. Semantic caching of common queries helps where usage is repetitive.

Bounding output. Long generation is expensive and often unwanted. Constrain length deliberately.

Reducing steps. Every step in an agent loop is a call. Many loops have a step that adds cost and no accuracy, and nobody has checked which.

Improving first-pass success. Retries are pure waste, so a prompt fix that raises success from 70 to 90 percent removes about a fifth of the total cost.

Two things to watch. Provider prices fall regularly, so a cost problem that is currently marginal may resolve itself — but do not architect on that assumption. And every optimisation trades against quality, so each one needs measuring on the same evaluation set rather than accepting on the basis that it should be fine.

Match the pricing model to the cost structure

Flat subscription is simple and familiar, and it inverts at scale: your heaviest users cost the most and pay the same. Viable when usage variance is genuinely low, or with a fair-use limit that is enforced rather than merely stated.

Usage-based aligns price with cost and makes customers anxious, because an unpredictable bill is a purchasing objection independent of the amount.

Credits are the common compromise: predictable spend for the customer, bounded exposure for you, and a natural upgrade path. The cost is explaining what a credit is.

Seat-based with limits works where usage per seat is predictable and is the easiest to sell into organisations.

Outcome-based aligns best with customer value and is hard to operate unless outcomes are unambiguously measurable.

Whatever you choose, model the heavy user explicitly — the customer at the ninety-fifth percentile of usage — and check that they are still profitable. If they are not, either the price or the cost has to change, and it is much easier to change either before launch.

Also budget for the fact that usage grows as a feature gets better. Success increases cost, and a plan that only works at current usage is a plan that fails when the feature works.

Practice

Instrument one feature and collect real per-task cost for a week. Report the median, the ninety-fifth percentile, and the maximum, and count how much of the total is retries.

Then model the ninety-fifth-percentile customer against your current price. If the margin is negative, list the levers in order of expected saving and test the top two on your evaluation set, measuring quality as well as cost.

Finally, write what your margin looks like if usage triples. That is the scenario where an unmodelled cost structure becomes visible, and it is the scenario you are working toward.

Common mistakes

Costing per request. Tasks take several requests, and retries are not free.

Reporting the mean. The tail is where the cost lives and the mean conceals it.

Optimising tokens while a human reviews every output. The human is the cost; find that first.

Flat pricing with high usage variance. Your best customers become your worst margin, and success makes it worse.