Decide Between Buying, Prompting, Retrieving, and Fine-Tuning

Teams reach for training far too early. The reasoning feels sound — our domain is unusual, general models will not understand it — and it is usually wrong, because the same conclusion was available for a fraction of the cost one rung down the ladder.

The discipline is to start at the cheapest option that could plausibly work, prove it insufficient with evidence, and only then climb.

The ladder, cheapest first

Buy a product. If a vendor already solves this and it is not your differentiation, buying is usually correct and teams resist it for reasons of pride rather than economics.

Prompt a hosted model. Days of work, no infrastructure, and it clears a surprising share of requirements outright. This is the baseline every other option must beat.

Add retrieval. When the gap is that the model lacks your specific knowledge, retrieval supplies it at query time. Weeks of work, and it has a property fine-tuning does not: update a document and the behaviour changes immediately.

Fine-tune. When the gap is consistent behaviour — format, tone, a specialised task — rather than knowledge. Months including the data work, which is most of the cost and the part that is always underestimated.

Train from scratch. Almost never correct outside a research organisation or a genuinely unusual modality.

The instruction that saves the most money: before climbing a rung, write down what specifically the current rung failed at, with examples. Teams that cannot produce that document usually discover the failure was a prompt problem or an evaluation problem.

Is this actually your differentiation?

Four questions, and it takes two yes answers to justify building.

Would a customer choose you because of this? Not "is it useful" — is it a reason to pick you over an alternative.

Do you have an asset nobody else has? Proprietary data, a distribution advantage, domain expertise that is hard to acquire. Without one of these, whatever you build is reproducible by anyone.

Will it still matter in two years? Capabilities that were differentiating are routinely absorbed into base models. Building on the assumption that a gap persists is a bet, and it should be a conscious one.

Can you maintain it? A fine-tuned model needs re-tuning as base models improve, evaluation as data drifts, and someone who owns it. A capability nobody maintains degrades into a liability.

Most infrastructure is not differentiation. The workflow, the interface, the trust design, and the integration with how customers actually work usually are — and they are also where teams underinvest while building the model layer.

What fine-tuning does and does not fix

Worth stating plainly, because the misconception is expensive.

Fine-tuning is good at format and structure, at a consistent voice, at a narrow repeated task, and at reducing prompt length for something you do at very high volume.

Fine-tuning is bad at adding knowledge reliably — the model will produce things that sound like your documents rather than reproducing their content, and it will still fabricate. It does not keep facts current, since anything you tune in is frozen at tuning time. It does not fix retrieval problems, and it does not fix an evaluation problem, which is the diagnosis in a surprising number of cases where a team believes their model is underperforming.

The single most common error is fine-tuning to inject knowledge that should have been retrieved. It costs months and produces a model that is confidently wrong in a more domain-specific accent.

Count the switching cost before committing

Every rung up the ladder increases what you would lose by changing direction.

Prompts are nearly free to move. Retrieval systems are mostly portable if the embedding layer is kept swappable — and it usually is not, which is worth deciding deliberately. A fine-tuned model does not move at all: it is specific to a base model and a provider, and when that base model is deprecated you re-tune.

Two things reduce the exposure without giving up much. Put an abstraction between your product and the provider, thin enough to be worth maintaining, so that swapping is an afternoon rather than a rewrite. And keep your evaluation set portable and provider-independent, because that set is the thing that lets you assess a replacement quickly. An evaluation harness is a more durable asset than any model artifact.

Rate of change matters in this calculation. A capability that improves quickly in base models argues for staying low on the ladder and re-testing periodically, since the work you avoid may simply arrive for free.

Practice

Take a capability you are planning to build. Write what the prompted baseline actually achieves, measured on a real evaluation set rather than estimated. Then write the specific gap, with three examples.

Run the four differentiation questions honestly. If fewer than two are yes, the decision is buy or prompt, and the discussion is over.

If you are still climbing, write the switching cost — what you would lose if the provider changed terms, deprecated the model, or a base model absorbed the capability next quarter. Making that number explicit changes a surprising number of decisions.

Common mistakes

Fine-tuning to add knowledge. Retrieval does this better, faster, and stays current.

Building infrastructure as differentiation. Customers choose you for the workflow, not the layer underneath it.

Skipping the prompted baseline. You lose the only evidence that climbing was necessary.

Ignoring maintenance. A fine-tuned model with no owner becomes a liability within two base-model generations.