Frame an AI Product Opportunity

Start from the work, not the technology

Most AI product failures are decided before any model is chosen. They are decided in the first meeting, when the framing is "we need a chatbot for support" rather than "support agents spend eleven minutes per ticket looking up entitlement rules across four systems."

The first framing has already selected an interface, an interaction model, and a failure mode. The second describes an observable friction and leaves the solution open — and the solution might turn out to be a better search index, which would be cheaper, faster, and more reliable than anything with a model in it.

Describe the workflow using the five parts above, and be concrete about each. Interview two people: the one who performs the work, and the one affected by it. They will disagree, and the disagreement is informative.

The last row carries the most weight. The cost of being wrong determines the product more than the potential upside does. Two workflows can look identical in every other respect and require completely different designs because one produces errors that are noticed in seconds and one produces errors that surface next quarter in someone else's reconciliation.

Write the non-AI baseline first

Before comparing models, write down at least one credible alternative that involves no model at all:

| Alternative | When it wins | | --- | --- | | Better search or filtering | The information exists; people cannot find it fast enough | | Explicit rules | The decision is genuinely deterministic and someone can state it | | Form or interface redesign | The friction is data entry or navigation, not judgement | | Conventional automation | The steps are fixed and the inputs are structured | | Training or documentation | The people are new, not the problem | | An extra review step | The error rate is the problem and throughput is not |

This is not a rhetorical exercise. A meaningful share of proposed AI features are solved better by the second row, and discovering that costs an afternoon rather than two quarters.

AI must earn its additional uncertainty, latency, and operating cost. The honest comparison is not "AI versus nothing." It is "AI versus the best thing we could build for the same effort," and the baseline is frequently competitive.

Choose the role deliberately

Once AI is genuinely the right tool, the next decision is which of seven roles it plays. This is the highest-leverage product decision you will make, and it is usually made by accident.

Reading down the ladder, each rung transfers more authority from the person to the system. The examples are all drawn from the same support workflow deliberately: the same underlying capability can be shipped at seven very different risk levels.

Two observations matter here.

The bottom rung is a different product. Drafting a refund and issuing one share a model, a prompt, and a codebase, and share almost nothing in terms of evidence requirements, review process, audit obligations, or what happens on a bad day. Teams routinely ship "draft" and then quietly add an auto-send toggle, which is a new product launch wearing the clothes of a feature flag.

Higher rungs are not more valuable. Retrieval — the top rung, the least glamorous — frequently captures most of the available value, because the actual bottleneck was finding the policy, not writing the sentence. Start at the top of the ladder and move down only when you have evidence that the rung above is insufficient.

Gate the pilot

A concept earns a pilot by passing four gates, and the third is the one that gets negotiated away.

Quality must be measurable without asking the model. If your only way to know whether an output is good is to ask another model, or to ask the user whether they liked it, you cannot run a meaningful pilot. You need a rubric a human can apply consistently, and preferably an outcome that is recorded anyway — a correction, an escalation, a reversal, a completed task.

The stop condition must be written before you start. A pilot without a pre-agreed failure threshold does not end; it accumulates explanations. Write it as a sentence with a number and a date:

Pilot: draft-reply assistant, tier-1 billing tickets, 20 agents, 6 weeks.

Proceed if:  median edit distance < 30% AND agent adoption > 60% in week 6
Stop if:     harmful-error rate > 0.5% at any weekly check
             OR adoption < 25% at week 4
             OR median handling time increases vs. control
Decide on:   14 November. Owner: R. Mehta.

Note that this includes a way to lose that is not about quality. A feature that produces good drafts nobody uses has failed, and without the adoption line it would be reported as a success.

Harm must be reversible during the pilot. Whatever the eventual design, the pilot version should not be able to produce an outcome you cannot undo. This constraint is temporary and worth accepting.

Practice: the one-page brief

Write a single page containing: the user and their job, the current workflow in the five parts above, evidence that the pain is real, at least one non-AI baseline with a rough cost, the proposed rung on the role ladder, what the human is responsible for, required data and permissions, one quality metric, one business metric, one risk metric, the pilot group, and the stop condition.

Then hand it to a skeptical domain expert — someone who does the work — and ask them to mark every sentence that contains an assumption. Expect a lot of marks. The two most common are that the data is representative and that the person receiving the output has time to check it.

Common mistake

The most common mistake is choosing chat as the interface by default.

Chat is a general-purpose interface, which means it makes no commitment about what the system can do. That is exactly its appeal in a demo and exactly its problem in production: users cannot form an accurate mental model of the capability, so they ask for things it cannot do, and every failure teaches them the tool is unreliable rather than that the request was out of scope.

A narrower interface — a button that says "draft a reply," a panel that shows the three relevant policies — sets an accurate expectation, produces evaluable interactions, and is usually faster for the user than typing. Choose chat when the range of user intents is genuinely open. Most workflows are not.