Chaining Prompts Across Multiple Calls

The intuitive approach to complex tasks is to write one giant prompt that tries to do everything in a single call: "Analyze this document, extract insights, create an outline, and draft a report." This mega-prompt often fails because it overwhelms the model. A more reliable approach is prompt chaining — breaking the task into smaller, independently testable steps, where each step's output feeds the next step's input. This lesson teaches you when to chain and how to do it well.

When to Chain vs. When to Use a Single Prompt

Single Prompt (Monolithic)

A single prompt is simpler and cheaper (one API call). Use it when:

1. The task is straightforward and fits in 1–2 sentences.
2. No intermediate validation is needed.
3. Speed is critical (fewer round-trips = lower latency).
4. The task does not require iterative refinement.

Example: Simple classification

Classify this customer email as "complaint", "praise", "question", or "other".
Email: "Your product broke after one week. I want a refund."
Classification:

Single call, ~100ms latency, done.

Chaining (Multi-Step)

Chain prompts when:

1. The task has clear intermediate stages (extract → transform → validate).
2. You need to debug or test each stage independently.
3. Later stages depend on earlier outputs being correct.
4. The task is complex enough to benefit from focusing on one sub-goal at a time.
5. You can afford slightly higher latency (multiple API calls) for better reliability.

The key insight: chaining is not just about splitting up work. It is about breaking a hard problem into easy pieces, where each piece has a clear input, clear output, and clear success criteria.

Example: Contract analysis

Step 1: Extract key clauses (liability, indemnification, termination).
Step 2: Summarize each clause in business language.
Step 3: Identify risks and trade-offs.
Step 4: Recommend negotiation points.

Each step is independently testable. Step 2 depends on Step 1's output. If Step 1 extracts the wrong clauses, Step 2–4 will be wrong. By chaining, you can validate Step 1 before proceeding.

The Cost-Latency-Maintainability Trade-off

Single Mega-Prompt

Latency: 1 round-trip = ~100–500ms (fastest). Cost: 1 API call (baseline). Maintainability: Poor. If it breaks, you don't know which stage failed. Debugging requires re-running the entire prompt. Reliability: 60–75% for complex tasks. Often produces partial output or misaligned results. Typical failure: Task requests 5 things; model does 3 and skips the others.

3-Step Chain

Latency: 3 round-trips = ~500–1,500ms (slower). Cost: 3 API calls (3× number of calls, but not necessarily 3× cost). Maintainability: High. Each step is independently debuggable. If step 2 fails, you run just step 2 in isolation. Reliability: 85–92% for the same complex tasks. Each step is focused and validated. Typical behavior: Step 1 produces output A. If A is invalid, step 1 is retried. If A is valid, step 2 proceeds with certainty.

Detailed Cost Comparison

Using Claude 3.5 Sonnet pricing (illustrative, July 2026 rates):

• Input: $0.003 per 1K tokens
• Output: $0.006 per 1K tokens

Single mega-prompt (complex document analysis):

Input: 2,000 tokens (system + document + question)
Output: 1,000 tokens (full analysis)
Cost: (2,000 × $0.003) + (1,000 × $0.006) = $0.006 + $0.006 = $0.012
Failure rate: 25% (model misses some analysis points)
Effective cost per successful run: $0.012 / 0.75 = $0.016

3-step chain (decomposed):

Step 1 (extract key facts):
  Input: 500 tokens | Output: 300 tokens | Cost: ($0.0015 + $0.0018) = $0.0033

Step 2 (analyze each fact):
  Input: 600 tokens (facts + analysis prompt) | Output: 400 tokens | Cost: ($0.0018 + $0.0024) = $0.0042

Step 3 (synthesize):
  Input: 800 tokens (all outputs so far) | Output: 300 tokens | Cost: ($0.0024 + $0.0018) = $0.0042

Total cost: $0.0117
Failure rate: 8% (each step validated; retries are cheap)
Effective cost per successful run: $0.0117 / 0.92 = $0.0127

RESULT: Similar cost, but 3-step chain is more reliable.

Decision rule:

• Chain if: Task is complex, reliability matters, or debugging is costly.
• Single prompt if: Task is simple, latency is critical (<200ms), or you're okay with partial results.

For most backend/batch tasks, chaining is worth it. For latency-critical user-facing APIs, use a single prompt optimized for the task.

Designing a Chain: Core Principles

1. Each Step Has a Single, Clear Goal

✗ STEP 1 (BAD): Extract, summarize, and identify risks in this contract.
✓ STEP 1 (GOOD): Extract the key clauses from this contract.

2. Each Step Specifies Input and Output Format

STEP 1: Extract Key Clauses
INPUT: Contract text (arbitrary length)
OUTPUT: JSON array of {clause_name, clause_text, line_numbers}

3. Later Steps Are Validated Before Proceeding

step1_result = extract_clauses(contract)

# Validate: Did we get the expected structure?
if not isinstance(step1_result, list) or len(step1_result) == 0:
    raise ValueError("Step 1 failed: no clauses extracted")

# Proceed to step 2
step2_result = analyze_clauses(step1_result)

4. Error Handling: Retry or Escalate

try:
    result = chain_step(input_data)
except JSONDecodeError:
    # Step output was not valid JSON; retry
    result = chain_step(input_data, retry=True)
except ValueError:
    # Step failed; escalate to human
    notify_team("Chain step failed", step_number, error)
    return {"status": "escalated"}

Worked Example: Blog Post Generation Chain

Task: Given a topic, generate a full blog post (title, outline, draft sections, final post).

Single Mega-Prompt (Bad)

Write a complete blog post about "Prompt Engineering Best Practices".
Include: engaging title, 3-section outline, full drafted content (800–1000 words),
and a conclusion. Make it informative and readable.

Problems: The model tries to do everything at once. The title might not match the content. The outline might not reflect the sections actually written. Quality is inconsistent.

3-Step Chain (Better)

Step 1: Brainstorm and Outline

Topic: "Prompt Engineering Best Practices"

Generate a blog post outline with:
- A compelling title (should hook the reader)
- 3 main sections (each with a sub-heading and 2–3 bullet points)
- A conclusion direction (1 sentence)

Output as JSON:
{
  "title": "...",
  "sections": [
    {"heading": "...", "points": ["...", "..."]},
    ...
  ],
  "conclusion": "..."
}

Output:

{
  "title": "Stop Winging It: A Practical Guide to Prompt Engineering",
  "sections": [
    {
      "heading": "The Three Pillars of Effective Prompts",
      "points": [
        "Clarity: Be specific about what you want the model to do.",
        "Context: Provide examples or reference material.",
        "Constraints: Set length, format, or quality limits."
      ]
    },
    {
      "heading": "Common Failure Modes and How to Avoid Them",
      "points": [
        "Over-relying on persona prompting without grounding in facts.",
        "Exceeding context window limits.",
        "Trusting unvalidated outputs."
      ]
    },
    {
      "heading": "From Prompt to Production: Testing and Iteration",
      "points": [
        "Build a test set of representative examples.",
        "Score outputs against rubrics.",
        "Iterate prompt until you hit targets."
      ]
    }
  ],
  "conclusion": "Prompt engineering is a skill, not magic — practice and validation are everything."
}

Step 2: Draft Each Section

You are a technical blog writer.
Title: "Stop Winging It: A Practical Guide to Prompt Engineering"
Current section to write: "The Three Pillars of Effective Prompts"
Sub-points to cover:
- Clarity: Be specific about what you want the model to do.
- Context: Provide examples or reference material.
- Constraints: Set length, format, or quality limits.

Write this section (200–300 words). Be practical, include a small code example or
screenshot reference. Maintain a conversational but professional tone.

Section text:

Output:

The Three Pillars of Effective Prompts

Writing an effective prompt is like building a house: without a solid foundation,
it falls apart. Three pillars support every great prompt.

First, clarity. Tell the model exactly what to do, not approximately. Instead of
"Write something about Python," try "Write a 200-word explanation of Python list
comprehensions for someone who knows basic Python syntax but has never seen list
comprehensions." The specificity guides the model toward your actual need.

Second, context. If you want an analysis, provide examples of the *kind* of analysis
you want. If you want code, show a working example of your preferred style. The model
learns by example faster than by description.

Third, constraints. Models produce their longest, most confident answer by default.
You control quality by constraining: maximum length, required format (JSON, bullet
points, code), confidence threshold. A constraint like "Flag anything you are unsure
about" prevents overconfident wrong answers.

Together, these three make the difference between a prompt that fails and a prompt
that ships.

Repeat for sections 2 and 3.

Step 3: Assemble and Polish

You are editing a blog post for coherence and flow.

Title: "Stop Winging It: A Practical Guide to Prompt Engineering"

Draft sections:
[Full text from Step 1, Section 1 draft]
[Full text from Step 1, Section 2 draft]
[Full text from Step 1, Section 3 draft]

Polish the draft:
1. Ensure consistent tone (conversational but professional).
2. Add one sentence between sections that bridges ideas.
3. Rewrite the opening (first 2 sentences) to hook the reader.
4. Add a 3-sentence conclusion that ties everything together.

Final polished post:

Output: Fully assembled, polished blog post.

Why This Chain Works

• Step 1 (Outline): Is testable independently. You can read the outline and say "yes, this is the right structure" or "no, let's add a section on X."
• Step 2 (Drafting): Takes the outline as input. Each section is independently drafted, so you can rewrite one section without redoing others.
• Step 3 (Polishing): Sees all sections together, so it can improve flow and coherence.

If Step 1 outline is bad, you catch it before investing in drafting. If one section is weak, you can re-run Step 2 for just that section. Maintainability is much higher than a single mega-prompt.

Looping and Retry Patterns

Validation Loop

max_retries = 3
for attempt in range(max_retries):
    step_output = run_step(input_data)

    try:
        # Validate output structure
        parsed = validate_step_output(step_output)
        break  # Success
    except ValidationError as e:
        if attempt == max_retries - 1:
            raise  # Give up
        else:
            # Retry with tighter constraints
            input_data['constraint'] = "Ensure output is valid JSON"
            continue

Early Stopping

if user_feedback == "this is good enough":
    return early_result
else:
    # Proceed to next step
    continue_chain()

Branching

if "risk: high" in step1_output:
    # High-risk path; add extra validation
    step2_result = detailed_analysis(step1_output)
else:
    # Low-risk path; quick assessment
    step2_result = quick_summary(step1_output)

When Chaining Pays Off: Cost-Benefit Analysis

Consider a realistic scenario: customer support ticket analysis.

Mega-prompt (1 call):

Analyze this support ticket. Extract: issue category, sentiment, urgency.
Identify any security risks. Suggest a response. Estimate resolution time.
Output as JSON.

Ticket: [text...]

Result: Often incomplete, garbled JSON, hallucinated fields.

3-step chain:

Step 1: Categorize (customer service, technical, billing, other).
Step 2: Assess sentiment and urgency (1–5 scale).
Step 3: Recommend action (response, escalation, or self-service link).

Cost comparison (illustrative estimate):

• Mega-prompt: ~1,500 input tokens + 400 output tokens = $0.022 (assume $0.003 per 1K input, $0.006 per 1K output).
• 3-step chain: Step 1: 400 input + 100 output ($0.003). Step 2: 600 input + 150 output ($0.004). Step 3: 700 input + 200 output ($0.005). Total: $0.012.

The 3-step chain is cheaper and more reliable. Each step is focused. You can validate Step 1 (Is the category reasonable?) before Step 2. You can log failures per step to see where the bottleneck is.

Common Mistake

The mistake: Chaining when a single prompt would do.

STEP 1: Extract the date from this email.
STEP 2: Format the date as ISO 8601.
STEP 3: Validate the date.

Three API calls for what should be one prompt. Cost: $0.003 instead of $0.001. Latency: 3× higher. Gain: minimal.

The fix: Use a single prompt if steps are trivial.

Extract the date from this email and return it in ISO 8601 format (YYYY-MM-DD).
Validate that the result is a valid date.

Email: ...

Date (ISO 8601):

One call, faster, cheaper. Chain only when individual steps are complex enough to warrant their own focus, or when earlier steps' correctness materially affects later steps.

Advanced: Conditional Branching in Chains

Not all chains are linear. Sometimes the output of Step 1 determines which Step 2 you run.

# Step 1: Classify the problem
classification = step1_classify(ticket)

# Step 2a or 2b depending on classification
if classification['is_security_risk']:
    analysis = step2_security_analysis(ticket)
else:
    analysis = step2_standard_analysis(ticket)

# Step 3: Generate response
response = step3_generate_response(classification, analysis)

This pattern is more maintainable than a single mega-prompt because:

• You can optimize Step 2a (security analysis) separately from Step 2b (standard analysis).
• You can test each branch independently.
• You can add new branches without rewriting the entire prompt.

When to Chain: Decision Framework

| Factor | Single Prompt | 2-3 Step Chain | 4+ Step Chain | |--------|---|---|---| | Task complexity | Simple | Moderate | Complex | | Latency requirement (ms) | <200 | 200–500 | 500+ acceptable | | Cost sensitivity | High (fewer calls) | Moderate | Low (reliability > cost) | | Debuggability needed | Low | High | Very high | | Intermediate validation | Not needed | Required | Required at each step | | User expectation | "Fast answer" | "Reliable answer" | "Thorough analysis" | | Example tasks | Classification, translation | Document analysis, summarization | Report generation, research |

Real-world scenario mapping:

• Single prompt: "Classify this email as spam/not spam." (Simple, fast, no intermediate state needed)
• 2–3 step chain: "Analyze this contract, extract risks, and recommend negotiation points." (Risks depend on extraction; benefits from intermediate validation)
• 4+ step chain: "Generate a quarterly business report: collect metrics, analyze trends, compare to competitors, synthesize insights, draft sections, polish final report." (Each step builds on previous; complexity warrants careful decomposition)

Failure Modes: Single Prompt vs. Chaining

| Failure Mode | Single Prompt Risk | Chaining Mitigation | Impact | |---|---|---|---| | Partial output | Model completes 3 of 5 tasks | Each step validates; incomplete steps are caught and retried | Single prompt loses work; chain catches it | | Hallucinated details | Model invents facts mid-stream | Earlier step is validated before proceeding; reduces false input | Hallucination compounds across tasks | | Context pollution | Early mistakes carry forward | Validation gates prevent bad data from propagating | Bad input → bad output → bad final result | | Debugging difficulty | "Why did the output fail?" is hard to diagnose | Each step is independently reproducible and testable | Hours to find bug vs. minutes | | Cascading failures | One error breaks the whole pipeline (hidden) | Errors surface at the step where they occur (visible) | Silent failures vs. clear failure points |

Monitoring and Observability in Chains

In production, log each step:

def chained_task(input_data):
    step1_result = step1(input_data)
    log(step=1, input_size=len(input_data), output_size=len(step1_result))

    step2_result = step2(step1_result)
    log(step=2, input_size=len(step1_result), output_size=len(step2_result))

    step3_result = step3(step2_result)
    log(step=3, input_size=len(step2_result), output_size=len(step3_result))

    return step3_result

By logging each step, you can:

• Identify which step is slow or expensive.
• See which step fails most often.
• Optimize the bottleneck first.

Summary: Single Prompt vs. Chaining

Single prompt:

• Pros: Fast, cheap (1 call), simple to implement
• Cons: Hard to debug, lower reliability on complex tasks, can't validate intermediate outputs
• Use when: Task is straightforward, latency <200ms, cost is critical

Prompt chaining:

• Pros: Reliable, maintainable, debuggable, intermediate validation possible, actually cheaper in many cases (due to fewer retries)
• Cons: Slower (multiple round-trips), requires careful step design
• Use when: Task has clear intermediate goals, reliability > latency, complexity warrants decomposition

The best chain design:

1. Each step has a single, clear goal
2. Each step specifies input and output format explicitly
3. Validate output from each step before proceeding
4. Error handling: Retry on failure, or escalate to human
5. Logging: Track performance per step to find bottlenecks
6. Optional: Branch on step outputs (different paths for different data)

The difference between a poorly-chained system and a well-chained one is the difference between a system that works 60% of the time and one that works 90%+. Start simple (1–2 steps), then add complexity only when observed failures demand it.

Prompt chaining is a powerful pattern for complex, multi-stage tasks. It trades latency for reliability, maintainability, and debuggability. Use it when your task has clear intermediate goals, when you need to validate outputs between stages, or when complexity warrants breaking into focused steps. Design each step with a single clear goal, validate outputs, and handle errors gracefully. Combined with the prompt engineering patterns from earlier lessons, chaining lets you build AI systems that are both capable and trustworthy. The best chains feel simple from the outside but are powerfully decomposed inside — each step is independently understandable and testable, yet together they accomplish complex goals that would be unreliable in a single call.