Reasoning and Planning Strategies

Three strategies, three different jobs

"Reasoning" in agent design usually refers to one of a few specific, published techniques, each suited to a different kind of task.

Each strategy is a different way to allocate tokens. CoT spends them on step-by-step reasoning. ReAct spends them on tool calls plus brief reasoning. Tree-of-thought spends them on exploring multiple branches. The choice depends on the task's structure, not on which is "latest" or most sophisticated.

ReAct is the pattern behind most agent loops

The ReAct paper (Yao et al., 2022) formalized the pattern most agent frameworks now use by default: at each step, the model produces a brief thought, then an action (a tool call), then reads the observation (the tool's result) before its next thought. This is worth recognizing because it's effectively a more structured version of the perceive-plan-act-observe loop from the earlier lesson, with the model's reasoning made explicit rather than implicit.

A real ReAct transcript:

User: What's the current weather in Boston, and is it a good day for outdoor jogging?

Thought: The user is asking about current weather (which changes daily) and
whether it's suitable for jogging. I need to:
1. Get current weather data
2. Assess whether those conditions are good for jogging

I don't have current weather in my training data, so I need to use a tool.

Action: get_weather(location="Boston")
Observation: {"location": "Boston", "temp_f": 68, "condition": "partly cloudy",
"humidity": 55, "wind_mph": 8}

Thought: Good weather data. 68F and partly cloudy is pleasant for jogging.
Low humidity (55%) is favorable. Wind is moderate at 8mph, not problematic.
I have enough to answer.

Answer: It's 68°F and partly cloudy in Boston -- excellent conditions for outdoor
jogging. Temperature is comfortable, humidity is low, and wind is moderate.

Making the "Thought" step explicit measurably improves tool selection accuracy compared to skipping straight to an action, because it:

• Prevents wrong tool choices: The model's reasoning ("I need current data, not training data") helps it pick the right tool.
• Reduces hallucination: Documenting what it's about to look for makes it harder for the model to fabricate information.
• Aids debugging: When an agent makes a mistake, the explicit thought shows where the reasoning went wrong.

The cost: a few extra tokens per step (the thought is usually 50-100 tokens). Whether this tradeoff is worth it depends on the task.

When chain-of-thought is sufficient

Chain-of-thought (CoT) is prompting the model to show its work without external tool calls:

User: If a store has 120 items and sells 40% of them, then sells half of what remains,
how many items are left?

(With CoT, the model writes:)

Thought: Let me work through this step by step.
- Start with 120 items
- Sell 40%: 120 × 0.40 = 48 items sold
- Remaining: 120 - 48 = 72 items
- Sell half of remaining: 72 ÷ 2 = 36 items sold in second batch
- Final remaining: 72 - 36 = 36 items

Answer: 36 items remain.

This works well for tasks that are entirely self-contained: math problems, logic puzzles, multi-step inference on information the model already has. It doesn't require tool calls. But it doesn't scale to tasks requiring external information (current data, proprietary databases, real-time APIs).

Use chain-of-thought when:

• The task is entirely self-contained reasoning (no external tools needed)
• The model has all the information upfront
• The reasoning is multi-step but deterministic (one clear path is right)

Don't use chain-of-thought alone when:

• You need external information (web search, database queries, live APIs)
• The answer depends on ambiguous intermediate results (use ReAct instead)

When branching search is worth the cost

Tree-of-thought (ToT) approaches explore and compare multiple reasoning branches rather than committing to one linear path. They can outperform a single CoT or ReAct pass on problems with several genuinely different valid approaches -- but they cost 4-10x as many model calls as the linear approaches.

This tradeoff only pays off when:

1. The task has real branching: several distinct strategies are plausible
2. Early decisions are hard to undo: picking the wrong approach early is expensive
3. Pruning is possible: you can evaluate branches and skip weak ones

For tasks with one clear best approach, ToT just adds cost without changing the outcome.

A tree-of-thought example

Scenario: "Find the best way to reduce this database query's latency from 5 seconds to <1 second."

Single-path approach (CoT or ReAct):

• Model commits to one idea: "add an index on the user_id column"
• Reasons through: "An index would speed up the WHERE clause lookup from O(n) to O(log n)"
• Concludes: "This will cut latency to ~1 second"
• But after implementation: turns out the real bottleneck is a network round trip to a remote database, not the query itself. The index barely helps.

Tree-of-thought approach:

1. Generate candidates: The model suggests four approaches:

   • A: Add an index on user_id
   • B: Batch multiple queries into a single call
   • C: Add caching (Redis) for repeated queries
   • D: Move the remote database closer (geographic locality)

2. Explore each branch briefly (2-3 reasoning steps):

   • A: "Index reduces query time from O(n) to O(log n), saves ~100ms"
   • B: "Batching reduces network round trips from N to 1, saves ~500ms if typical load is 5-10 queries"
   • C: "Cache hit rate estimated 60-70%, saves all latency for cache hits"
   • D: "Reduces network latency from 4000ms to 1000ms, most impact"

3. Evaluate and rank branches: D > B > C > A (based on estimated impact)

4. Commit to the top branch: Deep dive into D (geographic locality) because the early reasoning suggests it solves the real problem.

The cost: 4 initial explorations + 1 evaluation + deep dive into D = 6x the single-path cost. But the outcome is correct; a single-path approach would have wasted time on A.

When to use tree-of-thought:

• Multi-path optimization problems (architectural decisions, strategic choices)
• Problems where wrong early decisions are expensive to unwind
• Tasks where the space of solutions is large and diverse
• Tasks where you can evaluate branches objectively (estimated latency, cost, feasibility)

When NOT to use tree-of-thought:

• Simple, deterministic tasks (math, logic)
• Tasks with one clearly best approach
• Anything where the branching is theoretical but the outcome is obvious
• High-speed/low-latency applications (the cost of ToT is too high)

Choosing a strategy without guessing

Here's a simple decision tree:

Q1: Does the agent need external information (web search, API calls, database queries)?

• Yes → ReAct (interleave reasoning with tool calls)
• No → Go to Q2

Q2: Is the reasoning entirely within one approach, or are there multiple plausible paths?

• Single path (math, simple inference) → Chain-of-thought
• Multiple paths → Go to Q3

Q3: Would picking the wrong path early be expensive to undo?

• Yes (architectural decisions, strategic choices) → Tree-of-thought
• No (exploratory tasks, low-cost reversible steps) → ReAct with brief reasoning

Most agent tasks: Fall into the ReAct category. You're gathering external data, deciding which tool to call, reading results, and adjusting. ReAct's thought-action-observation loop is the right shape.

Occasional CoT-only tasks: Self-contained reasoning on known facts. Use it for specific complex steps, not the entire agent loop.

Rare ToT tasks: High-stakes decisions with multiple plausible approaches and reversibility costs (architectural decisions, research planning).

A practical example: research agent choosing strategies

An agent researching three competitors uses different strategies at different steps:

Steps 1-2 (ReAct): Search for each competitor's website and pricing.

• Thought: "I need to find Competitor A's website"
• Action: search("Competitor A pricing")
• Observation: [results with links]
• (No CoT or ToT here; the step is deterministic)

Step 3 (Brief CoT): Extract key features from the pages retrieved.

• Thought: "Given these pages, what are the top 3 features Competitor A emphasizes? Looking at the landing page... it mentions performance, reliability, and ease of use. Those are the top 3."
• (Single path, CoT is light here)

Step 4 (ToT, if needed): Decide which competitor is best overall.

• Generate branches: "Best for speed? Best for reliability? Best for cost?"
• Evaluate each branch on the extracted data
• (Multiple criteria, multiple valid orderings)

Most of the agent loop is ReAct. CoT appears in isolated reasoning steps. ToT is used only for the comparison step, where branching matters.

Strategy comparison: cost vs. benefit across task types

Here's how the three strategies compare in practice across different agent scenarios (illustrative estimates):

| Scenario | Strategy | Avg. steps | Accuracy | Token cost | Wall-clock time | |---|---|---|---|---|---| | Simple web search for a fact | ReAct | 2-3 | 85-90% | 1,000-2,000 | 3-5s | | Same search with CoT only | CoT | 1 | 60-70% | 500-800 | 1-2s | | Multi-source price comparison | ReAct | 5-8 | 88-92% | 3,000-6,000 | 8-15s | | Same with ToT exploration | ToT | 12-16 | 95-98% | 12,000-20,000 | 20-30s | | Architectural decision (3 options) | ReAct (picks 1) | 3-4 | 55-65% (wrong choice) | 2,000-3,000 | 5-8s | | Same with ToT | ToT | 8-12 | 85-92% (usually right) | 8,000-15,000 | 15-20s |

The tradeoff is clear: CoT is fastest but least accurate for anything requiring external data. ReAct is the standard. ToT is most accurate but most expensive, and only worth it for decisions that are hard to reverse.

Extended theory: why reasoning explicitness matters

There's a theoretical reason why making reasoning explicit (CoT and ReAct) improves accuracy: it creates an interpretable intermediate representation.

When a model jumps directly from input to action ("search for competitor A pricing") without stating why, if it chooses wrong, there's no signal for correction. But when the model states "The user wants current pricing, not historical data, so I need a fresh search on competitor A's website" before acting, wrong choices become debuggable. The reasoning reveals faulty assumptions.

This is why ReAct is so effective for agents: the "Thought" step isn't just for humans to read -- it's for the model to ground its action in explicitly-stated logic. A model that can't articulate its reasoning tends to make noisy choices; a model that must articulate it gets corrected when the reasoning is flawed.

Failure mode: A model takes an action without explicitly reasoning about it. If the action fails, it might retry the same failing action indefinitely, because it never stated what it expected to happen. But a model that stated "I expect search results containing the company's official pricing page" can read the failure ("got only outdated cached pages") and learn to refine.

Here's a concrete example:

def agent_step_without_explicit_reasoning(query, tools):
    """Bad: no reasoning before action."""
    # Model directly chooses a tool without explaining why
    results = tools['search'](query)
    return results

def agent_step_with_explicit_reasoning(query, tools, llm):
    """Good: model states reasoning before acting."""
    # Step 1: Model thinks about what's needed
    thought = llm.generate(
        f"User asks: {query}. What tool do I need? Why that tool?"
    )
    print(f"Thought: {thought}")

    # Step 2: Model chooses and acts
    action = llm.generate(
        f"Given the thought: {thought}, what tool call should I make?"
    )
    # Execute the action...
    results = tools[action['tool']](**action['args'])

    # Step 3: Evaluate against expectations
    expectation = llm.extract_expectation(thought)
    if not results_match_expectation(results, expectation):
        # Model can now reason: "I expected X, got Y, so next I should..."
        pass

    return results

The second version is more expensive (extra LLM calls) but more robust, because explicit reasoning creates a feedback loop.

Edge case: When reasoning strategies fail

ReAct with noisy tool results ReAct assumes tools provide clean, interpretable results. But real tools often return:

• Paginated results (top 10 of 5,000 matches)
• Partial failures (some fields missing)
• Ambiguous data ("results ranked by relevance" but relevance to what?)

When tool results are noisy, explicit reasoning in the "Thought" step helps the model notice the noise. But if the model hallucinates an interpretation of the noisy result, ReAct doesn't help -- the next step builds on a false premise.

Chain-of-thought with overconfidence CoT can make the model more confident in wrong reasoning. If a model reasons step-by-step through a logic error, each step might look right, but the conclusion is wrong. CoT doesn't prevent this; it just makes the error traceable.

Tree-of-thought with poor evaluation ToT requires evaluating branches to prune weak ones. But if the evaluation metric is wrong, ToT might prune the branch that actually works. For example:

• Evaluation metric: "Which approach completes fastest?"
• Right answer: Approach C (slowest to implement, but most reliable)
• ToT might choose: Approach A (fastest, but fragile)

Case study: Reasoning strategy failure in a documentation agent

Scenario: A large software company built an agent to help developers find answers in internal documentation. Initial deployment used simple ReAct with web search on their docs:

Thought: User asks "how do I authenticate with OAuth?"
Action: search("OAuth authentication")
Observation: Got 15 results about different OAuth flows

What went wrong: The search results were all technically valid pages from their docs, but scattered. One page was about client-side OAuth flow, another about server-side, another about token refresh. The agent would pick the first result, extract information, and return it -- but sometimes gave instructions for the wrong OAuth flow.

Root cause: ReAct without explicit evaluation of result quality. The agent didn't reason "I got multiple OAuth implementations; I need to figure out which one the user's scenario matches." It just took the first one.

Failed fix attempt #1: Add more CoT reasoning

Thought: User asks about OAuth. There are many OAuth flows.
What is the user's scenario?
[Model reasons for 5 steps about possible scenarios]
Action: search("OAuth authentication")

This just added delay without fixing the root issue: the search results were still ambiguous and the agent still picked the first one.

Successful fix: Add explicit evaluation logic (quasi-ToT for a single query)

Thought: The user asked about OAuth, but I got multiple implementations.
I need to identify which one matches the user's scenario.

Action 1: search("OAuth client vs server comparison")
Observation 1: Got a disambiguation page showing when to use which flow

Thought 2: Good, I now understand the flows. What is the user's scenario?
Action 2: search("user said [specific tech stack mentioned], OAuth")
Observation 2: Got pages specific to that tech stack

Thought 3: Now I have the right docs. I can answer confidently.

By adding an intermediate evaluation step (searching for context before diving into the main query), the agent went from 40% correct answers to 92% correct. The reasoning wasn't longer, but it was structured better to identify and resolve ambiguity.

Common mistake

Applying chain-of-thought prompting to every single step of an agent loop by default, or assuming tree-of-thought is always better than simpler strategies. In practice:

• CoT doesn't help when the task is deterministic or tool-driven (adding "think step by step" just adds tokens)
• ReAct is the backbone of most agent loops for good reason (interleaving reasoning and tool calls is the natural pattern)
• Tree-of-thought is reserved for the minority of high-stakes decisions where multiple approaches are genuinely plausible

Choose strategies based on task structure, not on what sounds sophisticated. Most agents use ReAct as the default, add CoT for ambiguous reasoning steps, and reserve ToT for rare, expensive decisions.