How an Agent Works: Inside the Agent Loop

The loop, one step at a time

Every agent, regardless of framework, runs some version of the same four-part loop: perceive the current state, plan the next step, act on that plan, and observe the result -- then decide whether to loop again or stop.

Perceive. The orchestrator assembles everything relevant into the model's context: the original goal, the conversation so far, any tool results from previous iterations, and relevant memory. This is the input to the current step.

Plan. Given that context, the model decides what to do next: answer directly, call a specific tool with specific arguments, or ask a clarifying question. This decision is just the model predicting the most useful next output -- there's no separate "planning module" unless the architecture explicitly adds one (more on that in the next lesson).

Act. If the model chose a tool, the orchestrator executes it -- calling an API, running code, querying a database -- and captures whatever comes back: a result, an error, or a timeout.

Observe. The tool's result is added back into context for the next iteration. The model reads it on the next pass through the loop, and the cycle repeats.

Minimal pseudocode

context = [system_prompt, goal]
turns = 0
max_turns = 10

while turns < max_turns:
    # Call the model with accumulated context
    response = call_model(context)

    # Check if the model has produced a final answer
    if response.is_final_answer:
        return response.text

    # Execute the tool the model requested
    if response.tool_call:
        try:
            result = execute_tool(response.tool_call)
        except ToolError as e:
            result = f"Error executing tool: {e}"

    # Append both the call and result to context for next iteration
    context.append(("tool_call", response.tool_call))
    context.append(("tool_result", result))
    turns += 1

# If we exit the loop without a final answer
return "Stopped: reached turn limit (10 turns) without completing the task"

This is deliberately minimal -- production agents add error handling, cost tracking, and permission checks around every tool call -- but the core shape (call model, check for done, otherwise act and append, repeat) is the same shape underneath frameworks like LangGraph or a hand-rolled loop.

Note the error handling: if a tool call fails (network timeout, invalid arguments, etc.), the orchestrator catches the error and returns it as a string in the context. The model observes that the tool failed and can decide whether to retry with different arguments, try a different tool, or give up. This adaptive error recovery is part of what makes agents resilient.

A worked example, traced through the loop

The pseudocode above is easiest to internalize against one concrete run. Say the goal is: "Check high-priority support tickets; notify the on-call engineer if any are overdue."

Walking through what actually happens at each stage: the orchestrator perceives the goal and an empty history, so the model's first plan is simply "I need the current ticket data before I can check anything" -- it can't answer from training knowledge because "overdue" depends on real, current timestamps. It acts by calling get_tickets(priority="high"). The tool returns a list of tickets with their due dates. The model observes that result and now faces the branch that makes this genuinely agentic rather than scripted: are any of them actually overdue? If none are, the correct behavior is to stop immediately and report that nothing needs attention -- sending a notification here would be a bug, not a feature. If one or more are overdue, the loop continues: the model checks memory for who is currently on call, then acts again by calling notify(oncall_id, message), observes that the notification succeeded, and only then produces its final answer.

Notice that the number of iterations wasn't fixed in advance -- an empty ticket list finishes in one perceive-plan-act-observe pass, while an overdue ticket needs two full passes (one to check tickets, one to notify) before the stopping condition is met. That variability, driven entirely by what the tool returned, is the actual mechanical difference between this and a fixed script that always sends a notification regardless of what it finds.

The two decisions on every iteration

Underneath the four perceive/plan/act/observe steps, the model is really only ever making two decisions on each pass: what single action moves the goal forward right now, and is the goal already met. Nearly every agent bug is a wrong answer to one of those two questions -- picking the wrong tool, or continuing to loop after the goal was already satisfied (or the reverse: stopping too early, before the goal is actually met).

What can go wrong at each stage

Because the loop has distinct stages, diagnosing a broken agent is straightforward once you know which stage failed:

Perceive stage breaks: context is incomplete or the wrong information is given to the model. Fix: ensure relevant facts are being retained and surfaced at the right time.

Plan stage breaks: the model makes a bad decision about what to do next. This happens when tool descriptions are ambiguous or when the model is confused about the goal. Fix: clarify tool descriptions and make the goal more explicit.

Act stage breaks: the tool fails, returns an error, or times out. Fix: add better error handling and retry logic, or ensure the model can gracefully handle tool failures.

Observe stage breaks: the tool result is not formatted clearly or is not appended to context properly, so the next iteration's plan is based on incomplete information. Fix: ensure results are structured and clearly labeled in context.

Debugging with detailed logs: a production example

In practice, diagnosing agent failures requires logging at each stage. Here's pseudocode for an instrumented agent loop:

def agent_loop_with_diagnostics(goal: str, tools: dict, max_turns: int = 10) -> dict:
    context = [goal]
    logs = []
    turn = 0

    while turn < max_turns:
        turn += 1

        # Perceive: log what context the model sees
        logs.append({
            "turn": turn,
            "stage": "perceive",
            "context_tokens": len(encode(str(context))),
            "context_summary": f"Goal + {len([x for x in context if 'result' in str(x)])} prior results"
        })

        # Plan: call model and log the decision
        response = call_model(context=context, tools=tools)
        logs.append({
            "turn": turn,
            "stage": "plan",
            "model_decision": response.tool_choice or "final_answer",
            "confidence": response.confidence_score
        })

        if response.is_final_answer:
            logs.append({"turn": turn, "stage": "stopping_condition_met", "reason": "model_signal"})
            return {"answer": response.text, "logs": logs, "turns_taken": turn}

        # Act: execute tool and log result
        try:
            tool_result = execute_tool(response.tool_call)
            logs.append({
                "turn": turn,
                "stage": "act",
                "tool": response.tool_choice,
                "status": "success",
                "result_tokens": len(encode(str(tool_result)))
            })
        except ToolError as e:
            logs.append({
                "turn": turn,
                "stage": "act",
                "tool": response.tool_choice,
                "status": "error",
                "error_message": str(e)
            })
            tool_result = f"Tool failed: {e}"

        # Observe: append and log
        context.append(f"Turn {turn} result: {tool_result}")
        logs.append({
            "turn": turn,
            "stage": "observe",
            "total_context_tokens": len(encode(str(context)))
        })

    logs.append({"turn": turn, "stage": "stopping_condition_met", "reason": "max_turns_reached"})
    return {"answer": "Reached max turns without completion", "logs": logs, "turns_taken": turn}

With these logs, you can instantly see:

• If perceive stage is losing information (context_tokens not growing correctly)
• If plan stage is making bad decisions (model_decision looks wrong for the context)
• If act stage is failing (status: error)
• If observe stage is not appending results (total_context_tokens drops instead of growing)

This turns "the agent is broken" into "turn 3, plan stage: model decided to call the wrong tool."

Building intuition with a more complex example

Let's trace a slightly more involved task: "Find three GitHub repositories related to AI agents that were updated in the last month, and for each one, check the main branch's most recent commit message."

Turn 1 - Perceive: Goal is loaded. Context is empty of results.

Turn 1 - Plan: Model decides: "I need to search GitHub for agent-related repos."

Turn 1 - Act: Calls search_github(query="AI agents", sort="recently-updated", limit=10). Returns a list of 10 repos with their last update dates.

Turn 1 - Observe: Results appended to memory. Context now contains: the goal + the 10 repos.

Turn 2 - Perceive: Model receives context with goal + 10 repos.

Turn 2 - Plan: Model observes that most were updated in the last month. It selects the top three and reasons: "I need the most recent commit message for each." Decides to call fetch_latest_commit(repo="repo-1").

Turn 2 - Act: Calls the function. Returns commit message for repo 1.

Turn 2 - Observe: Result appended.

Turn 3 & 4: Model repeats the plan-act-observe cycle for repos 2 and 3.

Turn 5 - Perceive: Model has goal + 10 repos + 3 commit messages, one from each selected repo.

Turn 5 - Plan: Model recognizes it has enough information. It composes a final answer summarizing all three repos and their most recent commits.

Turn 5 - Observe: Model signals "this is my final answer," orchestrator detects the stopping condition, and the loop ends.

Notice: the number of iterations was not fixed in advance. The loop had to do turns 1-5 because each new piece of information revealed whether the next step was needed. Turn 1 had to happen before the model could decide which repos to use. Turns 2-4 depended on Turn 1's results. Turn 5 depended on having all three commit messages. That data dependency is what makes this genuinely agentic; a fixed script would always fetch three commits regardless of whether repos were actually recent.

Tokens and cost in the loop

Each iteration of the loop consumes tokens: context from prior iterations, new tool results, the model's next response. An agent task that takes 5 turns will cost roughly 5x the tokens of a single model call, though the exact multiple varies based on whether earlier context is truncated or summarized in later iterations (a strategy called "context compression," covered in a later lesson). This is why the cost-benefit tradeoff matters: a task that could be done in one call costs money per additional iteration, so agents are only justified when those iterations actually improve the outcome.

Cost and token accumulation table

Here's an illustrative breakdown of how token usage grows across a typical multi-turn agent run (illustrative estimates based on average token counts):

Note: These costs are illustrative estimates for a small model at ~$0.003 per 1K input tokens. Larger models and longer context can multiply costs significantly. The key insight: agent loops pay linearly per turn, so 10 turns costs roughly 10x a single call. Build agents only when the outcome quality justifies the cost.

Case study: Multi-step research agent

Consider an agent researching "What are the technical differences between lithium-ion and solid-state batteries?" over three turns.

Turn 1: Goal is loaded. Model decides: "I need to search for current technical specs on both battery types." Calls search_technical_database(query="lithium-ion vs solid-state battery specs 2024-2026"). Result: 800 tokens of abstracts from recent papers.

Turn 2: Model reads the abstracts and observes that solid-state mentions "higher energy density" repeatedly, but the papers are sparse on cost comparisons. Calls search_market_analysis(query="solid-state battery cost and commercialization timeline"). Result: 600 tokens of analyst reports on commercialization.

Turn 3: Model has abstracts + market analysis. Recognizes it can now synthesize a comprehensive answer. No more tool calls. Produces a final answer comparing energy density, safety, cost trajectories, and expected commercialization dates.

Trace the loop:

• Perceive: Turn 1 has empty context (just goal). Turn 2 has goal + search results. Turn 3 has goal + both search results.
• Plan: Turn 1 model decides to search tech specs. Turn 2 model observes the gap and searches market data. Turn 3 model observes it has enough.
• Act: Turn 1 executes search_technical_database. Turn 2 executes search_market_analysis. Turn 3 produces final answer (no act).
• Observe: Turn 1 results are added to context. Turn 2 results are added to context. Turn 3 recognizes no new tools are needed.

This is genuinely agentic: each tool call depended on observing the prior results. A fixed script would call both tools regardless, or have a human decide the search order in advance. The agent's adaptive branching is what makes it valuable despite the higher cost.

Edge case: Off-by-one errors in stopping conditions

A subtle bug: the stopping condition is checked after the tool executes, not before. This means an agent might reach its turn limit and still execute one more tool call, consuming extra cost and potentially causing side effects.

Example bug: The agent has a 5-turn limit. On turn 5, the model decides to call delete_old_logs(). The orchestrator runs the tool and then checks "are we at the limit?" If the check happens after the execute, the deletion happens even though we're at the limit.

Fix: Check the stopping condition before executing the tool on the next iteration. Or count the tool execution as a turn before running it, not after. Logging the turn number is crucial here.

Another edge case: early stopping without verification. An agent might signal it's done before actually completing the task, especially under a vague goal. Always pair the model's stopping signal with a secondary verification -- either a human review or a final validation step. Production systems often add a fourth step to the loop: Perceive → Plan → Act → Observe → Verify (check that the goal was actually met before returning).

Common mistake

Letting the loop run without an explicit, checkable stopping condition beyond "the model said it's done." Models can be wrong about whether they've actually finished, especially under a vague goal. Production agents pair the model's own judgment with hard limits -- a maximum turn count, a cost budget, or a required verification step -- so a confused agent can't loop indefinitely or stop with the job half done. A common failure mode: the model gets into a loop where it keeps trying the same tool with slightly different arguments, never recognizing that it's stuck. Always have a hard turn limit, and log every turn for debugging.