Tool Calling: Giving Agents Hands

What is tool calling?

Tool calling is how an agent requests the execution of an action. Instead of generating free-form text like "let me search for that," the model outputs a structured request: "call the search tool with this query." The agent parses that request, executes the tool, and feeds the result back to the model.

This structured handoff is what enables agents to be reliable and debuggable. Without it, the agent would have to parse natural language ("I'll search for the latest news on X") and guess what to do. With tool calling, the model's intent is explicit.

How tool calling works

Step 1: Define the schema

First, you tell the model what tools are available and how to call them. This is typically a JSON schema:

{
  "tools": [
    {
      "name": "search",
      "description": "Search the web for recent information.",
      "input_schema": {
        "type": "object",
        "properties": {
          "query": {
            "type": "string",
            "description": "The search query"
          }
        },
        "required": ["query"]
      }
    },
    {
      "name": "calculator",
      "description": "Perform arithmetic or mathematical computations.",
      "input_schema": {
        "type": "object",
        "properties": {
          "expression": {
            "type": "string",
            "description": "A mathematical expression like '2 + 2' or 'sqrt(16)'"
          }
        },
        "required": ["expression"]
      }
    }
  ]
}

Each tool has:

• name: A short identifier the model uses to request the tool.
• description: Clear English explaining when and why to use it.
• input_schema: JSON Schema defining the parameters the tool accepts. This constraint helps the model call the tool correctly.

Step 2: The model outputs a tool call

When you send a prompt to the model along with the tool definitions, the model decides whether it needs to call a tool. If so, it outputs structured data:

{
  "type": "tool_use",
  "id": "tooluse_abc123",
  "name": "search",
  "input": {
    "query": "latest AI research 2024"
  }
}

The model has decided, "I need to search for this," and it communicates how to invoke the tool. It's not generating natural language; it's generating an intent. This is the key difference from older LLM interactions.

Step 3: Dispatch the call

The agent receives the tool-call object and executes it:

def dispatch_tool_call(call: dict) -> str:
    """Execute a tool call and return the result."""
    tool_name = call["name"]
    tool_input = call["input"]

    # Route to the correct tool implementation
    if tool_name == "search":
        return search(query=tool_input["query"])
    elif tool_name == "calculator":
        return calculate(expr=tool_input["expression"])
    else:
        return f"Error: Unknown tool '{tool_name}'"

# Example execution
result = dispatch_tool_call({
    "name": "search",
    "input": {"query": "machine learning"}
})
# result = "1. Machine Learning Overview\n2. Deep Learning Tutorials\n..."

Step 4: Feed the result back

The result is appended to the conversation, and the model sees it:

messages = [
    {"role": "user", "content": "What's the latest in AI research?"},
    {
        "role": "assistant",
        "content": [
            {
                "type": "tool_use",
                "id": "tooluse_abc123",
                "name": "search",
                "input": {"query": "latest AI research 2024"}
            }
        ]
    },
    {
        "role": "user",
        "content": [
            {
                "type": "tool_result",
                "tool_use_id": "tooluse_abc123",
                "content": "1. Transformers Explained...\n2. Recent Breakthroughs..."
            }
        ]
    }
]

# Call model again; it now sees the search results
response = model.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=1024,
    tools=tool_definitions,
    messages=messages
)

The model now has the search results and can decide whether to:

• Call another tool to get more information.
• Generate a final answer to the user.
• Raise an error if the result doesn't help.

Comparing tool-calling providers

Different AI providers implement tool calling with slightly different latency and reliability characteristics. Below is an illustrative comparison based on typical performance benchmarks (illustrative estimates, not official SLAs):

| Provider | Tool schema format | Avg latency | Parse success rate | Supports parallel calls | Notes | |----------|------------------|-------------|-------------------|------------------------|-------| | Anthropic Claude | JSON Schema (compact) | ~150ms | 98.5% | Yes (up to 10) | Explicit tool_use blocks, most deterministic | | OpenAI GPT-4 | JSON with function object | ~200ms | 97.2% | Yes (sequential) | Slightly higher parsing variance | | Google Gemini | JSON (tool_config format) | ~180ms | 96.8% | Yes (batched) | Growing adoption, good for multi-tool scenarios | | Open-source Llama2 (via vLLM) | Custom function_calling plugin | ~100ms | 94.5% | Partial | Lower latency, higher error rates on complex schemas |

Key takeaway: Anthropic's tool_use mechanism has the lowest parsing errors and most explicit control, making it ideal for reliability-critical applications. Latency differences are small; choose based on schema explicitness and reliability needs rather than speed alone.

Writing effective tool schemas

Your schema guides the model's behavior. Poor schemas lead to incorrect tool calls; clear ones guide the model.

Be specific about purpose

Bad schema:

{
  "name": "run_code",
  "description": "Run code",
  "input_schema": {
    "type": "object",
    "properties": {
      "code": {"type": "string"}
    }
  }
}

This is too vague. The model may try to run anything — including commands that are slow, dangerous, or off-topic.

Better schema:

{
  "name": "run_python_analysis",
  "description": "Execute Python code to analyze data or perform calculations. Use this ONLY for numerical computations, data transformation, or statistical analysis. Do NOT use for generating prose, making API calls, or accessing external systems.",
  "input_schema": {
    "type": "object",
    "properties": {
      "code": {
        "type": "string",
        "description": "Python code that reads from 'data' (pandas DataFrame) and returns a result"
      }
    },
    "required": ["code"]
  }
}

The long description includes both what the tool does and when not to use it. This significantly improves the model's judgment.

Constrain inputs with enums

If a tool parameter has a fixed set of allowed values, say so:

{
  "name": "database_query",
  "description": "Query a database table.",
  "input_schema": {
    "type": "object",
    "properties": {
      "table": {
        "type": "string",
        "enum": ["users", "orders", "products"]
      },
      "operation": {
        "type": "string",
        "enum": ["select", "count", "filter"]
      }
    },
    "required": ["table", "operation"]
  }
}

By defining enum, you prevent the model from asking for tables that don't exist. The model sees only valid options.

Use examples in descriptions

{
  "name": "email_summary",
  "description": "Summarize emails from a mailbox. For example, 'summarize emails from [email protected]' or 'summarize support tickets from the past week'. Use this to extract key information from large email batches.",
  "input_schema": {
    "type": "object",
    "properties": {
      "mailbox": {
        "type": "string",
        "description": "Email address or mailbox name, e.g., '[email protected]' or 'archive'"
      },
      "time_range": {
        "type": "string",
        "description": "Optional. Time range like 'last 7 days', 'this month', or 'all time'."
      }
    },
    "required": ["mailbox"]
  }
}

Examples help the model understand the expected format without having to infer it.

Handling tool-call errors

Not every tool call will succeed. The agent must handle failures gracefully:

def dispatch_and_handle_errors(call: dict) -> tuple[bool, str]:
    """Execute a tool call. Return (success, result_or_error)."""
    tool_name = call["name"]
    tool_input = call["input"]

    try:
        if tool_name == "search":
            result = search(query=tool_input["query"])
            return True, result
        elif tool_name == "calculator":
            result = calculate(expr=tool_input["expression"])
            return True, result
        else:
            return False, f"Unknown tool: {tool_name}"

    except ValueError as e:
        # Invalid input (e.g., malformed math expression)
        return False, f"Invalid input: {str(e)}"
    except TimeoutError:
        # External service is slow
        return False, "Tool call timed out. Try a simpler query."
    except Exception as e:
        # Unexpected error
        return False, f"Tool execution failed: {str(e)}"

# In the agent loop:
success, result = dispatch_and_handle_errors(tool_call)
if success:
    messages.append({
        "role": "user",
        "content": [{"type": "tool_result", "content": result}]
    })
else:
    # Tell the model what went wrong
    messages.append({
        "role": "user",
        "content": [{"type": "tool_result", "content": f"Error: {result}"}]
    })

By returning errors as tool results (not raising exceptions), you let the model see what went wrong and decide how to recover. For example:

• If the calculator fails on an expression, the model might try a simpler calculation or ask the user for clarification.
• If a search times out, the model might try a more specific query.

Tool calling vs. function calling: terminology

Different vendors use different terms:

• Anthropic calls it "tool use" and structures it with type: "tool_use".
• OpenAI calls it "function calling" and structures it differently.
• LangChain and other frameworks abstract over both.

The underlying concept is identical: the model outputs a structured request for a tool/function to be executed. This lesson uses the generic term tool calling to cover both, but keep the terminology straight when reading vendor docs.

Case study: Travel booking agent with schema-driven reliability

Scenario: A travel booking agent needed to book flights, hotels, and rental cars. Early versions had a single, generic book_travel tool that accepted a natural language description. The model frequently made mistakes: booking refundable instead of non-refundable, confusing dates, or requesting unavailable options.

Solution: Redesign the tool schema to be extremely explicit about valid values and constraints.

{
  "tools": [
    {
      "name": "search_flights",
      "description": "Search for available flights between two cities on a specific date. Returns multiple options with prices.",
      "input_schema": {
        "type": "object",
        "properties": {
          "from_airport": {
            "type": "string",
            "enum": ["JFK", "LAX", "ORD", "ATL", "DEN"],
            "description": "3-letter airport code for departure"
          },
          "to_airport": {
            "type": "string",
            "enum": ["JFK", "LAX", "ORD", "ATL", "DEN"],
            "description": "3-letter airport code for arrival"
          },
          "departure_date": {
            "type": "string",
            "pattern": "^\\d{4}-\\d{2}-\\d{2}$",
            "description": "YYYY-MM-DD format, must be at least 2 days in the future"
          },
          "passengers": {
            "type": "integer",
            "minimum": 1,
            "maximum": 9,
            "description": "Number of passengers"
          },
          "cabin_class": {
            "type": "string",
            "enum": ["economy", "premium_economy", "business", "first"],
            "description": "Cabin class"
          }
        },
        "required": ["from_airport", "to_airport", "departure_date", "passengers"]
      }
    },
    {
      "name": "confirm_flight_booking",
      "description": "Confirm and book a specific flight option from the search results. ONLY use after calling search_flights and showing the user options.",
      "input_schema": {
        "type": "object",
        "properties": {
          "flight_id": {
            "type": "string",
            "description": "Flight ID returned from search results, e.g., 'AA_1234_2024-08-15'"
          },
          "refund_policy": {
            "type": "string",
            "enum": ["refundable", "non_refundable"],
            "description": "Choose explicitly; refundable options cost 15-20% more"
          }
        },
        "required": ["flight_id", "refund_policy"]
      }
    }
  ]
}

Key improvements:

1. Enum constraints for airport codes prevent typos and invalid searches.
2. Separation of concerns: search_flights finds options; confirm_flight_booking commits only after the user approves.
3. Explicit policy choices: The model can't accidentally book non-refundable when the user wanted refundable.
4. Pattern matching on dates ensures correct formatting.

Result: Booking success rate improved from 89% to 97% with no additional model training. The schema did the heavy lifting by constraining the model's output space.

Edge case: Handling ambiguous tool-call outputs

Sometimes a model generates a tool call that technically parses but semantically doesn't make sense. For example:

{
  "name": "search_flights",
  "input": {
    "from_airport": "JFK",
    "to_airport": "JFK",  // ← Searching from JFK to itself
    "departure_date": "2025-12-25",
    "passengers": 1
  }
}

The JSON is valid, but the request is illogical. A rigid dispatcher would execute it (and return zero results). A smarter agent would catch this and ask for clarification:

def validate_tool_call(call: dict) -> tuple[bool, Optional[str]]:
    """
    Validate a tool call for semantic correctness beyond schema parsing.
    Returns (is_valid, error_message).
    """
    tool_name = call["name"]
    inputs = call["input"]

    # Flight search sanity checks
    if tool_name == "search_flights":
        if inputs.get("from_airport") == inputs.get("to_airport"):
            return False, "Cannot search flights from and to the same airport. Did you mean a different destination?"

        from_date = datetime.fromisoformat(inputs.get("departure_date", ""))
        if from_date <= datetime.now() + timedelta(days=1):
            return False, "Flights must be booked at least 2 days in advance."

        if inputs.get("passengers", 0) < 1:
            return False, "Number of passengers must be at least 1."

    return True, None

# In dispatcher:
success, error_msg = validate_tool_call(tool_call)
if not success:
    # Return error to model so it can retry
    messages.append({
        "role": "user",
        "content": [{"type": "tool_result", "content": f"Invalid request: {error_msg}"}]
    })
else:
    # Proceed with dispatch
    result = dispatch_tool_call(tool_call)

This pattern catches logical errors that the schema alone couldn't prevent, improving overall agent reliability.

Common mistake

Building a tool schema for every possible action, leading to decision paralysis. Agents are most reliable when they have few, well-scoped tools. A search agent should have a search tool (maybe a calculator if needed). A code-review agent should have file-read and comment tools. Don't create 20 tools and expect the model to pick the right one every time. Start with 1–3 tools. Add more only when you observe the agent failing because a needed capability is missing.