Chapter 51 — The agent loop, spelled out

The loop

   +-----------------------------------------------------------+
   |                                                           |
   |   1. THINK   send the conversation + tool list to the LLM |
   |                                                           |
   |   2. ACT     if it returns a TOOL CALL:                   |
   |                  YOUR CODE executes the tool              |
   |                                                           |
   |   3. OBSERVE append the tool's result to the conversation |
   |                                                           |
   |   4. REPEAT  ------------------------------------->-------+
   |
   |   ...until the model returns a normal answer instead of a
   |      tool call, OR you hit a limit (steps, time, cost).
   +-----------------------------------------------------------+

That's the whole idea. It's often called the ReAct pattern (Reasoning + Acting), because the model alternates between thinking and doing.

What it looks like as a conversation

   USER:       "What's the weather in Paris, and should I take a coat?"

   MODEL:      [tool_call: get_weather(city="Paris")]        <-- step 1: ACT

   YOUR CODE:  runs get_weather("Paris")  ->  {"temp_c": 8, "rain": true}

   YOU APPEND: [tool_result: {"temp_c": 8, "rain": true}]    <-- step 2: OBSERVE

   MODEL:      "It's 8 C and raining in Paris -- yes, take a coat
                and something waterproof."                   <-- done, no tool call

Notice: the model never runs anything. It emits a request to run something. Your code decides whether to honour it. That gap is where every guardrail in Chapter 60 lives.

The loop in ~40 lines

This is a real, working agent. Read it slowly; it's the heart of the Part.

import json

def run_agent(user_message, tools, tool_impls, client, model,
              max_steps=10, max_cost_usd=0.50):
    """
    tools:      list of tool SCHEMAS (what the model sees)
    tool_impls: dict of name -> python function (what actually runs)
    """
    messages = [
        {"role": "system", "content":
            "You are a helpful assistant. Use tools when needed. "
            "When you have the answer, reply normally without calling a tool."},
        {"role": "user", "content": user_message},
    ]

    spent = 0.0
    for step in range(max_steps):                       # <-- HARD LIMIT
        response = client.chat(model=model, messages=messages, tools=tools)
        spent += estimate_cost(response)                 # <-- BUDGET LIMIT
        if spent > max_cost_usd:
            return "Stopped: cost limit reached."

        messages.append(response.message)

        # No tool call? The model is done.
        if not response.tool_calls:
            return response.message["content"]

        # Otherwise: execute each requested tool
        for call in response.tool_calls:
            name = call.function.name
            args = json.loads(call.function.arguments)

            print(f"  step {step}: calling {name}({args})")   # <-- OBSERVABILITY

            if name not in tool_impls:                        # <-- ALLOWLIST
                result = {"error": f"unknown tool {name}"}
            else:
                try:
                    result = tool_impls[name](**args)
                except Exception as e:
                    result = {"error": str(e)}                # <-- errors go BACK
                                                              #     to the model
            messages.append({
                "role": "tool",
                "tool_call_id": call.id,
                "content": json.dumps(result)[:4000],         # <-- TRUNCATE
            })

    return "Stopped: step limit reached."

The five things in that code that aren't the loop

They look like details. They are the difference between a demo and something you can run in production.

Why does this work at all?

Because the model was trained on enormous amounts of text where people reason step by step, and post-trained (Chapter 34) on examples of using tools correctly. Each loop iteration is just next-token prediction (Chapter 31) — but the context now includes what actually happened last step, which grounds the next decision in reality rather than guesswork.

import ast, datetime, math, operator

OPS = {
    ast.Add: operator.add, ast.Sub: operator.sub,
    ast.Mult: operator.mul, ast.Div: operator.truediv,
    ast.Pow: operator.pow, ast.USub: operator.neg,
}

def safe_calculate(expression):
    def eval_node(node):
        if isinstance(node, ast.Expression):
            return eval_node(node.body)
        if isinstance(node, ast.Constant) and isinstance(node.value, (int, float)):
            return node.value
        if isinstance(node, ast.BinOp) and type(node.op) in OPS:
            return OPS[type(node.op)](eval_node(node.left), eval_node(node.right))
        if isinstance(node, ast.UnaryOp) and type(node.op) in OPS:
            return OPS[type(node.op)](eval_node(node.operand))
        if (isinstance(node, ast.Call) and isinstance(node.func, ast.Name) and
                node.func.id == "sqrt" and len(node.args) == 1):
            return math.sqrt(eval_node(node.args[0]))
        raise ValueError("Only numbers, +, -, *, /, **, parentheses, and sqrt() are allowed.")
    return eval_node(ast.parse(expression, mode="eval"))

tool_impls = {
    "get_time": lambda timezone="UTC":
        {"now": datetime.datetime.now(datetime.timezone.utc).isoformat()},
    "calculate": lambda expression:
        {"result": safe_calculate(expression)},
}

tools = [
  {"type": "function", "function": {
     "name": "get_time",
     "description": "Get the current UTC time.",
     "parameters": {"type": "object", "properties": {}}}},
  {"type": "function", "function": {
     "name": "calculate",
     "description": "Evaluate a simple arithmetic expression.",
     "parameters": {"type": "object",
                    "properties": {"expression": {"type": "string"}},
                    "required": ["expression"]}}},
]

print(run_agent("What time is it, and what is 17 * 23 + sqrt(144)?",
                tools, tool_impls, client, model))