Aitishiku.comThe AI Agent Engineer's Guide: 60 Patterns for Building Autonomous Systems [Full Book]
This book is a capability-led field guide to the architectures that make modern AI agents actually work. It includes code, failure modes, and illustrative composite case studies for every pattern.
About This Book
The first wave of agent literature was organized by domain. It told you how to build a healthcare agent, a finance agent, or a coding agent, as if the discipline were a set of vertical recipes.
That framing was useful while the field was young. But it can now be misleading. The healthcare agent and the coding agent, when you look past the prompts and the toolsets, are running the same five or six architectural patterns. The variation is cosmetic. The substance is capability.
This book reorganizes agent engineering around the capabilities themselves. There are eight that matter: perception, reasoning, planning, memory, tool use, coordination, learning, and alignment.
Every working agent on the planet, from the cron-job-with-a-prompt that summarizes your inbox to the multi-agent system that drafts merger documents, is a composition of these eight, in different ratios and at different fidelities.
If you understand the patterns inside each capability, you can build any agent on demand. But if you understand only the domain templates, you'll spend the rest of your career rediscovering the same architectures with slightly different prompts.
The number sixty in the subtitle is not a marketing flourish. It's the number of distinct, named patterns this book defines. Some are well-known under other names, while many are formalized here for the first time. Each pattern is presented with eight things:
A one-line tagline.
The problem in technical detail: what specifically goes wrong without this pattern.
Why naïve approaches fail: the false fixes that look reasonable and aren't.
The mechanism: the architectural moves that define the pattern, in enough depth that you can implement it.
A code skeleton: a working Python sketch, schematic rather than runnable, that captures the load-bearing structure.
Trade-offs and alternatives: when not to use the pattern, and what to use instead.
Production failure modes: what breaks first, and how to detect it.
A case study: a real-world deployment shape, with concrete numbers where they exist, demonstrating the pattern's value.
A pattern entry ends with a Pairs with line that names the patterns it most often appears alongside in real systems, because composition is the point.
The book has no chapter on "AI agents in healthcare" or "AI agents in finance." Those chapters write themselves once you have the underlying capabilities in hand.
Instead, every domain example is folded into the case studies attached to individual patterns. A clinical decision-support workflow appears under the Provenance Tracker Agent and the Refusal Calibrator Agent, not under a "healthcare" heading. A contract-analysis pipeline appears under the Hierarchical Decomposer Agent, the Constraint-Satisfaction Agent, and the Side-Effect Auditor Agent.
Domain is a lens through which capabilities are exercised, never a substitute for understanding them.
A note on framing: this book treats agents as software artifacts, not as quasi-people. An agent is a system with a defined input contract, a defined output contract, an internal control loop, and a set of side effects. It's built, tested, observed, and decommissioned.
The mystification that surrounds the word "agent" in popular writing has cost the field years. So this book strips it back to engineering. The cognitive metaphors (perception, memory, reasoning) are useful as taxonomy, not as ontology. None of the systems described here perceive anything in the way a person does, and pretending otherwise produces both bad code and bad ethics.
A second note: the patterns here are deliberately model-agnostic. Where a specific large language model is mentioned, it's for concreteness, not endorsement. The shape of these architectures has been remarkably stable across three generations of frontier models, and there's no reason to expect that to change.
Throughout this book, substrate refers to the underlying technology layer an agent is built on: the model, the embedding model, the vector store, and the tool-execution environment beneath the agent's own code. Chapters 4A and 4B look at how that layer has been shifting. The substrate gets better, and the patterns persist.
Code samples in this book are schematic. They are written to make the pattern legible, not to drop into production.
Specifically:
Error handling is elided unless it's the point being made
Type hints are present but not exhaustive
Imports are at the top of each block but framework dependencies aren't pinned
Concurrency primitives are illustrative
And where a real production implementation would use a particular vendor SDK, the code here uses a placeholder
llm.call(...)ortool.invoke(...). You're expected to adapt these to your stack.
Read this book linearly if you're new to the field. Treat it as a reference if you're not. Each pattern is self-contained, and the cross-references at the end of each entry will lead you to its natural collaborators.
Foreword: Why Capabilities, Not Domains?
Every classification system is a hypothesis about how the world cleaves. Domain classification like "healthcare agents," "finance agents," "coding agents" embeds the hypothesis that the determining variable for how an agent is built is the industry it operates in.
This hypothesis was reasonable when agents were primarily prompt-engineering exercises wrapped around a single model call. But today, it's no longer reasonable.
Consider three agents from three industries: a clinical-decision-support agent, a credit-underwriting agent, and a code-review agent. Their prompts are extremely different. Their toolsets are extremely different. Their evaluation criteria are different. But their architectures, if you draw them, are nearly identical.
Each one perceives a complex document, decomposes it hierarchically, retrieves comparable cases from a curated memory, reasons via a self-consistency vote, attaches provenance to every claim it makes, escalates to a human at decision points the constitution flags, and audits every state-modifying action it takes.
Replace the prompt and the toolset and you've moved an agent across industries without changing its design.
The implication is practical: an engineer who has internalized the eight capabilities and the sixty patterns within them can build any of those three agents in a similar amount of time. An engineer who has memorized "how healthcare agents are built" has to relearn the work to move sideways. Capability literacy generalizes, while domain literacy does not.
The capability axis is also where the actual engineering decisions live. When you build a real agent, you don't lie awake at night deciding whether yours is "really a finance agent or a coding agent." You lie awake deciding whether your retrieval should be embedding-based or hybrid, whether your planner should produce a plan upfront or interleave with action, whether your safety enforcement should sit before or after the model call, or whether your memory should be flat or hierarchical.
These decisions are capability decisions. The catalog in this book is a vocabulary for naming them precisely and a record of the choices other engineers have made.
A final reason: the alignment chapter has nowhere to live in a domain taxonomy. Provenance, refusal calibration, off-switch compatibility, and drift detection aren't "the alignment chapter for healthcare agents and a separate alignment chapter for coding agents." They're the same patterns, applied to the same problems, and they belong in one place: adjacent to the patterns they compose with. The domain taxonomy hides this, but the capability taxonomy makes it visible.
What Domain Does Determine
The argument above is "capabilities are the primary axis." That's not the same as "domain is irrelevant." Domain shapes at least four things that capabilities alone don't capture, and a serious agent design has to address them up front:
First, regulatory constraints determine which alignment patterns are mandatory rather than optional. HIPAA forces Privacy-Preserving (57) into the structural core of a healthcare agent. SOX and equivalent regimes force Provenance Tracker (55) into financial-reporting agents. GDPR forces Persistent Identity (29) with deletion to be a first-class concern in any EU-touching deployment. A coding agent has none of these structural mandates and can ship with looser versions.
Next, the risk profile of mistakes ranges across orders of magnitude. A wrong-code commit is minutes-of-impact and easily reverted, but a wrong clinical recommendation can be years-of-impact and irreversible. A wrong trade is dollars-of-impact in seconds.
The risk profile sets the cost ceiling for alignment patterns. In low-risk domains, lighter patterns are sufficient, while in high-risk domains, more thorough composition is justified.
Evaluation harness shape is also domain-determined. Coding has formal correctness (does it compile, does it pass tests?). Medicine has expert-review-driven ground truth. Trading has market-reality feedback. Customer support has user-rating feedback. The available evaluation signal shapes which Learning patterns (Chapter 11) are even possible.
And finally, user-population characteristics shape Refusal Calibrator and Explainer requirements. An agent serving a professional audience (lawyers, doctors, engineers) can produce dense technical output, while one serving the general public has to behave very differently.
So: domain determines the non-negotiable alignment patterns, the cost envelope for everything else, the evaluation strategy, and the output register. Capabilities determine the architectural shape inside those constraints.
Both axes matter. And this book's contribution is that the capability axis has been under-served by previous treatments. The right design conversation is "given the domain's constraints, which capabilities does the agent need, and which patterns within each."
Who This Book is For
This book is written for the engineer who has built one agent and now needs to build twenty. It assumes you can write Python, you have used a frontier language model from an SDK, and you have at least felt the pain of an agent silently going off the rails in production.
It doesn't assume a background in cognitive science, control theory, or formal logic, though readers with those backgrounds will recognize their fingerprints throughout.
The book is also useful for:
Technical leaders making build-versus-buy decisions about agent-shaped features. The chapter intros are written at a level that is digestible without code, and the pattern taglines are sharp enough to use as criteria during product scoping.
Product managers scoping agent-shaped features. Every pattern's case study is written in product terms. You can read those alone to understand what each architecture enables.
Security and compliance reviewers evaluating agent deployments. Chapters 9 (Tool Use) and 12 (Alignment) are written with the reviewer's questions in mind, and the failure-mode discussions name the specific risks each pattern introduces or mitigates.
Researchers looking for a working taxonomy of the practitioner-facing literature. The book is opinionated about naming and structure in ways that should make it citable as a stake in the ground.
The book is not for readers looking for a beginner's tour of large language models, a course in machine learning, or a survey of agent products on the market. Those resources exist elsewhere and are better than anything a chapter here could fit.
How to Read This Book
Part I covers the substrate: the four chapters that establish the model, framework, prompting, and operational concerns shared by every agent in the book. None of it is agent-specific, and an experienced engineer can skim it in a single sitting.
Skip it if you're confident your foundations are solid, but read the gateway pattern at the end of Chapter 4 even then. It's the highest-leverage piece of infrastructure most teams skip.
Part II is the catalog: eight chapters, one per capability, each containing seven or eight distinct agent patterns. The chapters can be read in any order. Each pattern entry follows the same internal structure (tagline, problem, naïve fixes, mechanism, code skeleton, trade-offs, failure modes, case study, neighbors).
The structure is deliberate: the same fields, the same headings, in the same order, every time. Once you've read three entries you've internalized the format and can read any other entry by skimming.
Part III covers composition: how patterns combine into real systems, how to evaluate the result, and how the composition itself fails. Read it after you've at least skimmed Part II.
The epilogue argues for what comes next — capability composition as the frontier — and is short enough to read on a coffee break.
A note on the code. Every pattern has a Python skeleton. Read the skeletons. The prose tells you what the pattern does and the code tells you what the pattern is.
They aren't redundant. Patterns that look interchangeable in prose often have very different code, and patterns that look different often have nearly identical code with different framing. The code is the ground truth.
Table of Contents
Front Matter
Prologue
Part I — Foundations
Part II — The Eight Capabilities (60 patterns)
Chapter 5 — Perception: Turning Signals into Percepts (7 patterns)
- Agents 1–7
Chapter 6 — Reasoning: Inferring Beyond the Given (8 patterns)
- Agents 8–15
Chapter 7 — Planning: From Goal to Sequenced Action (7 patterns)
- Agents 16–22
Chapter 8 — Memory: Persistence Across Time (7 patterns)
- Agents 23–29
Chapter 9 — Tool Use: Reaching Outside the Model (8 patterns)
- Agents 30–37
Chapter 10 — Coordination: Many Minds, One Outcome (8 patterns)
- Agents 38–45
Chapter 11 — Learning: Becoming Better at What It Does (7 patterns)
- Agents 46–52
Chapter 12 — Alignment: Behaving by Design, Not by Accident (8 patterns)
- Agents 53–60
Part III — Composition
Part IV — Operating Agents in Production
Epilogue — The Capability-Composition Frontier
Appendices
About and Further Reading
Chapter 0 — Should This Be an Agent at All?
The single most important chapter in this book is the one that argues against using anything in the rest of it.
Agent framing is intellectually fashionable. It's also, for a large fraction of the problems it gets applied to, the wrong frame.
Most things that get scoped as "agent use cases" are better solved by simpler architectures: a static prompt, a deterministic workflow, a small piece of glue code around an existing tool, or an outright "no, this isn't ready to be automated yet."
Before reaching for any of the sixty patterns in this book, ask whether you should be building an agent at all.
0.1 The Four-Level Ladder
For any candidate problem, place it on this ladder, from cheapest to most complex:
A static prompt: One model call, one prompt template, no tools, no memory. Input goes in, and output comes out. The simplest possible thing.
A deterministic workflow: Multiple model calls or model+tool steps, but the sequence is fixed: step A, then step B, then step C, then done. The model produces content and the harness controls the flow. No agent decisions about what to do next.
A bounded agent: The model decides which tool to call next, but within a small fixed toolset and a small step budget. Closer to a smart script than to an autonomous system.
A full agent: The model holds a goal across many steps, decides actions, manages memory, recovers from failures, and operates at a level of autonomy that genuinely warrants the term "agent."
The right level for any problem is the lowest one that solves it. The book's patterns are mostly for level 3 and level 4. If level 1 or level 2 solves your problem, the patterns are overhead.
0.2 Heuristics for Picking the Right Level
Pick level 1 (static prompt) when:
The input fits comfortably in one model call.
The output structure is fully specified by the prompt.
There's no need for tools that change state, no need for memory across calls.
A wrong output is recoverable by re-prompting.
Examples that should be level 1: most summarization, most translation, most format conversion, most "write me a draft of X," most classification, most extraction-from-known-shape, most rewording.
Pick level 2 (deterministic workflow) when:
The problem decomposes into a fixed sequence of steps.
Each step has a well-defined input and output.
The sequence doesn't vary by input. The content varies but the flow doesn't.
You can write the flow as a flowchart that fits on a napkin.
Examples that should be level 2: most content pipelines (research → draft → fact-check → format), most data-enrichment workflows (parse → normalize → enrich → store), most form-processing pipelines, most "extract X then look up Y then summarize."
Pick level 3 (bounded agent) when:
The right next step depends on what the previous step returned.
The number of distinct possible sequences is large but the toolset is small (say, under 15 tools).
The step budget is small (under 20 steps for a normal session).
Wrong actions are easily reversed.
Examples that fit level 3: customer-support ticket triage with a defined toolset, SQL question-answering against a known schema, ticket-routing-with-disambiguation, per-document analysis with a small standard set of operations.
Pick level 4 (full agent) when:
The problem genuinely requires holding a goal across long horizons.
Multiple specialists may need to coordinate.
Memory across sessions matters.
The toolset is large or dynamic.
Failure modes need first-class handling (rollback, replanning, escalation).
The stakes warrant the investment.
Examples that fit level 4: a research analyst that drafts reports across hours of operation, a workflow-automation agent acting on production systems, a code agent that submits pull requests, a long-running monitoring agent.
0.3 The Five Questions to Ask Before Building an Agent
Before committing to level 3 or level 4, force yourself through these five questions. If you can't answer them, you aren't ready to build the agent.
What does success look like, measurably? If your only criterion is "users like it," you don't have a goal. Pick a metric you can measure on day one, like completion rate, escalation rate, accepted-output rate, time-to-resolution, and commit to it.
What does failure look like, in production? What does the worst case do to your users, your data, and your bill? If you can't describe the worst case, you can't bound its blast radius, and you shouldn't give the agent permission to act.
What is the cost ceiling per session, and is the agent's value above it? A level-4 agent with a full pattern stack costs many multiples of a single model call. If the user-perceived value of a session is below the cost of the session, the agent doesn't have a viable business model regardless of how well it works.
What does the evaluation harness look like? Not "we will figure this out later." If you haven't specified the labeled set you'll use to measure quality, you'll ship without measuring quality, and you won't know when something breaks.
What does the off-switch look like? Who can stop the agent, how fast, with what state preservation, and with what rollback semantics? If the answer is "we will add this later," you haven't finished designing the agent.
A team that can't answer all five shouldn't be at level 3 or level 4. Drop down a level and ship something simpler that works.
0.4 Common Mistakes in Picking the Level
There are tree patterns of misallocation that recur across teams the author has reviewed:
Pattern 1: Agent-as-marketing.
The product team wants the word "agent" in the press release. The engineering team builds an agent for what should have been a workflow. The result is more expensive, slower, and less reliable than the workflow would have been, with no offsetting user benefit.
The cure is to separate the engineering decision (what level is right) from the product positioning (what the marketing copy says). They're different problems.
Pattern 2: Premature autonomy.
The team builds a level-4 agent before they have a level-1 or level-2 version working. Without the simpler version, they can't tell whether the agent's complexity is adding value or hiding bugs.
The cure is to ship the simpler version first: build the agent if and only if the simpler version's failure mode demonstrably warrants it.
Pattern 3: Sunk-cost escalation.
A team built an agent six months ago. It works at 60% of the desired quality. The team keeps adding patterns from the catalog, hoping the next one will close the gap.
The right move is sometimes to drop the agent framing entirely and reach for a different architecture (a workflow, a constrained-search system, or a hand-coded heuristic). The pattern catalog can become a trap when used to defer the harder question of whether the agent framing is right at all.
0.5 If the Answer is "Yes, This Should Be an Agent"
Then the rest of the book applies. The pattern catalog is your design vocabulary, Part III is your composition discipline, and the alignment chapter is your structural-safety floor.
Build deliberately, evaluate the composition, keep the off-switch responsive, and revisit Section 0.3 every six months. The answer to "should this still be an agent?" can change as the substrate, the costs, and the deployment context change.
The rest of this book assumes you have correctly answered "yes." If you got that decision wrong, no amount of pattern composition rescues the outcome.
Part I — Foundations
Chapter 1 — The Agent Substrate
An agent is a program with three properties: it observes an environment, it maintains some persistent state across observations, and it emits actions whose effects on that environment feed back into its next observation.
The interesting word in that sentence is environment. For the agents in this book, the environment is almost never the physical world. Instead, it's a software surface: an API, a database, a web page, a filesystem, a chat history, or a stream of events. Treating the environment as a software surface is what makes agent engineering tractable. Treating it as a fuzzy social or physical reality is what makes agent engineering pseudoscience.
1.1 The observation-action loop
The simplest agent is a loop:
def run_agent(goal: str, env: Environment, max_steps: int = 50) -> Result:
state = State(goal=goal, history=[])
for step in range(max_steps):
observation = env.observe()
state.history.append(observation)
action = policy(state) # the LLM-driven choice
if action.type == "terminate":
return Result(success=True, state=state)
outcome = env.act(action) # mutates the world; returns observation-like
state.history.append(outcome)
return Result(success=False, state=state, reason="step_budget_exhausted")
This is the entire abstraction. Every agent in the book is a refinement of this loop. The refinements take the form of:
Replacing the policy: From a single model call to a planner, a debate, a constraint solver, or a composition of all three.
Replacing the state: From a flat history to typed memories, hierarchical plans, belief distributions, or skill libraries.
Replacing the environment: From a single tool to a curated toolset, a sandboxed shell, a browser, a multi-agent surface, or a human-in-the-loop.
Replacing the termination condition: From step-budget exhaustion to goal-check verification, plan-completion, constitutional refusal, or operator override.
The discipline of this book is that each replacement is named: it gets a pattern, a code shape, a failure profile, and a case study. There's no such thing as a generic "more sophisticated agent." There are agents with specific patterns in specific slots of the loop.
1.2 Policy versus tool
The distinction between policy and tool is the most-confused boundary in agent engineering. The policy is the deciding component. It reads the state and chooses what to do next. The tool is the acting component. It carries out the chosen action against the environment. The two are not the same and should never share an implementation.
A policy without tools is a chatbot. A tool without a policy is a function call. An agent is the combination, mediated by a loop. Every pattern in this book either modifies the policy, modifies the tool surface, or modifies the loop that combines them — never all three simultaneously, because patterns that modify all three are usually two patterns in a trench coat.
class Policy(Protocol):
"""Reads state, returns the next action."""
def __call__(self, state: State) -> Action: ...
class Tool(Protocol):
"""Executes one action, returns the outcome."""
name: str
description: str
parameters: dict # JSON Schema for arguments
def invoke(self, args: dict) -> Outcome: ...
These two interfaces are the type signature of agent engineering. If your code doesn't cleanly separate them, or something equivalent, you'll end up building the separation anyway, under pressure, the first time a policy change and a tool change collide in the same bug.
1.3 The role of the planner
The policy in a sophisticated agent is rarely a single model call. It's typically a planner that produces a multi-step plan and an executor that runs the plan. The split matters because the failure modes of planning are different from the failure modes of execution.
A planner fails by being wrong about the world. It produces a plan whose steps don't connect, don't respect the constraints, or don't lead to the goal. An executor fails by mis-binding parameters, mis-handling tool errors, or failing to detect that the plan has gone off the rails. Treating these as the same component conflates the failures and makes neither addressable.
class Planner(Protocol):
def plan(self, goal: Goal, state: State) -> Plan: ...
class Executor(Protocol):
def run(self, plan: Plan, state: State, env: Environment) -> ExecutionResult: ...
class Agent:
def __init__(self, planner: Planner, executor: Executor):
self.planner = planner
self.executor = executor
def run(self, goal: Goal, env: Environment) -> Result:
state = State(goal=goal)
while not state.terminated:
plan = self.planner.plan(goal, state)
outcome = self.executor.run(plan, state, env)
state = state.update(outcome)
if outcome.replan_required:
continue # the executor noticed the plan was wrong
if outcome.complete:
state.terminated = True
return Result(state=state)
This split is the topic of Chapter 7. The patterns in that chapter (Hierarchical Decomposer, Tree-of-Thought, Plan-Then-Execute, Adaptive Replanner, and Backward Goal-Regression) are all variations on which side of the split does which work.
1.4 In-context state versus persistent memory
The state visible to a policy at a given moment is the union of two things: the in-context state (what is in the prompt, including tool results) and the persistent memory (what is stored in some external store the agent can read from and write to).
The mistake to avoid is conflating them. In-context state is volatile, expensive, and limited in size by the model's context window. Persistent memory is durable, cheap to expand, and limited only by what you choose to retain.
The patterns in Chapter 8 (Episodic Buffer, Semantic Curator, Working-Memory Manager, Forgetting Policy, Memory-of-Self, Vector-Store Curator, Persistent Identity) exist to manage the boundary between these two, and they all assume the boundary is explicit.
@dataclass
class Memory:
in_context: list[Message] # current prompt content
episodic: EpisodicStore # event log
semantic: SemanticStore # promoted facts
skills: SkillLibrary # learned procedures
self_model: SelfModel # what the agent thinks it is
def compose_prompt(self, step: Step) -> list[Message]:
"""The Working-Memory Manager (Agent 25) lives here."""
...
The act of composing the prompt for each step is itself an agent pattern (the Working-Memory Manager, Agent 25). Most teams discover this only after building one agent without it and watching context costs spiral.
1.5 Deterministic harness, stochastic policy
A useful invariant: the harness is deterministic, the policy is stochastic. The loop, the executor, the memory layer, the tool layer, the observability layer are all deterministic Python that you wrote. The policy is the part that calls a large language model and gets a non-deterministic answer.
This separation matters for two reasons. First, it confines the non-determinism to a single point. When something goes wrong, you can rerun the harness against a recorded policy output and reproduce the failure exactly. Second, it makes the policy substitutable. You can swap a frontier model for a smaller one, a single-shot call for a self-consistency vote, an API call for a local model, or an entire model for a deterministic stub during testing — without rewriting the rest of the system.
class RecordedPolicy:
"""For replay debugging: deterministic substitute for an LLM-backed policy."""
def __init__(self, recording: list[Action]):
self.recording = list(reversed(recording))
def __call__(self, state: State) -> Action:
return self.recording.pop()
# Production
agent = Agent(
policy=LLMPolicy(provider="<your-provider>", model="<your-model>"),
tools=production_tools,
memory=production_memory,
)
# Debugging an incident
trace = load_trace(incident_id="incident-2026-04-19-0034")
replay_agent = Agent(
policy=RecordedPolicy(trace.actions),
tools=production_tools,
memory=production_memory,
)
result = replay_agent.run(trace.goal, trace.env_snapshot)
assert result.failure == trace.failure # the bug reproduces
If your agent code doesn't admit this substitution, your debugging story is much worse than it has to be.
1.6 The five canonical failure modes
Every pattern in the book is, in some sense, a response to one or more of five canonical failure modes. They appear so often, across so many otherwise unrelated systems, that they deserve names. The names recur throughout the book:
Looped reasoning: The agent thinks-acts-thinks-acts forever without progress. This is caused by the policy proposing actions that don't change the state in a way the policy can perceive. You can address it with the bounded ReAct loop (Agent 17), the Adaptive Replanner (Agent 20), and any plan-based pattern that maintains an explicit progress measure.
Tool spoofing: The agent is talked into calling a tool against the wrong target, with the wrong arguments, or under the wrong context. It's caused by input the model treats as instruction when it should treat as data. You can address it with the Constitution-Bound Agent (Agent 53), the Side-Effect Auditor (Agent 37), and structural input/instruction separation in the prompt architecture.
Context exhaustion: The agent loses track of its goal in the middle of a long session because the goal has scrolled out of context. It's caused by treating the context window as if it had infinite memory semantics. You can address it with the Working-Memory Manager (Agent 25), the Hierarchical Decomposer (Agent 16), and per-step prompt composition.
Goal drift: The agent gradually pivots from the original objective to a related but different one. It's caused by the policy interpreting intermediate results as if they were the goal. You can address it with the Plan-Then-Execute pattern (Agent 19), the Drift Detector (Agent 59), and any pattern that maintains an explicit goal-check separate from the policy.
Silent success on the wrong task: The agent confidently completes a task adjacent to the one it was asked. It's caused by the policy "rounding the user's intent" to something it knows how to do. You can address it with the Chain-of-Thought Auditor (Agent 8), the Reflection Agent (Agent 47), and verification patterns that compare the output to the input rather than to itself.
When something goes wrong in production, the first question is which of the five it is. The second question is which patterns the agent doesn't yet have for that failure class.
1.7 A reference harness
The chapter closes with a working reference implementation in roughly three hundred lines of Python. Every later pattern in the book is described as a modification of, or addition to, this harness.
# agents/harness.py — the canonical reference implementation
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Protocol, Callable, Optional
# ---- Core types ---------------------------------------------------------------
@dataclass
class Observation:
source: str # tool name or environment channel
payload: dict
timestamp: float
@dataclass
class Action:
type: str # "tool_call" | "terminate" | "ask_human" | ...
tool: Optional[str] = None
args: dict = field(default_factory=dict)
rationale: str = ""
@dataclass
class Outcome:
observation: Observation
error: Optional[str] = None
@dataclass
class State:
goal: str
history: list = field(default_factory=list) # interleaved Observations/Actions
memory: "Memory" = field(default_factory=lambda: Memory())
terminated: bool = False
failure_reason: Optional[str] = None
@dataclass
class Memory:
episodic: list = field(default_factory=list)
semantic: dict = field(default_factory=dict)
self_model: dict = field(default_factory=dict)
# ---- Protocols ----------------------------------------------------------------
class Tool(Protocol):
name: str
description: str
parameters: dict
def invoke(self, args: dict) -> Outcome: ...
class Policy(Protocol):
def __call__(self, state: State, tools: dict[str, Tool]) -> Action: ...
class Observer(Protocol):
"""Observability hook called on every loop event."""
def on_action(self, state: State, action: Action) -> None: ...
def on_outcome(self, state: State, outcome: Outcome) -> None: ...
def on_terminate(self, state: State) -> None: ...
# ---- The harness --------------------------------------------------------------
@dataclass
class Harness:
policy: Policy
tools: dict[str, Tool]
observers: list[Observer] = field(default_factory=list)
max_steps: int = 50
goal_check: Optional[Callable[[State], bool]] = None
def run(self, goal: str) -> State:
state = State(goal=goal)
for step in range(self.max_steps):
action = self.policy(state, self.tools)
for obs in self.observers:
obs.on_action(state, action)
state.history.append(action)
if action.type == "terminate":
state.terminated = True
break
outcome = self._execute(action)
for obs in self.observers:
obs.on_outcome(state, outcome)
state.history.append(outcome.observation)
if self.goal_check and self.goal_check(state):
state.terminated = True
break
else:
state.failure_reason = "step_budget_exhausted"
for obs in self.observers:
obs.on_terminate(state)
return state
def _execute(self, action: Action) -> Outcome:
if action.type != "tool_call":
return Outcome(observation=Observation(
source="harness", payload={"action_type": action.type}, timestamp=0.0))
tool = self.tools.get(action.tool)
if tool is None:
return Outcome(
observation=Observation(source="harness", payload={}, timestamp=0.0),
error=f"unknown_tool:{action.tool}")
try:
return tool.invoke(action.args)
except Exception as e:
return Outcome(
observation=Observation(source=action.tool, payload={}, timestamp=0.0),
error=f"tool_exception:{type(e).__name__}:{e}")
If you can hold this harness in your head, you can hold the rest of the book in your head. Every pattern in Part II is a refinement, replacement, or extension of one of its components.
Chapter 2 — The Engineer's Toolkit
The framework wars are over and nobody won. LangChain, LlamaIndex, AutoGen, CrewAI, DSPy, Haystack, Pydantic-AI, and the half-dozen serious in-house frameworks at the large labs all converge on the same five abstractions: a model client, a tool registry, a prompt template system, a memory interface, and an orchestration loop. They differ on which abstraction they make most pleasant and which they make most painful.
This chapter walks through those trade-offs without partisanship and gives a decision rubric for picking one. Or, more often, for picking none and building the five abstractions yourself in a few hundred lines.
2.1 The five abstractions every framework converges on
When you strip a framework down to its load-bearing components, you find these five:
Model client: A typed interface to one or more LLM providers, with the parts that matter for agents (function-calling, structured output, streaming, prompt-caching, retry, rate-limit handling) actually exposed. Frameworks differ on whether the client is leaky (you see the provider's quirks) or capping (you see a least-common-denominator interface).
Tool registry: A catalogue of tools the policy can choose from, with structured descriptions, typed parameter schemas, invocation semantics, and (in the better frameworks) per-tool middleware for logging, retry, and authorization.
Prompt template system: A way to compose prompts from invariant pieces, role-specific pieces, task-specific pieces, and dynamically-retrieved pieces. The frameworks that get this right treat prompts as versioned artifacts. The ones that don't treat prompts as string concatenations.
Memory interface: A surface for reading and writing episodic events, semantic facts, retrieved documents, and prior conversations. Frameworks differ wildly on how opinionated this is, from "you decide" to "here is one giant vector store, use it."
Orchestration loop: The actual run-the-agent loop. Frameworks differ on whether this is a fixed loop with hooks (LangChain's AgentExecutor) or a graph engine (LangGraph), or a debate harness (AutoGen), or a typed pipeline (DSPy).
If you understand these five, you can read any framework's source in an afternoon. You can also decide whether to use one. The decision rubric is: do you need to ship in two weeks (use a framework), or do you need to operate this for years (build the five abstractions, even if they sit on top of a framework as a thin internal layer)?
2.2 Building the five abstractions yourself
Here's what the minimal-but-real version looks like. It's roughly two hundred lines and avoids every common mistake.
# toolkit/client.py
from __future__ import annotations
from dataclasses import dataclass
from typing import Optional, Any
@dataclass
class LLMResponse:
text: str
tool_calls: list[dict]
finish_reason: str
usage: dict # tokens in/out, cost cents
class LLMClient:
"""Thin wrapper that normalizes provider quirks AND exposes them when needed."""
def __init__(self, provider: str, model: str, defaults: dict | None = None):
self.provider = provider
self.model = model
self.defaults = defaults or {}
self._native = _load_provider(provider)
def call(self, messages: list[dict], *, tools: list[dict] | None = None,
schema: dict | None = None, **kwargs) -> LLMResponse:
params = {**self.defaults, **kwargs}
# Normalize tool-calling shape across providers.
# Honor structured-output schemas via the right native mechanism.
# Apply prompt caching where supported.
raw = self._native.call(self.model, messages, tools=tools, schema=schema, **params)
return _normalize(raw, self.provider)
The key word in that file is normalizes. The provider differences matter for half the things and don't matter for the other half. Pinning them all behind a least-common-denominator interface looks clean and is wrong. Agents need access to provider-specific features (prompt caching with Anthropic, structured outputs with OpenAI, tool-use modes with Bedrock). The toolkit's job is to expose them when needed and to keep callers from depending on them when not.
# toolkit/registry.py
from dataclasses import dataclass
from typing import Callable
@dataclass
class ToolSpec:
name: str
description: str
parameters: dict # JSON Schema
invoke: Callable[[dict], Any]
metadata: dict # cost, latency, side-effect class, owner
class ToolRegistry:
def __init__(self):
self._tools: dict[str, ToolSpec] = {}
def register(self, spec: ToolSpec) -> None:
if spec.name in self._tools:
raise ValueError(f"duplicate tool: {spec.name}")
self._tools[spec.name] = spec
def select(self, query: str, k: int = 10) -> list[ToolSpec]:
"""Tool Selector (Agent 30) lives here."""
return _embedding_retrieve(self._tools, query, k)
def describe_for_prompt(self, names: list[str]) -> list[dict]:
return [
{"name": self._tools[n].name,
"description": self._tools[n].description,
"parameters": self._tools[n].parameters}
for n in names
]
# toolkit/prompt.py
@dataclass
class PromptTemplate:
"""Four-layer prompt architecture: invariant, role, task, frame."""
invariant: str # never changes; cached
role: str # changes per agent role
task: str # changes per task
frame: str # changes per call (RAG, working memory, etc.)
version: str
def render(self, **kwargs) -> list[dict]:
return [
{"role": "system", "content": self.invariant.format(**kwargs)},
{"role": "system", "content": self.role.format(**kwargs)},
{"role": "system", "content": self.task.format(**kwargs)},
{"role": "user", "content": self.frame.format(**kwargs)},
]
The four-layer split is not cosmetic. Each layer has a different change cadence and a different cacheability profile. Treating them as one string conflates them and loses both maintainability and (with providers that support prompt caching) money.
# toolkit/memory.py
class MemoryStore:
"""Pluggable backend; the interface stays the same."""
def write(self, namespace: str, key: str, value: dict, ttl: int | None = None) -> None: ...
def read(self, namespace: str, key: str) -> dict | None: ...
def search(self, namespace: str, query: str, k: int = 10) -> list[dict]: ...
def delete(self, namespace: str, key: str) -> None: ...
# toolkit/loop.py
class AgentLoop:
def __init__(self, *, policy, registry, memory, observers):
self.policy, self.registry, self.memory, self.observers = (
policy, registry, memory, observers)
def run(self, goal: str, max_steps: int = 50) -> State:
# The reference harness from Chapter 1, plumbed with these abstractions.
...
These five files plus the Chapter 1 harness give you a real toolkit in under 400 lines of code. It's missing nothing that production frameworks have for production-grade work. But it's missing many things that they have for novice users, which is a different problem.
2.3 The components that aren't optional
Beyond the five abstractions, there are concerns no agent in production should be built without:
Vector stores and the embedding lifecycle: This is the topic of Agent 28 in detail. For the toolkit level, treat the vector store as a first-class store with its own lifecycle (ingestion, re-embedding, sharding, eviction), not as a magic "memory" that you write to and forget.
Structured-output enforcement: When the model is supposed to produce JSON, don't parse free text. Use the provider's structured-output mode, validate against a JSON Schema, and reject-and-retry on failure. The retry should be parameterized: if a JSON Schema is failing repeatedly, the schema is wrong, not the model.
Evaluation harnesses: You won't pick the right model, the right prompt, or the right pattern combination without one. Build it first. It doesn't have to be sophisticated: a YAML file with cases, a function that runs them, and a pass/fail rate gets you eighty percent of the value.
Prompt-version control: Every prompt the agent uses is a versioned artifact with a name, a version, and a hash. When a bug shows up in production, you can attribute it to the exact prompt revision that produced it.
Secret management for tool credentials: Tools call APIs. APIs need credentials. The credentials shouldn't be in the prompt, in the trace, or in the agent's working memory. They live in a secret manager, are fetched at tool-invocation time, and never appear in any artifact the agent persists.
Observability stack: Traces, span hierarchies, prompt diffs, tool-call inspection. The minimum bar is per-step tracing with structured data, and the higher bar is replay of any historical session.
2.4 Model selection
The rule is simple: you can't pick the right model until you have a working evaluation harness, so build the harness first. Every other selection heuristic, like price-per-token, context window, function-calling support, or vendor stability, matters but is downstream of the evaluation.
Build twenty cases that represent your deployment distribution, run them against three candidate models, look at pass-rate and cost-per-pass, and decide.
A practical wrinkle: the right model often varies by step within a single agent. A small,
