Chapter 4 · Tools

Choosing the Form of Capability Expression: Dedicated Tools vs. Skills + General Executors#

Before discussing specific tool types, we must first answer a more fundamental design question: in what form should an Agent’s capabilities be expressed? The sections that follow discuss tool granularity, generality, and the art of description, but all of that rests on one assumption—that the capability should become a dedicated tool. In fact, an Agent’s capabilities can take two basic forms:

• Dedicated Code Tools: Structured function calls—deterministic and testable, but each tool costs hundreds of tokens, and a growing roster breaks the KV Cache.
• Skills + General Executors: Skill documents written in natural language describe the operational workflow, which the Agent executes via a terminal or code interpreter. This requires only a small number of general tools to cover a wide range of scenarios (as Chapter 5 will argue with seven core tools).

For example, a Skill document for “deploying an application” might read: 1. Run npm run build to build the project; 2. Run docker build -t app:latest . to package the image; 3. Run kubectl apply -f deploy.yaml to deploy to the cluster—the Agent executes these instructions step-by-step using a bash tool, without needing a dedicated tool for each step.

Choosing between these forms depends on three dimensions.

• Parameter Complexity: For operations involving nested objects, multi-field joint validation, or complex type constraints, the structured schema of a dedicated tool better guides the model to pass parameters correctly; for operations with simple parameters, passing them via CLI commands is equally reliable.
• Frequency of Change: Frequently changing capabilities are far cheaper to maintain as Skills—editing a passage of text beats changing code, re-testing, and redeploying. Stable, low-level operations belong in dedicated tools.
• Model Capability: State-of-the-art (SOTA) models can express more capabilities and reduce the number of tools using the Skills + General Executors approach; weaker models require structured tool schemas to guide correct invocation. Chapter 8 will discuss how an Agent makes the same choice when consolidating new capabilities during self-evolution.

Trade-offs in Tool Granularity: Integration vs. Separation#

Tool granularity is a critical decision point. Too fine, and tools proliferate, adding to the LLM’s selection burden; too coarse, and each tool grows unwieldy. Once the count gets too high (say, past 100), even the most advanced language models start picking the wrong tool.

The core criteria for deciding whether to integrate are functional similarity and overlap in usage scenarios. Taking document processing as an example, tools like extract_pdf_text, extract_docx_content, and extract_pptx_content share one job: extracting text from a document—input a file path, output a string. A better design is to provide a unified read_document tool, distinguishing formats via a file_type parameter. Integration reduces the LLM’s cognitive load (it only needs to understand the simple rule “use read_document to read documents”), makes descriptions clearer, and facilitates extensibility (supporting a new format only requires adding a file_type option). Not all tools should be integrated—for example, image parsing (OCR) and video parsing (keyframe extraction), while both being “content extraction,” have vastly different parameter forms and latency characteristics; forcing them together would blur the interface semantics.

When functions are similar but have very different parameter sets, or when a particular function is used extremely frequently, keeping them separate is more reasonable.

Designing for Tool Generality#

General tools are preferable to dedicated tools, unless there is a clear security, permission, or performance reason—for example, code_interpreter saves more tokens and is more flexible than a dozen specialized calculators, but in scenarios involving writes to a production database, a dedicated tool can provide finer-grained permission control and audit trails. Returning to the calculation example: instead of providing a four-function calculator, it’s better to provide a general code_interpreter tool, pre-installed with libraries like sympy, numpy, and pandas in a sandboxed environment (a secure execution space isolated from the host, where code cannot affect external systems), allowing the Agent to perform any mathematical computation by executing Python code.

The logic behind this principle: an LLM already possesses powerful reasoning and code-generation abilities; leverage them rather than constrain them. A general tool hands the Agent a “meta-capability”—a single Python interpreter replaces dozens of single-purpose tools and handles the edge cases nobody anticipated.

However, generality has its limits. For operations requiring special permissions, complex configuration, or posing security risks, well-encapsulated dedicated tools are still necessary. For example, the syntax for grep differs across Mac, Windows, and Linux; providing a dedicated grep tool is better than letting the Agent improvise.

The Art of Tool Description#

The quality of a tool’s description directly determines the accuracy with which an Agent uses it.

The core of a tool description is to let the LLM know “when to use it,” not just “what it can do.” Taking web search as an example, saying “Search for relevant content” is far less effective than saying “Use when needing to obtain real-time information or find unknown facts”—the former merely describes the function, while the latter helps the LLM make an invocation decision.

Boundaries are equally important. A file search tool should explicitly state that it can only match based on file names, not search file contents—if such negative examples are missing, the LLM will guess. Clearly listing a tool’s boundary conditions—what it cannot do, what input it does not accept—is often more important than describing its capabilities, because the root cause of most tool call failures is not that the model doesn’t know what the tool can do, but that it doesn’t know what the tool cannot do.

Parameter descriptions should use concrete examples instead of abstract specifications. “timestamp: RFC3339 format, e.g., 2024-03-15T14:30:00Z” is far more effective than “RFC3339 format” alone. An LLM focused on a single problem can parse such terms; but mid-task—juggling multiple tools, mining the trajectory history, weighing decisions—it spares only a sliver of attention for parameter formats, and errors creep in. Similarly, don’t write “phone: Use E.164 format,” but rather “phone: Phone number, use E.164 format (country code + number, no spaces or special characters), e.g., +8613888888888 (China) or +12025551234 (USA).” These concrete examples allow the Agent to apply them directly without an extra reasoning step.

Return values also need description—“Returns a JSON array, each element containing three fields: title, url, snippet”—such explanations reduce errors during subsequent parsing. For time-consuming tools, noting the execution cost helps the LLM plan the invocation order reasonably, e.g., “This tool needs to download the entire webpage; large websites may take 5-10 seconds. If only metadata is needed, consider using get_page_metadata.”

Beyond describing parameters and return values item by item, a further step is to include 1-5 real invocation examples for each tool. JSON Schema (a specification for describing JSON data structures, defining the type, constraints, and description of each field) can only describe parameter types, but cannot express invocation patterns or typical parameter combinations—such as whether timestamps are in seconds or milliseconds, or how filter conditions are nested—these implicit conventions are best conveyed through examples. Adding examples often significantly improves tool call accuracy—in some benchmarks, from about 72% to 90% (exact figures vary by task).

A practical debugging principle: when an Agent keeps picking the wrong tool, check the tool descriptions first rather than doubting the model. Most tool selection errors trace back to inaccurate descriptions—unclear boundaries, missing negative examples, ambiguous parameter meanings. Fixing the descriptions usually pays far better than switching to a stronger model.

Fidelity of Parameter Passing#

A more insidious anti-pattern than missing functionality is silent input transformation—where the tool quietly “corrects” the model’s input parameters before execution, causing the actual operation to deviate from the model’s intention.

Consider a version of Cursor from early 2026. Its edit tool accepts old_string and new_string parameters and performs an exact match-and-replace in a file. However, the tool’s parameter passing layer silently converts Chinese curly quotes (\u201c and \u201d) to English straight quotes ("). The result is a failure mode that leaves the model utterly confused: reading the file, the model sees text containing curly quotes (the read tool returns them unchanged, without conversion), so it passes them verbatim to the old_string parameter of the replace tool. But the parameter passing layer has already converted the curly quotes to straight quotes, which don’t match the actual content in the file, causing the tool to return “no match found.” The model tries repeatedly and fails repeatedly—it cannot understand why the tool can’t find what it clearly saw.

The same problem occurs in the write direction. When the model calls a file writing tool, intending to write curly quotes (the correct choice for Chinese typography), the parameter passing layer silently replaces them with straight quotes. The model thinks it has written content conforming to Chinese typographic standards, but the actual content in the file has been tampered with. If the model then reads the file to verify the written result, it sees the converted straight quotes, leading to confusion.

Another type of fidelity violation is silent parameter injection—where a tool appends extra parameters to a command without the model’s knowledge. For example, a bash tool in an IDE automatically adds an extra parameter (to mark the commit as AI-generated) to every git commit command. If the user’s Git version is older and doesn’t support this parameter, the silently injected parameter causes git commit to fail. The model might repeatedly adjust the commit message wording or try different parameter combinations, but it will fail no matter what.

These issues reveal a more fundamental tool design principle: there must be no systematic discrepancy between the world the model perceives and the world the tool operates on. Tool parameter passing must remain transparent; inputs or outputs must not be modified without the model’s knowledge. If input normalization is necessary (e.g., unifying encoding formats), it must be documented in the tool description and explicitly communicated to the model in the tool’s return. Otherwise, the tool’s “smart corrections” don’t help the model but instead create a systemic failure that the model cannot diagnose on its own.

The Evolution of Tool Design#

Looking at the development of tool design, it has roughly gone through three stages. First-generation tools were direct API wrappers—mapping each API endpoint to a tool, resulting in overly fine granularity where an Agent often had to coordinate multiple tools to accomplish a single goal. Second-generation tools are based on the ACI (Agent-Computer Interface) principle discussed in this section—tools should correspond to the Agent’s goals rather than underlying API operations. The granularity trade-offs, generality design, and description specifications mentioned earlier all belong to this stage. ACI is a concept proposed in analogy to HCI (Human-Computer Interaction)—if HCI studies how humans interact with computers, ACI studies how Agents interact with computers, with the core focus on making tools friendly to Agents, not humans.

Third-generation tools, building on the design of individual tools, further optimize how tools are invoked, chained, and discovered, addressing three separate questions. “How are tools accurately invoked?” is solved by example-driven invocation (introduced earlier in “The Art of Tool Description”). “How are tools discovered?” is solved by dynamic tool discovery—no longer injecting all tool definitions into the context at once (detailed in this chapter’s “Proactive Tool Discovery” section). “How are tools chained?” is solved by code orchestration execution—for complex tasks requiring chaining multiple tools, the model uses code to orchestrate the call sequence. As an analogy: the traditional approach is like emailing your boss after every step and waiting for a reply telling you what to do next—each round-trip “email” is token consumption. Code orchestration is like the boss writing the complete operation manual up front; you follow it and report back only when everything is done. Specifically, the LLM generates a script in one go, intermediate variables remain in the code execution environment, and only the final result is returned to the LLM. For example, when scraping multiple web pages and then extracting fields in bulk, the full page content exists only in the execution environment’s variables; only the aggregated structured results are returned to the context, avoiding the entire page content going in and out of the context repeatedly, potentially reducing token consumption by about two orders of magnitude. This “code orchestrates the tool calls” paradigm belongs to the “code as a general Agent meta-capability” framework developed systematically in Chapter 5; here it serves only as a signpost in the evolution of tool design, with the mechanics left to Chapter 5.

The common background for third-generation optimizations is the rapid growth in the number of tools, and the vehicle for this growth is the MCP protocol and its ecosystem, which will be introduced in the next section.

Why Asynchrony is Needed#

Let’s start with an analogy to explain why asynchrony is needed. Synchronous means “do one thing before you can do the next,” while asynchronous means “multiple things can happen concurrently.” A traditional synchronous Agent architecture is like a single-line counter at a store—it can only handle one customer at a time, and only calls the next number after finishing with the current one. A truly intelligent assistant is more like a flexible secretary—with multiple pending items on the desk (emails, phone calls, visitors), the secretary decides which to handle first based on urgency, and can pause and switch to a more urgent task mid-way. In synchronous mode, the Agent either has to wait for a background task to complete before talking to the user, or wait for the conversation to end before processing a newly arrived event. It cannot deliver the core capabilities a real assistant scenario requires:

• Asynchronous execution is the norm—Many tasks require long runtimes and should not block user interaction.
• Dynamic judgment of event priority—Not all events are equally important. The Agent needs to intelligently choose a handling strategy: cancel the current operation (urgent), add to a queue (routine), or process in parallel (independent lightweight query).
• Fluency in interruption and resumption—An interrupted conversation or task should be able to resume naturally.

The asynchronous paradigm, however, collides with a fundamental fact about current LLMs: their training assumes synchrony—after a tool call, the next message must be the tool result—while real deployment demands asynchrony: users interrupt at will, tasks progress concurrently, and external events arrive before a tool returns. This “synchronous training / asynchronous deployment” contradiction runs through every engineering trade-off in the rest of this section.

For this, we need an event-driven asynchronous Agent architecture. Technically, this means the system no longer actively and repeatedly checks for “new messages” (this is polling, which is inefficient), but instead automatically triggers processing logic when a new message arrives. All inputs, outputs, thought processes, and external interactions are uniformly modeled as an event stream—a sequence of event records arranged on a timeline. Figure 4-3 shows the overall architecture of an event-driven asynchronous Agent, illustrating the relationship between event sources, the event queue, and the Agent processing flow.

Understanding the Real-World Need for Event-Driven Architecture from OpenClaw#

The open-source framework OpenClaw (its architecture will be detailed in Chapter 5) receives multi-channel messages through a Gateway control plane and routes them to the Agent runtime. It provides three built-in automation mechanisms:

• Hooks: Respond to events in the Agent’s lifecycle, such as session creation and reset, similar to event triggers in GitHub Actions
• Cron (Scheduled Scheduler): Execute periodic tasks according to cron expressions (a widely used syntax for scheduled tasks in Unix systems, e.g., 0 9 * * 5 means 9 AM every Friday), such as generating a weekly report every Friday or summarizing data at the beginning of each month
• Heartbeat (Heartbeat Daemon): Wakes up the Agent every N minutes to check if there are any matters requiring attention, using judgment to avoid alert fatigue

These three mechanisms give OpenClaw Agents the appearance of autonomy—even with the user offline, the Agent can generate reports on schedule, check system status, and handle routine chores. Look closer, though, and a fundamental limitation appears. To be precise: the Gateway already handles messages from built-in channels (IM, the web interface) in push fashion—they are routed to the Agent the moment they arrive. And of the three automation mechanisms, only Cron and Heartbeat let the Agent act without a user message, and both are time-driven—Heartbeat checks at fixed intervals, Cron fires at preset times. Hooks merely react to the framework’s internal lifecycle events and cannot bring in new changes from the outside world. The real gap is this: for any third-party event source beyond the built-in channels—a new email, an external API callback pushing data, an urgent notification demanding immediate attention—OpenClaw has no instant way in. The Agent cannot respond the moment the event occurs; at best it notices at the next Cron/Heartbeat tick.

This delay is unacceptable in many scenarios. Take PineClaw (Pine AI’s OpenClaw plugin) as an example: Pine AI is an AI assistant that makes real phone calls on behalf of the user, with typical scenarios including negotiating bills, canceling subscriptions, and handling insurance claims. When a user initiates a Pine phone task through an OpenClaw Agent, Pine’s voice AI will make the call on behalf of the user, but the user may need to intervene at any time during the call:

• Real-time Identity Verification: The customer service representative asks to verify the account holder’s identity, and Pine needs the user to immediately provide a security code or OTP (One-Time Password) verification code
• Three-Way Call Confirmation: The customer service representative asks to speak directly with the account holder, and Pine needs the user to answer the phone within seconds
• Progress Sync and Decision Confirmation: At a critical point in the negotiation (e.g., the other party proposes a price reduction), Pine needs the user to confirm whether to accept

With Heartbeat’s periodic polling—say a 5-minute interval—the user might not get the notification while the representative is still waiting for the verification code; the representative hangs up and the call fails. Shortening the interval to seconds would simply flood the system with useless requests.

PineClaw’s solution is to introduce a Channel mechanism—establishing a real-time event channel between OpenClaw’s Gateway and the Pine API. When key events occur, such as a call connecting, needing user input, or the call ending, the message is instantly pushed to the OpenClaw Agent. The Agent processes it immediately and notifies the user, reducing response latency from minutes to seconds.

This case reveals the core value of an event-driven architecture for Agent frameworks: true “proactive service” requires not only that the Agent can periodically check the world, but also that the world can actively notify the Agent. Unifying all inputs—user messages, tool returns, external callbacks, scheduled triggers—into an event stream, and driving the Agent’s thinking and actions through an event loop, is the architectural foundation for achieving this goal. Under this architecture, we will first introduce the two tool categories directly related to events, as well as the virtual identity and isolated execution environment that support the Agent’s independent actions, before discussing the specific design of the event handling mechanism.

Event-Triggered Tools#

Event-triggered tools are the entry points through which external events drive an Agent’s actions. Without them, an Agent can only operate in a continuous loop of thinking, calling tools, and finally outputting a result, then waiting for the user’s next input. To translate changes in the world into events an Agent can process, there are three common types of event-triggered tools.

Timers (set_timer) handle events tied to physical time. If an email goes unanswered, the Agent should follow up after a while to ask about progress; if a call reaches someone outside working hours, it should retry during the next business window. To support this, tools like OpenClaw and Claude Code include timer functionality, letting the Agent wake itself at a specified physical time. One-shot timers are used for tasks with a specific deadline: for instance, if a user asks to “call the DMV” on a Saturday, the Agent sets a timer for “next Monday at 10<00> AM to call the DMV,” which triggers the call automatically. Recurring timers are used for periodic tasks: such as checking server health every hour or sending a progress report every Friday. Additionally, some external services don’t support proactive progress updates, requiring the Agent to actively poll for status. In such cases, a recurring timer is needed for repeated queries—the Heartbeat mechanism in OpenClaw from the previous section is a systematized form of this, and it’s the root of OpenClaw’s “proactive service” capability.

Background Task Monitoring (monitor_shell) handles events from asynchronously executing tools or command-line tasks. Some command-line tasks run in the background for a long time, and the Agent needs to track their progress. If the Agent “stares at the command line,” repeatedly calling a tool to poll for progress, it burns tokens; if it waits until the task has fully finished before thinking again, it misses critical problems as they unfold—and if the command hangs, it cannot intervene at all, stalling the whole task. Claude Code solves this by introducing a monitor tool, allowing the Agent to monitor new output from the command line or output containing specific keywords.

External Event Channels (connect_channel) push external events like new emails, API callbacks, or IM messages to the Agent in real-time. The Channel mechanism in PineClaw from the previous section is a typical implementation.

From a design perspective, event-triggered tools should define clear trigger conditions and filtering rules to prevent irrelevant events from waking the Agent and wasting computational resources. The event payload should contain sufficient context information to minimize the number of additional queries the Agent needs to make after being woken up.

User Communication Tools#

User communication tools arise from the increasing diversification of communication channels between the Agent and the user. Many Agents (like Claude Code, Manus, Genspark) use a native ReAct loop, where everything the Agent “says” (i.e., assistant messages) is sent directly to the user, who must open a specific session in the app to converse with the Agent. OpenClaw is one of the most influential general-purpose Agents that breaks this human-computer communication paradigm: its sessions are transparent to the user—the user doesn’t need to be aware of the session’s existence or care about the details of the Agent’s tool calls; both the user and the Agent can send messages to each other at any time, rather than a strict user-message, Agent-response pattern. Consequently, many users feel OpenClaw has a “human-like presence,” messaging them asynchronously the way a secretary would. These text messages are not the model’s assistant messages piped straight to the user; they are sent through dedicated tools, can carry image and file attachments, and can trigger push notifications according to urgency.

Beyond text-based communication, an increasing number of Agents possess multimodal communication capabilities, such as sending structured card messages or reminder emails. Some Agents have begun experimenting with generative UI, using HTML or other methods to create interactive interfaces for presenting information to users in a more user-friendly way. From a design perspective, user communication tools should support asynchronous messaging (the user may not be online), provide read/unread status tracking, and maintain message consistency across multiple channels.

Multi-channel User Communication and Recall.

One category boundary is easy to confuse: both kinds of tool “send notifications,” but if the recipient is an approver or collaborator (requesting admin approval, reporting progress to a collaborating Agent), the tool belongs to the collaboration category; only when the recipient is the end user does it count as a user communication tool. The distinction lies not in the channel but in who is being notified, and why.

An Agent’s response should not be limited to a single channel; the notification mechanism also serves as a user recall mechanism. Message sending extends to instant messaging, SMS, email, phone calls, push notifications, and other channels. The Agent decides on the channel based on a combination of urgency, user status, content nature, and user preferences, ensuring important messages are not missed while avoiding redundant interruptions.

For long-running tasks, the Agent needs to proactively notify the user upon completion to recall their attention. For periodic tasks (like daily summaries or weekly reports), notifications can help establish a regular interaction habit for the user.

User communication tools solve the problem of “how to reach the user.” However, the identity the Agent assumes on these channels and the environment in which it performs actions on behalf of the user require a layer of identity and environmental infrastructure, which is the topic of the next section.

Virtual Identity and Isolated Execution Environment#

A word on this section’s placement: virtual identity and isolated execution environments are fundamentally execution-environment infrastructure, of a piece with the sandboxes discussed under execution tools. They appear here, in the asynchronous architecture section, because the Agents that need them most urgently are the ones that run independently, stay resident, and act on the user’s behalf at any moment.

As mentioned at the beginning of this chapter, Samantha in Her has an independent identity and operating environment. Achieving such a general-purpose assistant forces a key architectural choice: should the Agent manage the user’s personal accounts directly, or hold a virtual identity of its own? Direct management looks convenient, but one Agent error or compromise exposes the user’s entire digital identity. The safer approach is to give the Agent an independent virtual identity—the way a secretary has their own office phone and mailbox—comprising dedicated communication accounts, storage, and computing environments, so the Agent can work openly on the user’s behalf. Far from weakening trust, a clearly declared identity makes the communication more authentic.

Virtual identities need to be grounded in isolated execution environments. Virtual computers (VMs/containers) and virtual phones (Android emulators) provide the Agent with operating system-level isolation and full desktop/mobile operation capabilities: the Agent has its own user account, home directory, and login credentials within them, making all operations traceable and auditable; even if erroneous operations are performed, the host system and the user’s real device remain unaffected. This is an extension of the sandbox concept discussed in the execution tools section into the “digital identity” dimension—sandboxes isolate code execution, while virtual computers and phones isolate the entire digital identity.

An independent identity also presents two practical challenges. First is anti-automation mechanisms: many websites use CAPTCHAs and IP reputation checks to block automated access. Virtual environments using data center IPs are easily identified; in practice, normal access often requires configuring a residential proxy network (which uses real household IPs). Second, access to the user’s real accounts: when a task must log in as the user themselves, use Human-in-the-Loop authentication—a VNC/RDP remote desktop where the user logs in personally, sees the full interface the Agent is operating, and understands why authentication is needed. The session token is then reused within its validity period to avoid interrupting the user repeatedly, balancing autonomy and security.

Data exchange between the main Agent and the virtual environment is accomplished through a shared file system: using volume mounts (e.g., /workspace/shared) to connect the main Agent, virtual computer, and virtual phone. Data is passed by file path references rather than content copying, avoiding context window consumption. For example, in a data analysis task: the user uploads a CSV file to the shared directory, the Agent in the virtual computer reads the file, performs analysis, generates charts, and saves them back to the shared directory. The main Agent only needs to return the file path of the chart to the user—what is passed between parties is always a lightweight path string.

Event-triggered tools allow the world to wake the Agent, user communication tools allow the Agent to reach the user, and virtual identities with isolated execution environments allow the Agent to act independently and auditably. The remaining question is: when multiple events converge on the same Agent instance simultaneously, how should they be handled?

Event Handling Mechanism#

A single Agent instance may face multiple events concurrently: a new message from the user, a result from a tool, a timer expiring, a collaboration request from another Agent. How these events are handled efficiently and correctly directly impacts performance and user experience.

Structured Event Modeling.

Handling requires understanding. A general-purpose Agent’s input doesn’t come only from the user—a message from a third party wasn’t addressed to the Agent at all, yet the Agent must understand it, weigh its importance, and decide whether to step in. This requires modeling each input as a structured event rich with semantics:

• Source (who): The user themselves, a contact, a stranger, a system notification
• Channel (how): Phone call, SMS, instant message, email, social media, timer trigger, asynchronous tool call result, command-line monitoring status update
• Content (what): Message text, emotional tone, urgency, whether a reply is needed
• Context (background): Whether it’s a reply to a previous conversation or a new communication, its relevance to the current task

Taking a customer refund request email as an example, the structured event looks like this:

1
{
2
  "source": {"type": "email", "sender": "client@example.com"},
3
  "channel": "gmail_webhook",
4
  "content": {"subject": "Refund Request", "body": "Order #12345, requesting a refund..."},
5
  "context": {"priority": "high", "customer_tier": "vip", "related_orders": ["#12345"]}
6
}

Only when these dimensions are clearly modeled as structured events can the Agent maintain clear cognition in multi-party communication, avoiding mistaking user input for a tool result, or mistaking a tool result containing hidden instructions for a user command (prompt injection). The complexity of multi-threaded context management also requires the Agent to understand the relationships between multiple conversation threads—how a message from a third party affects the user’s mood, the user’s role transitions across different conversations, and when to synthesize information from different threads to provide advice. The trigger ecosystem of workflow platforms like n8n—webhooks, timers, emails, database changes, file watchers—shows the same picture: each trigger is a “sense organ” through which the Agent perceives the world. Once these heterogeneous events are modeled into one structured format, the Agent can process stimuli from any source consistently. The urgency determination and processing strategies below are all built on this unified modeling.

Dynamic Processing Strategy Based on Urgency.

Humans juggling multiple tasks adapt their strategy to urgency: an emergency makes them drop what they’re doing; a routine to-do goes on the list for later. An Agent’s event handling should show the same intelligence.

Cancellation-Based Processing is used for urgent events. When an urgent event arrives (e.g., the user clicks “stop” or a supervisory system sends a high-priority instruction): (1) Stop the current operation—if the LLM is reasoning, immediately cancel the streaming response; if a synchronous tool is executing, send a cancel signal; (2) Drain the pending queue, taking out all its events; (3) Append those events together with the urgent event to the end of the trajectory; (4) Immediately re-invoke the LLM with the updated complete trajectory as input to assess the situation. For example, if the user inputs “Stop! I said the wrong thing” while the Agent is about to perform a potentially erroneous operation, the Agent will immediately see this new input, re-understand the true intent, and thus avoid executing the wrong action.

Queued Processing is used for routine events. When a non-urgent event arrives (e.g., an asynchronous tool returns a result or the user sends supplementary information): (1) Add the event to the end of the queue without interrupting the current operation; (2) Wait for the current operation to complete—let the LLM finish reasoning, let the synchronous tool finish executing; (3) When any tool call completes and returns a tool.result, check the queue. If the queue is non-empty, append all events to the trajectory at once; (4) The LLM processes the updated trajectory comprehensively. This enables batch processing, improving efficiency—for example, while the Agent is waiting for a search tool result, the user adds “only show results from the last month.” This supplementary information enters the queue, and when the search results return, both events are presented to the LLM together, avoiding unnecessary round trips.

Parallel Processing is used for independent, lightweight queries. For example, while the Agent is analyzing a large amount of data, the user suddenly asks, “What’s the weather like today?” Such queries have three characteristics: unrelated to the main task, requiring a quick response, and low execution cost. Neither cancellation-based (would interrupt the important main task) nor queued processing (would make the user wait too long) is suitable. The system first assesses the query’s independence and complexity, then executes it independently in a parallel reasoning session, calling necessary tools to generate a response and returning it immediately. The query and response are appended to the main task’s trajectory, clearly marked as “executed in parallel with the main task” to avoid confusing the LLM.

Urgency Determination.

Urgent events: User interrupt (user.interrupt), supervisor instruction (supervisor.instruction), inter-Agent interrupt (agent.interrupt), external triggers marked as urgent (e.g., system alerts, payment failures).

Non-urgent events: Regular user input (user.input), Agent input (agent.input), tool results (tool.result), timer triggers (timer.trigger), regular external triggers.

Hardcoded rules have limitations; the semantics of the event dictate the handling method—“Stop immediately!” uses cancellation-based, “What’s the weather like today?” uses parallel, “Send the report in Chinese” uses queued. It is recommended to use a lightweight classification LLM as an event router, quickly determining which strategy to adopt when an event arrives.

The following experiment, an event-driven email processing Agent, implements the event handling strategies discussed above into a runnable implementation.

Experiment 4-4 ★★★: Event-Driven Email Processing Agent

This experiment builds the simplest event-driven Agent: an Automated Email Processing Assistant. The Agent monitors the email inbox, and whenever a new email arrives, it automatically triggers a processing workflow—classification, summarization, draft reply, and notifying the user if necessary. This is the most intuitive introductory scenario for an event-driven Agent: an external event (new email arrival) triggers a complete Agent thinking cycle.

Experiment Objective: to understand the core idea of event-driven architecture—the Agent no longer waits passively for user input but acts on its own in response to external events. Through this experiment, readers will master the basic closed loop of event source registration, the event queue, and “event arrives → Agent processes → result delivered”.

Event Sources and Event Queue.

The system supports unified access for multiple event sources:

• Email Events (on_email_received): Triggered when a new email arrives, either by periodically checking the inbox or receiving push notifications.
• IM/SMS Messages (on_im_message, on_sms_message): Triggered by instant messaging messages.
• GitHub Events (on_github_pr_update, on_github_issue_update): Triggered by PR review comments or status changes.
• Timer Triggers (on_timer_expire): Triggered by scheduled tasks (e.g., daily summaries, weekly report generation).
• Webhooks (on_webhook_received): Generic callbacks from external systems.
• System Events (on_user_inactive, on_process_timeout, on_resource_alert): Triggered by internal state changes.

All events enter a unified event queue and are processed sequentially in order of arrival. Each event triggers an independent Agent thinking loop: the Agent reads the event content, calls relevant tools (e.g., querying the knowledge base, reading attachments, searching related email history), generates a processing result (classification labels, summaries, draft replies), and finally notifies the user or directly executes an action via notification tools.

Validation Scenario: Configure the Agent to monitor a test mailbox. Simulate receiving three emails—a meeting invitation, a customer complaint, and a marketing advertisement. The Agent processes them sequentially: for the meeting invitation, it automatically checks for calendar conflicts and drafts an accept/decline reply; for the customer complaint, it extracts key information, marks it as high priority, and notifies the user to handle it; for the marketing advertisement, it automatically archives it. The entire process requires no user intervention.

Experiment 4-4 demonstrates the simplest event-driven pattern—events enter a queue, and the Agent processes them sequentially. However, when the Agent needs to respond to interruptions during long-running tool executions, or manage multiple concurrent tasks simultaneously, a simple event queue is insufficient. Next, we discuss deeper engineering challenges.

Engineering Implementation: How to Make Synchronous Models Support Asynchronous Interruptions#

Experiment 4-4 only handles serial events—events enter the queue one by one, and the Agent processes them one after another. Now, let’s return to the “synchronous training / asynchronous deployment” contradiction raised at the beginning of this section: when the user interrupts while a tool has not yet returned, how can the synchronous format accommodate it? This section lays out the engineering workarounds the industry uses today.

Let’s first illustrate this contradiction with a specific scenario. Suppose the Agent is helping a user draft an email (tool call: search for contact information). Before the search returns results, the user suddenly says, “Wait, first check tomorrow’s weather for me.” In a synchronous ReAct loop, the Agent must wait for the search to return before processing the next message—because the API requires that “after issuing a tool call, the next message must be the tool result.” But in the asynchronous real world, events can interrupt ongoing tasks at any time. How to express the semantics of “asynchronous interruption” under the constraints of a “synchronous format” is precisely the question this engineering solution aims to answer.

Engineering Expedient: An Asynchronous Implementation Simulating Synchronous Behavior.

The core idea is: Under normal conditions without interruptions, let the LLM see a standard synchronous trajectory; only when an interruption occurs, insert placeholders to fix the format. Here are five key rules:

Rule 1: Immediately record the assistant message (including thinking, content, and tool call) when the LLM outputs.

Rule 2: Record the tool result only when the tool call is complete. The trajectory is in a “partially completed” state during execution.

Rule 3: Interruptions during tool execution require placeholders. Generate a placeholder response for the unfinished tool (e.g., “The tool is executing in the background, please prioritize the new event”), append the interruption event, and re-invoke the LLM. From the LLM’s perspective, the assistant message still has a paired tool result.

Rule 4: Interruptions during LLM thinking directly discard the current thinking. Do not write it to the trajectory; directly append the new event and start a new round of thinking.

Rule 5: Non-interrupting events enter the queue for batch processing. They are appended all at once only after the current cycle is complete.

Using the example of the Agent drafting an email when the user interrupts to ask about the weather, the operation of these five rules is as follows:

1. The Agent calls search_contacts to search for contact information, and the assistant message is immediately written to the trajectory (Rule 1).
2. Before the search tool returns results, the user sends “First check tomorrow’s weather for me.” Since this is a user interruption, the system generates a placeholder tool result for the unfinished search_contacts (“The tool is executing in the background, please prioritize the new event”, Rule 3), then appends the user’s weather query to the trajectory and re-invokes the LLM. At this point, the trajectory format seen by the LLM is completely valid—the assistant message and tool result are perfectly paired.
3. After the weather query is completed and the user is replied to, the original search_contacts result arrives and is appended to the trajectory as a new event (Rule 2). The Agent reads the contact information and continues drafting the email.

The core advantage of this scheme: under normal conditions, the LLM sees a perfect synchronous trajectory—assistant messages and tool results strictly paired, the timeline clear, no placeholders or anomalous states. This is the friendliest arrangement for LLMs trained under the synchronous paradigm, and it preserves thinking quality. The placeholder—a necessary compromise—appears only when an interruption genuinely occurs.

The risk of hallucination remains, however. Even though the placeholder states explicitly that the tool “has not yet completed,” the model may still fabricate a tool result in later thinking—convincing itself the tool returned valid data and making bad decisions on top of the invention. This is because, in the vast majority of trajectories seen during training, a tool call is immediately followed by the real result; the model has never learned how to handle situations where “the result hasn’t come back yet.” Therefore, in practice, interruptions are only triggered in truly urgent situations (when the user explicitly requests a stop); non-urgent events are placed in a queue for batch processing.

Asynchronous Tool Interfaces Suitable for Existing Models.

Since the synchronous assumption of models is difficult to break, a more fundamental strategy is to embrace asynchronous semantics from the design level of the tool interface.

Traditional tool design implies a “call equals completion” semantics. For example, the name phone_call suggests “calling will dial the phone and wait for the call to end, returning the call log.” Under the asynchronous paradigm, “initiation” and “completion” should be decoupled:

• initiate_phone_call: Initiates a phone call, immediately returning a task identifier and initial status (e.g., “Call initiated, dialing…”)
• Call progress is communicated via event notifications (phone_call_connected, phone_call_ended)

The key is that the tool’s name and description themselves should convey asynchronous semantics. When the model sees initiate_phone_call, its language understanding capabilities will naturally infer this is “initiating” rather than “completing.” The tool description should further reinforce this: “This tool initiates a phone call task handled by a sub-agent. It returns the task ID immediately upon successful initiation, allowing you to continue with other matters. A separate notification event will be sent when the call ends.”

Attention Dispersion in Queue-Based Processing.

When processing batch events, the model often only focuses on the last event. The root cause is that the model is trained to react to the most recent input, and batch events break this assumption.

Intervention can be applied at two levels:

Prompt Level: Inform the model, “When you receive multiple consecutive events, please ensure you comprehensively consider all the information.”

Agent Status Bar Markers: Add explicit markers before each event:

1
[Unprocessed Event 1/4] Tool result from database_query: ...
2
[Unprocessed Event 2/4] User supplementary note: Only look at Beijing data
3
[Unprocessed Event 3/4] System reminder: Report deadline is in 30 minutes
4
[Unprocessed Event 4/4] User asks: What's the progress?

Add a summary at the end: “There are 4 unprocessed events above, including 1 tool result, 2 user messages, and 1 system reminder. Please ensure your response covers all the information.”

Deeper Contradictions and Future Directions#

Ultimately, the placeholders, asynchronous tool interfaces, and status bar markers from the previous sections are all using prompt engineering to patch the same “synchronous training / asynchronous deployment” contradiction (Figure 4-6)—the cause of this contradiction has been detailed at the beginning of this section and will not be repeated here, focusing only on its fundamental solution.

Anticipating Model Evolution: From Synchronous to Asynchronous.

The engineering techniques above are essentially using prompt engineering to compensate for the shortcomings of model training, a temporary expedient during a transitional period. The real solution requires a paradigm shift at the model training level.

VLA (Vision-Language-Action, see Chapter 9) models in the robotics field are already beginning to face similar challenges: there is an unavoidable delay between perception and action. The success of VLA points the way for the evolution of Agent models. The next generation of models needs to acquire three core capabilities through reinforcement learning in asynchronous environments:

1. Understanding Asynchronous Interleaving of Events in Trajectories: This is the most critical capability deficiency. Current models expect a strictly synchronous sequence, but in a real asynchronous environment, a tool call might be followed not by a tool result but by a new user message; thinking might be interrupted halfway, but the intermediate state should be retained in the trajectory, and thinking should continue after the new message is processed, rather than starting over. The model needs to maintain clear cognition in such “out-of-order” trajectories—which tool calls are still waiting for results, and which thoughts are unfinished fragments.
2. Resuming Interrupted Tasks and Thoughts: When interrupted to handle an urgent event, the model must still remember the unfinished task. For example, if the user suddenly asks about the weather while the Agent is executing a data analysis tool, after answering, the Agent should naturally wait for the data analysis result, rather than forgetting that a tool is still running. It is particularly important to avoid hallucinations where the model mistakenly believes the interrupted tool call has completed.
3. Comprehensive Processing of Batch Events: When multiple events are appended to the trajectory in a batch, the model must not only focus on the last one; it must comprehensively consider all unprocessed information.

Achieving this asynchronous RL training requires new infrastructure: an asynchronous environment simulator (generating scenarios like delayed tool returns, random user interruptions, etc.) and specialized rewards for asynchronous capabilities (correctly understanding out-of-order trajectories, successfully resuming interrupted thoughts, avoiding hallucinations, comprehensively processing batch events).

Continuous thinking, however, need not wait for the next generation of models. A thin layer of orchestration logic (about two hundred lines) can turn an off-the-shelf text-thinking model into a continuous-time Agent on the spot¹—neatly bridging the “engineering expedient” and “model evolution” halves above. The mechanism is Rule 4, upgraded: instead of discarding a half-finished thought on interruption, build the entire interaction as one uninterrupted stream of thought—at any moment, forcibly close the <think> block the model is writing, inject the newly arrived observation (a tool return, a user interruption, a fresh recognition result) as an ordinary message, and let the model keep decoding. This exploits a resource that usually goes to waste: a model can generate thousands of tokens per second, while a tool call or a user utterance takes several seconds—those waits are free computation, usable for thinking ahead. Two behaviors emerge: thinking while waiting—rather than waiting for the tool to return or the user to finish speaking, the model reasons over the partial information it already has, even firing off the next tool call early (this “anticipatory thinking” tendency reproduced zero-shot across multiple model families; see the paper cited in the footnote for the data); and thinking while doing—continuing to think while producing output, able to correct itself mid-action.

But the more critical half of this research concerns training, and it answers the “anticipating model evolution” call above: orchestration alone makes continuous thinking possible; whether it becomes useful depends on the training signal. The research found that with an “LLM-as-judge” style reward, the model learns to hide its thoughts—trading silence for the judge’s approval—while objective metrics actually worsen; only verifiable objectives that safeguard information coverage make continuous thinking pay off. In a nutshell: orchestration makes the behavior possible; training makes the behavior good—which confirms this section’s judgment that asynchronous capability must ultimately be consolidated through the right training, not patched forever with prompt engineering.

Experiment 4-5 ★★★: Asynchronous Agent with Parallel Execution and Interruption Capabilities

Building on the simple event queue of Experiment 4-4, this experiment moves into the hard parts of asynchronous Agents: parallel tool execution, execution cancellation, and state management. The Agent no longer just processes events one by one; it needs to manage multiple concurrent tasks simultaneously, handle interruptions and recoveries, and make dynamic decisions based on real-time state.

1. Asynchronous Tool Execution: Supports asynchronous execution of time-consuming tools (at least 3-5 seconds), returning a placeholder immediately upon initiation. Validation Scenario: The Agent executes a long-running terminal command. During this time, the user asks, “What time is it now?” The Agent responds immediately, and then presents the analysis result when it returns.

2. Event Queue and Batch Processing: Accumulates non-urgent events and appends them to the trajectory in a batch. Validation Scenario: The Agent is executing a long task. The user sends consecutive messages: “Remember to reply in Japanese” and “Format it as a webpage.” When the task completes, the Agent processes all events at once, generating a Japanese webpage.

3. Interruption Mechanism: A user’s “stop” command immediately terminates the execution flow and cancels the asynchronous tool. Validation Scenario: The Agent is executing a long task. The user sends “Cancel.” The Agent stops immediately, and the trajectory records the interruption event and the cancellation operation.

4. Cancellation and Status Query for Parallel Tools: After an asynchronous tool completes, the real result is injected into the conversation via a new event. Supports cancellation or progress query via task ID. Validation Scenario: The user requests, “Run these three scripts simultaneously for me. Whichever finishes first, check the progress of the remaining scripts. If any hasn’t exceeded 50%, cancel it.” The three scripts simulate analysis processes, outputting progress continuously at speeds of 3%, 2%, and 1% per second, respectively. The Agent starts three asynchronous terminal commands simultaneously. When the script at 3% per second finishes in about 33 seconds, the Agent queries the status of the remaining two terminals, finding one at about 66% and the other at about 33%. It then cancels the one that hasn’t exceeded 50%. After both terminals complete, it integrates the results to generate a complete report.

Existing Tool Discovery Methods#

The traditional approach injects every tool’s schema into the system prompt at once, and it breaks down fast once tools number in the thousands: the context clogs with tool manuals, and selection accuracy drops. Retrieval-based pre-filtering (discussed in the “Tool Ecosystem” section above), which screens candidates by semantic similarity first, eases the problem but carries an inherent limit—it matches once, against the user’s initial query. A request as innocent-looking as “debug the file” may pull in a multi-step, cross-domain tool chain—file access, code analysis, command execution—that no one can foresee when the task begins.

From Passive Selection to Proactive Discovery. The next step is to turn the Agent from passive recipient into active discoverer: when it hits a capability gap mid-execution, it declares in natural language what capability it needs, and the system matches and injects the tool on the fly. MCP-Zero² is the representative work. No tool schema is pre-loaded in the system prompt; the Agent emits structured request blocks in its thinking (e.g., “GitHub server: search repositories and return metadata”), and the system routes through two levels of semantic matching (server-level → tool-level) across thousands of candidates before injecting. The paper reports roughly 98% token savings over full injection on about 2800 tools. The more common engineering equivalent keeps only a few basic tools (web search, code interpreter) plus a “tool search tool” in the system prompt, and lets the Agent describe its needs in natural language to retrieve and load the rest—Anthropic’s Tool Search Tool in the Claude API is one such. What they share: the Agent declares the gap; the system injects on demand.

Hierarchical Matching and Fallback. Efficient matching exploits the hierarchy already present in how tools are organized. In protocols like MCP, tools are grouped by server (like apps on a phone, each bundling a set of related functions), so matching can run in two layers: locate the relevant servers by capability description, then match specific tools within them. That shrinks the search space from “thousands of tools” to “dozens of servers × dozens of tools each,” saving compute and cutting cross-domain semantic confusion. In engineering terms this rests on an embedding index built offline and updated incrementally. And when both layers’ candidates score below threshold, the system should return an explicit “not found,” prompting the Agent to rephrase and retry, to improvise with basic tools, or to create a new tool outright (the subject of Chapter 8).

Dynamic Loading and KV Cache. Proactive discovery carries a subtle engineering cost: dynamically loading tools breaks the KV Cache—put the tool list in the system prompt, and every newly loaded tool invalidates the whole cached prefix. The fix matches Chapter 2’s discussion of Skill injection position: append the variable part (the new tool’s complete schema) as a user message at the end of the conversation, keeping the system prompt prefix stable and the KV Cache fully reusable, with only a short list of tool names maintained in the Agent’s status bar. This pattern is now natively supported by the major APIs and has become the default architecture of mainstream frameworks: the OpenAI Responses API provides a tool_search tool and a defer_loading: true flag, with loaded schemas appended at the end of the context as tool_search_output items so the prefix cache keeps hitting; Claude Code defers MCP tools by default (injected on demand via tool_reference blocks, with only tool names and server instructions kept at session start); and Codex CLI’s tool_search (BM25 retrieval) is an always-on architecture rather than an optional feature. A dynamic tool environment also asks more of the model itself—weaker models struggle with tool definitions appearing at a non-standard position mid-context and tend to emit malformed calls (mismatched JSON brackets, missing parameters), often needing dedicated reinforcement learning training (see Chapter 7).

One easily misunderstood point is worth clarifying: “appended at the end” happens only on the turn when the tool is discovered. From then on, the schema block stays fixed at its original position in the trajectory—new messages in later turns are appended after it, and it becomes ordinary history, rather than being re-moved to the newest tail on every turn (if it were re-injected each turn, it would indeed need re-prefilling every time, and the cache would be pointless). Both APIs guarantee this: OpenAI requires subsequent requests to preserve the tool_search_output item’s position, and the same tool never needs loading again across turns; Anthropic expands the tool_reference block inline at its original position in the conversation history, and the official documentation states that the cache keeps hitting on every subsequent turn. Only two situations actually cause recomputation: the Prompt Cache TTL expiring (which recomputes the entire prefix together—not a cost specific to tool definitions), and modifying, removing, or reordering the loaded tool set (which invalidates the cache from that point on).

Figure 4-10 shows the full picture after several rounds of dynamic discovery: the static prefix holds only the system prompt, core tools, and the tool-search meta-tool, while the schemas discovered along the way are scattered across the trajectory, pinned where they were first injected and served from cache as ordinary history on later turns. This also means “tool definitions must sit at the very front of the context” is no longer an iron rule—the prefix is still static and append-only; tool definitions have simply gained the ability to enter the trajectory on demand. The cost is that the model must be post-trained to understand tool definitions scattered throughout the context.

Plainly, the whole declare-match-inject machinery works, but it is a lot of engineering: an embedding index to maintain offline, KV Cache invalidation to manage, dedicated training for weaker models. The shared premise underneath it all is treating every tool as a formal definition addressed to the model—registered, retrieved, injected. The Skills mechanism in the next section drops that premise for something lighter.

Experiment 4-6 ★★★: Proactive Tool Discovery

Through a controlled comparison, this experiment validates the significant value of proactive tool discovery for small models. Use the Qwen3-4B model to access 120+ tools from the MCP server built in the Perception Tools experiment above.

Experiment Setup: Prepare a set of tasks requiring cross-domain tool collaboration, for example:

• “Query the latest stock price of Apple Inc., search for related news to analyze the reasons” (requires Yahoo Finance + Web Search)
• “Search arXiv for the latest papers on transformers, download the top three papers” (requires arXiv Search + File Download)
• “Analyze the contributor statistics of a GitHub repository, generate a visualization report” (requires GitHub + Code Interpreter)

Control Group: Inject the complete schemas of all 120+ tools into the system prompt at once (over 50K tokens). The 4B model’s instruction-following ability severely degrades with such a long context, exhibiting typical problems: when faced with “query stock price,” it might incorrectly select Web Search instead of the specialized Yahoo Finance tool, or “forget” certain tools in the list, leading to task failure.

Experiment Group: Implement the hybrid scheme described earlier (MCP-Zero’s proactive discovery concept + tool-search-tool implementation): (1) The system prompt retains only the web_search, code_interpreter, and discover_tools meta-tools; (2) discover_tools accepts natural language requests (e.g., “I need the ability to query stock prices”), returns 3-5 candidate tools with complete schemas via embedding vector similarity matching; (3) New tool definitions are appended to the conversation history (as a user message), and the Agent status bar updates the tool name list; (4) Guide the model to proactively call discover_tools when encountering capability gaps.

Expected Observations: Significant improvement in accuracy and task completion rate. Proactive tool discovery not only helps capable LLMs handle scenarios with thousands of tools but also keeps small models usable in scenarios with hundreds of tools.

Skills: Turning Tool Discovery into “On-Demand Lookup”#

The line of thought that has lately gained ground comes from the Skills mechanism. Chapter 2 introduced Skills’ Progressive Disclosure as context engineering; here we treat it as a tool discovery paradigm—and its defining difference from the previous section is that the “embedding index + semantic matching” infrastructure disappears entirely.

Not full exposure up front, but lookup layer by layer. Protocols like MCP tend to lay every tool’s complete schema before the model at once (whether by full injection or retrieval pre-filtering). Skills invert this: at startup the Agent sees only a thin catalog—each skill’s name and description, a few hundred tokens in total. Only when the current context genuinely calls for a capability does the model read the corresponding sub-skill, then follow its internal references down another layer to specific scripts or sub-documents. Discovery is driven by what the model actually needs, in context, as it works—not by a one-shot pre-match against the initial query.

Like consulting a reference book or Wikipedia. This is how humans actually use reference material: nobody reads a handbook or all of Wikipedia cover to cover; you follow the index and the table of contents, looking up exactly the entry you need, when you need it. Tool definitions likewise needn’t live permanently in the context. And compared with the previous section, the Agent needs nothing beyond general file-reading ability (grep, reading files) to browse the skill directory—no vector index to maintain, no need to model tool discovery as a special semantic-retrieval task. It is the more modern, lower-maintenance way to discover tools.

Once Skills are loaded, what about the KV Cache? The previous section’s KV Cache optimization targeted traditional tool definitions—append the schema at the end of the conversation, keep the system prefix intact. Skills face a similar issue: loading a sub-skill is, at bottom, inserting content into the context, and Chapter 2’s injection-position trick—place it at the end, reuse the prefix—applies unchanged. But Skills add a wrinkle: the same skills get loaded again and again, at different positions, across sessions and across users. Prefilling them from scratch alongside the conversation history every time adds up. The “editable, composable KV Cache” introduced at the end of Chapter 2 exists for exactly this: pre-compile and cache each skill’s KV representation once, then use RoPE relocation to “paste” it into any context position at O(L) cost instead of O(L²); if a skill changes slightly (a field update, say), patch it incrementally like an errata note rather than recomputing the whole segment³. A skill thus graduates from “text that must be prefilled every time” to “a reusable, composable cache object”—so the repeated loading that progressive disclosure entails doesn’t give back in latency what it saved in tokens.