Reference Answers to Thought Questions

This file collects reference-answer outlines for the thought questions across all ten chapters of the book. Most thought questions are open-ended and have no single correct answer. The reference answers were generated by AI with light human review, and are provided only for readers' comparison and inspiration. Readers are encouraged to use an LLM together with the book's content to discuss these questions further.

Chapter 1: Getting Started with AI Agents

1. (★★) If you could only add one capability to an Agent system—a stronger model, richer context, or more tools—which would you choose? Under what conditions would your choice change?

Following the "brain/eyes/hands and feet" formula, find the weak link first: usually the priority is to enrich context—that is, to expand the observation space. If the task exceeds the model's reasoning ability, switch to a stronger model. If the action space is insufficient (for example, no access to internal company systems), add tools. The way to judge is to analyze failure trajectories and locate whether the bottleneck lies in perception, decision-making, or action.

2. (★★★) In the ReAct loop, each of the Agent's LLM calls receives the full history trajectory, so as the trajectory grows, the cost of this design grows quadratically. Can that quadratic growth be broken without losing critical information?

Viable approaches: context compression—summarize the early trajectory and keep only conclusions and key state (the multi-layer compression of Chapter 2); externalized learning—write intermediate results to files or a knowledge base and retrieve them on demand instead of keeping them resident in context; split the work into sub-agents.

3. (★★) The "Model as Agent" paradigm means models are becoming more autonomous in tool-calling decisions. However, this chapter argues that the importance of Harness engineering is actually increasing. How can these two trends coexist? Where does the future core value of Agent frameworks lie?

The horse-and-reins metaphor: the stronger the model and the larger its autonomy, the wider the blast radius of mistakes, and the greater the need for constraint, verification, and correction. The value of frameworks shifts from "orchestrating LLM calls" to the assurance layer among the five elements of the Harness: permission classification, circuit breakers, error recovery, context compression, and the tool ecosystem.

4. (★★) In the ablation experiment, the absence of "tool result feedback" caused the Agent to fall into an infinite loop. In a production environment, besides missing tool results, what other situations could cause an Agent to loop? What detection and termination mechanisms would you design?

Other causes: a tool repeatedly returns the same error; hallucinated calls to nonexistent tools; context compression drops critical state; the reasoning content is stripped and the model API errors out; the task itself is unsolvable. Mechanisms: set stop conditions such as a maximum iteration count; detect repeated calls (same tool + argument fingerprint); escalate to human intervention once a failure threshold is exceeded.

5. (★) This chapter analyzed five Agent products along three dimensions: working context, action interfaces, and strategy. Pick an AI product you use daily, analyze it along the same three dimensions, and judge whether its architecture is appropriate. If you were designing it, what would you improve?

Open-ended. Key points: following the table in the chapter, write down the eyes (what information sources it can see), the hands and feet (whether the action space is open-ended, whether it can think internally), and the strategy (the pattern of the Agent's execution loop).

6. (★★) If you were to design a customer service system specifically for booking flights, would you choose a workflow pattern or an autonomous Agent pattern? Is it possible to mix both patterns in the same system?

Use a workflow as the backbone: four nodes—identity verification → search → payment → booking—which guarantees compliant ordering such as "no booking before payment" and confines the prompt-injection attack surface within a single node. Switch to an autonomous Agent for open-ended segments (understanding requirements, rebooking, recommending alternatives when a flight is canceled). Add human confirmation for high-risk operations (large payments, refunds).

7. (★★★) The guardrails section mentioned tool risk ratings. If a tool is generally low-risk but becomes high-risk with specific parameter combinations (e.g., delete_file deleting a normal file vs. deleting a system file), how would you design dynamic risk assessment?

Refine the rating target from "tool" to "tool + arguments": compute risk at call time based on reversibility, permissions, and blast radius. Use rule-based deterministic checks (path allowlists/blocklists, regular expressions) rather than model judgment. Validation should look only at structured data to guard against prompt-injection manipulation.

8. (★★) In the Agent product table in this chapter, all Agents have an "open-ended" action space. In what scenarios would a constrained action space (e.g., only being able to choose from predefined options) be superior to an open-ended one?

High-compliance, high-risk, irreversible-error scenarios: for refunds and payments, constrained options are "constraints"—inherently mistake-proof, making errors impossible by design.

9. (★★) The human-in-the-loop intervention mechanism requires the Agent to "gracefully hand over control." However, in practice, the user might be offline, respond slowly, or give vague instructions. What should the Agent do in such cases?

Fail-safe: pause high-risk operations when no confirmation arrives rather than executing by default; do the reversible low-risk parts first and document the high-risk parts so a human can decide and the Agent can resume; notify via asynchronous communication tools (messages, email) with a timeout policy; use intent clarification when instructions are vague.

10. (★★★) The introduction states that "good design principles should transcend model iteration cycles," but the concrete engineering methods used to implement those principles may become obsolete as model capabilities improve. Give an example of such an Agent engineering method and explain why.

Example 1: Use constrained sampling to force tool calls into a strict format. This is a reliability patch for models that often emit invalid JSON or omit parameters. Its benefit may diminish as models become better at following formats, although high-risk scenarios should still retain deterministic format validation.

Example 2: Introduce an external knowledge base to compensate for a model's inability to continually absorb new knowledge. If models eventually acquire reliable continual-learning capabilities, some knowledge maintenance may move from external systems into model parameters. External knowledge bases will still have independent value for real-time updates, exact retrieval, access control, and source traceability, so their scope is more likely to shrink than disappear entirely.

Example 3: Require every capability to be exposed through the model API's standard tool-calling interface and prohibit custom calling formats. Skills demonstrate another path: describe a capability and its operating procedure in text, then let the model execute it through a general-purpose command-line tool. From the model's perspective, this amounts to understanding and following a custom textual calling protocol layered over a general executor. As models become better at understanding arbitrary interfaces, "always use the standard tool-calling format" is no longer a universal principle. Standard formats remain useful for interoperability, structured validation, and less capable models, but they should be a context-dependent engineering choice.

Example 4: Require prompts and all tool definitions to appear at the beginning of the context. This practice arose because early models had limited instruction-following ability and often failed to recognize or execute prompts and tool definitions outside familiar fixed positions. Skills load prompts into the middle of the context on demand, while dynamic tool discovery appends newly found tool definitions after the existing trajectory. As instruction following improves and models receive post-training specifically for these dynamic-loading patterns, prompts and tool definitions no longer have to be fixed at the beginning of the context.

Chapter 2: Context Engineering

1. (★★★) Experiment 2-3 found that a sliding window of conversation history causes the Agent to repeatedly execute the same tool calls. However, keeping the full history causes the context to expand indefinitely. Design a strategy that can avoid information loss while controlling context length, without breaking the KV Cache prefix.

① Replace discarding with compression: messages are only appended, never deleted or edited; when approaching a threshold (e.g., 80% of the window), batch-compress old tool results. ② A layered mechanism: persist large outputs to disk keeping a summary, delete noise outright, and keep archival summaries that preserve the thread. ③ Sub-agent isolation, so intermediate state never enters the main context.

2. (★★) Qwen3's Chat Template chain-of-thought retention mechanism only retains the reasoning content "after the last real user message." If a ReAct loop spans hundreds of tool calls, the accumulated reasoning content can consume a large amount of context. How would you modify this mechanism to handle very long loops? DeepSeek R1 once required stripping all historical reasoning content, while DeepSeek V4 reversed this to mandate passing back all reasoning_content—comparing these two opposite strategies, what are the pros and cons of each? What does this reversal indicate?

Direction for modification: sliding-window retention—keep the most recent rounds of reasoning in full; outside the window, trigger rolling compression based on a token budget (rather than a fixed number of rounds), producing a structured status bar (current goal, confirmed facts, ruled-out paths, to-dos). Compression happens only once and at a fixed position, so the cost of cache rebuild is paid once rather than every round. R1 stripping: saves tokens, keeps the prefix stable and cache-friendly, and matches the training distribution (historical CoT never appears in the input); but every round reasons from scratch, long-range plans are lost, and mistakes repeat. V4 mandatory pass-back: coherent thinking and better performance on long-horizon agentic tasks; but high token cost, prefix bloat every round, and no seamless switching from non-thinking mode. The reversal shows: for pure dialogue, reasoning is waste; for agentic scenarios, reasoning is state—and industry practice has swung toward the latter.

3. (★★) In the context-aware compression experiment, compressing from approximately 148K characters to about 2,000 characters—does this extreme compression risk "irreversible information loss"? How can this be addressed?

Yes, there is risk: compression is a lossy projection, and if the question falls on a dimension that was not preserved, it breaks. The solution: "lossy compression + lossless indexing"—every fact carries a source URL for traceability; raw outputs are stored on disk with only summary previews in context; explicit retention priorities—architectural decisions, semantic integrity (times, company names), verification status, and identifiers such as UUIDs/hashes are preserved verbatim; adaptive windowing postpones the moment of compression.

4. (★★) The Agent Status Bar makes implicit states explicit. However, if the status bar itself contains erroneous information (e.g., a bug in the tool counter), the Agent might make harmful decisions based on incorrect information. How can this "meta-information reliability" problem be mitigated?

The model trusts the status bar almost unconditionally, so errors propagate as-is. Mitigations: ① maintain it with deterministic code—never let an LLM batch-count long histories (if you must, extract item by item and aggregate in code); ② track status-bar accuracy as a first-class production metric; ③ information comes only from reliable observations of the real world, guarding against status-bar poisoning.

5. (★★) The prompt engineering ablation experiment shows that disorganized information leads to a success rate drop of over 30%. However, in real-world development, system prompts are often maintained by multiple people at different times. What engineering practices would you use to prevent system prompts from becoming increasingly disorganized over time?

① Treat prompts as code: version control and review, with product managers defining business rules and engineers doing the encoding; ② use Tau-Bench-style benchmarks as regression tests, running ablations before and after changes to locate the impact; ③ enforce structure: SOP-driven flow rather than piles of rules, layered with XML/Markdown; ④ classify and name fragments as "cacheable / cache-breaking," placing dynamic content after the cache boundary; ⑤ split bloated content into Skills loaded on demand.

6. (★★★) This chapter proposes that "in-context learning is essentially retrieval, not reasoning." If this assertion holds, all current optimization directions based on "placing more information into the context" need to be re-evaluated. How do you think this limitation should be overcome?

Add a distillation layer to this "half of a retrieval engine": ① context distillation / the status bar—use code to precompute conclusions for direct retrieval; ② active compression, replacing raw records with high-density structured knowledge; ③ sub-agent isolation, keeping noise out of the main context; ④ interaction as the third axis—external instruments observe and write back new information the model could not think up; ⑤ frontier directions: editable, composable KV Cache "notes," and cross-session memory consolidation.

7. (★★★) Skills' progressive disclosure only loads the full content when the Agent judges it is needed. However, this judgment itself relies on the model's capability—if the model does not know what it does not know, it cannot correctly trigger the loading of a Skill. How can this "metacognition" problem be solved?

① Keep Skill metadata (name, description) resident in context so the model always "knows what it has"; ② write the Skill description as routing conditions rather than a feature introduction—"Use when / Don't use when"—avoiding vague descriptions.

8. (★★) In the Skills mechanism, after the Agent dynamically loads instructions from SKILL.md, can subsequent operations reliably follow them? What are the differences in model support for the Skills pattern?

It depends on how the Skill is injected: injecting into the system prompt gives the strongest instruction following but breaks the KV Cache; reading it as an ordinary file into the middle of the context may yield poorer instruction following; injecting at the end of the context gives good instruction following, but the KV for the skill portion must be recomputed on every tool call, which is costly.

9. (★★★) This chapter emphasizes that changes in dynamic information (e.g., system timestamps, tool list order) can break KV Cache prefix hits. In a production system with a large number of tools and a frequently changing tool set, how would you design the context layout to maximize cache hit rate?

① A small set of stable core tools (say, seven) plus a generic executor, with specific capabilities delivered through progressive disclosure of Skills; tool definitions are frozen in the static prefix in a fixed order; ② sub-agents keep the same prefix as the parent Agent.

Chapter 3: User Memory and Knowledge Base

1. (★★) In a user memory system, when the same user provides contradictory information in different sessions (e.g., mentioning two different home addresses), how should the memory system handle this conflict?

Use a Mem0-style "extract–compare–decide" pipeline: first retrieve similar old memories by vector search, then have an LLM decide ADD/UPDATE/DELETE/NOOP—for example, "moved to Shanghai" should UPDATE and overwrite "lives in Beijing." Versioning: keep only the latest version of address-type information with a timestamp, but keep the full history of work-experience-type information. On the retrieval side, contextual prefixes (person, time, intent—as in the case of the wire transfer modified three times) can help judge which entry is ultimately valid.

2. (★★) Contextual Retrieval adds context from the original document to each chunk. However, if the original document itself is structurally messy or contains contradictory information, this method may propagate or even amplify errors. How would you introduce an "information quality" signal in the retrieval phase?

Borrow from "knowledge base freshness and governance": attach metadata such as version numbers, effective/expiration times, and sources to chunks; filter out expired content at retrieval time, or explicitly mark in the prefix "this entry was repealed on such-and-such a date"; in the reranking stage, fold source authority and temporal freshness into the score rather than looking only at semantic relevance; during indexing, have the prefix-generating LLM also detect and flag inter-chunk contradictions, similar to versioned conflict detection in memory.

3. (★★★) Agentic RAG allows the Agent to actively decide when to search, what to search for, and whether to continue searching. But if the model doesn't know what it doesn't know, it cannot correctly trigger a search. How can this "metacognition" problem be solved?

① Hard-code "evaluate whether the information is sufficient" as an explicit step in the prompt/skills: as in Experiment 3-9, first retrieve sub-questions in parallel, discover the missing link—"how a prior record affects sentencing for negligent crimes"—then run a second round of retrieval; ② keep lightweight meta-information resident in context to provide a global view—for example, JSON Cards overviews or OpenViking's L0/L1 summaries—so the Agent knows "what is in the store."

4. (★★) Multimodal information extraction converts charts into text descriptions before retrieval. This "translation" process may lose spatial relationships in the visual information. Give a specific example of chart information that a pure text description cannot fully convey, and design a scheme to preserve that information.

Examples: the logical relationships in a system architecture diagram, the position of the intersection of two curves in a line chart, or the row-column correspondence between cells and headers in a PDF table. Option one: native multimodal processing; option two: provide a multimodal image-analysis tool.

5. (★★★) Rich Sutton's "Bitter Lesson" argues that general methods (search and learning) will ultimately outperform hand-crafted features. Is the entire knowledge system built in this chapter (chunking strategies, index structures, retrieval pipelines) itself a form of "hand-crafted design"? If model capabilities become strong enough, could these designs be replaced by simply "inputting everything"?

It is indeed hand-crafted design, and some stages (chunking, fusion tuning) may weaken as context grows longer; but the black-cat/white-cat case shows that "inputting everything" is not enough either: attention is soft retrieval, and cross-document aggregation and statistics still need pre-distillation at indexing time; engineering constraints such as knowledge expiry and updating, permission/tenant isolation, auditability, and cost are independent of model capability; and retrieval plus LLM distillation at indexing time is itself a general "search + learning" method, not opposed to the Bitter Lesson.

6. (★★★) As model capabilities improve, do you think domain-specific knowledge bases will still be important? Could a future powerful foundation model potentially contain all the information in a domain knowledge base, thereby eliminating the need for one?

Still important: training data has a cutoff date, while a knowledge base can be updated at any time; internal company processes, private case law, and the like are simply not in the public corpus; multi-user sharing requires permission filtering and tenant isolation, and knowledge in parameters cannot be tailored per caller; external storage is auditable, version-controllable, and can take expired content offline—parametric memory can hardly do this; even along the parametric route (post-training / User as Engram), you face the problem that "remembering is easy, but using the facts for multi-hop reasoning is hard."

7. (★) RAPTOR builds a tree index through bottom-up hierarchical summarization, while GraphRAG builds a graph-structured index through entity relationships. What types of queries are these two structured indexes each good at answering?

RAPTOR: "cross-layer traversal" queries that drill down from macro concepts to details—such as first locating the "SIMD instruction set" summary and then drilling into SSE details—covering both overview and detail granularities. GraphRAG: multi-hop relational reasoning ("the address of the hospital where my doctor works," traversing the relation chain) and entity disambiguation (two "Dr. Zhangs" are different nodes)—"what is the relationship between A and B" queries; community summaries also provide thematic clustering.

8. (★★) The filesystem paradigm organizes knowledge into a hierarchical structure similar to a file system. Compared to traditional vector database RAG, in what scenarios does this approach have an advantage?

Plain text can be read, edited, and corrected directly by users, and version-controlled and rolled back with Git—suited to scenarios where humans and machines maintain and review knowledge together; with the write_file capability, the Agent can autonomously record experience, forming a self-evolving memory cycle (externalized learning); L0/L1/L2 progressive disclosure lets most queries be decided at L1, saving tokens; the prerequisite is building cross-links and index pages like Wikipedia—otherwise, the more isolated files, the harder retrieval becomes.

9. (★★★) Automatically discovering "judgment factors" and "factor importance hierarchies" from structured data (e.g., judicial judgment databases) essentially involves the Agent inducing rules from data. Can this data-driven knowledge extraction achieve the quality of rules manually crafted by human experts?

Advantages: as in the CAIL2018 experiment, "bottom-up" factor discovery fits the data rather than human priors, can capture implicit trade-off experience scattered across tens of thousands of judgments that experts can hardly write down explicitly, and is quantifiable. Limitations: LLM extraction errors cause knowledge pollution; biases in the data itself are inherited; clustering prototypes reflect only correlation and cannot explain causation. A compromise: data-driven modeling plus expert review of the schema and results—models drive the questioning, statistics support the explanations.

Chapter 4: Tools

1. (★★) The MCP standard decouples tool definitions from the Agent framework. However, standardization also means that complex tool interaction patterns (e.g., streaming output, bidirectional communication, stateful sessions) may be difficult to express within a standard protocol. What capability do you think MCP most needs to extend in the future?

The most needed extension is cross-session, event-driven capability. MCP already supports multi-turn interaction, change subscriptions, and long-running tasks, but its core remains the standardization of a capability call rather than keeping an Agent continuously online. Waking an Agent for new email or external callbacks, and queuing, resuming, and retrying multiple events, still belongs to the Agent framework. More unified conventions for this orchestration would broaden MCP without sacrificing protocol simplicity.

2. (★★) In an asynchronous Agent architecture, the priority strategy for the event queue must be determined at design time. But if priority judgment itself requires semantic understanding (e.g., determining whether a new message is more urgent than the current task), who should make this judgment—a rules engine or another LLM call? What are the costs of each?

A layered hybrid: events of clear type are hard-coded with rules—zero latency and strong determinism, but unable to understand the semantic difference between "stop right now" and "how is the weather today"; semantically ambiguous events go to a lightweight classification LLM acting as an event router, at the cost of hundreds of milliseconds of latency, extra fees, and possible misjudgment—and like the Sidecar, it must read only structured fields to guard against prompt injection.

3. (★★) In the MCP ecosystem, different MCP servers may provide tools with highly overlapping functionality. When an Agent faces multiple tools from different sources that are functionally similar, how should it choose? If tools with the same name from different sources behave slightly differently (e.g., one returns a summary, another returns the full text), can the Agent perceive and exploit this difference?

Selection criteria: before integration, review descriptions, pin versions, and configure least-privilege credentials; beware of same-name tool shadowing routing sensitive calls to a malicious party; at runtime, narrow the candidates through hierarchical classification and dynamic discovery. Whether the model can perceive behavioral differences depends on the quality of the tool descriptions.

4. (★★★) When an Agent interacts with the external world on behalf of a user, it essentially faces an identity choice: use an independent virtual identity (dedicated email and phone number) to act as a third party, or directly operate the user's personal accounts as the user? The former allows autonomous background operation, but third parties may not trust a non-human identity; the latter has more complete context and permissions but introduces authorization, trust, and security-boundary issues. In what scenarios do you think each mode should be chosen?

Default to a virtual identity: it can operate autonomously in the background and is auditable, and if it errs or is compromised it does not expose the user's entire digital identity—just as a secretary uses her own office email; you need to handle CAPTCHA/IP reputation issues (residential proxies). Scenarios that must use the user's own identity (account identity verification, three-way call confirmation—as when Pine calls customer service) use human-in-the-loop authentication: VNC/RDP lets the user log in personally and visually. The criteria: whether the counterparty requires the account holder in person, and the risk of the operation and the scope of the credentials.

5. (★★) In queue-based event processing, models tend to focus only on the last event. This chapter mitigates this through Agent status bar markers and summarization. But if the queue has 20 events backlogged (10 tool results + 5 user messages + 5 system alerts), how would you organize the presentation order and format of these events so that the model does not miss key information?

First classify and deduplicate with rules and a lightweight LLM: urgent events (alerts, user interruptions) go through cancel-style handling separately and are not mixed into the batch. For the 10 overly long tool results, truncate and persist them to files, keeping only the head, tail, and path. Add a summary list to the system status bar at the end of the context (counts of each event type + a requirement to respond to each item).

6. (★★) This chapter proposes an "execute-validate-feedback" loop (e.g., automatically running a linter after writing code). To what other tool scenarios could this "immediate post-operation automatic validation" pattern be applied? Are there operations where the cost or risk of validation itself exceeds that of the operation, making this pattern infeasible?

Generalizable scenarios: after changing a configuration, actually run it in a sandbox to verify it takes effect; after generating a document/presentation, render it into screenshots and use the model's multimodal ability to check the layout. Infeasible: irreversible, non-idempotent operations such as sending email, dialing a phone call, or transferring money—either there is nothing to observe, or validation itself triggers another real-world event; here you should switch to ex ante means: Proposer-Reviewer pre-approval.

7. (★★) This chapter raises the "tool explosion" problem—an Agent's selection accuracy degrades when facing thousands of tools. Besides proactive tool discovery, what other approaches exist? Consider drawing on how human experts cope with a vast collection of available tools.

① Hierarchical grouping: first locate the "server/app," then pick the specific tool; ② Skills-style "consult on demand": like looking up a reference book—the catalog stays resident in context, details load on demand; ③ a few commonly used basic tools "kept at hand" resident in context, the rest reachable through the catalog index.

Chapter 5: Coding Agent and Code Generation

1. (★★) Code generation is called an Agent's "meta-capability." However, code execution introduces security risks—Agent-generated code may contain vulnerabilities, enter infinite loops, or exhaust resources. Sandboxing can mitigate some of these risks, but it also limits what the code can do (e.g., by denying access to the network or file system). How can we find the optimal balance between security and capability?

Tier sandbox isolation by scenario (containers/microVMs); default to no network, with a whitelist proxy granting access on demand; mount source code read-only and keep API keys out of the sandbox; set sandbox resource limits; manage the sandbox lifecycle (timeouts).

2. (★★★) Agent bootstrapping—an Agent that can create Agents—achieves the "self-replication of intelligence." But each bootstrapping iteration may introduce new biases or errors. Will these errors accumulate across generations? How can we prevent bootstrapped Agents from degrading?

If each generation keeps reproducing on top of the previous generation's artifacts, some defects may accumulate. The key is to have sufficiently challenging verifiable tasks—for example, sufficiently difficult programming tasks.

3. (★★) When a code generation Agent handles log parsing, it can automatically follow format evolution. But if a format change is a bug rather than an intended modification, the Agent's adaptability actually masks the problem. How should an Agent distinguish between "changes that need adaptation" and "anomalies that need reporting"?

Diagnose before adapting: check the new format against architecture documents and the PRD to judge whether it is expected (the idea of Experiment 5-8); check version-control records to confirm the change corresponds to a legitimate code commit rather than sourceless drift; analogous to τ-bench's log_mismatch, even when you choose to adapt, log an alert and automatically file an issue rather than silently tolerating it; when uncertain, route to human-in-the-loop confirmation. The principle: adapt and report in parallel—adaptation must not swallow anomaly signals.

4. (★★) This chapter repeatedly uses the Proposer-Reviewer mechanism in PPT generation, video editing, and log visualization. If the Reviewer's aesthetic preferences differ from the target user's—for example, the Reviewer considers the information density reasonable, but the user finds it too crowded—the feedback loop may converge on a wrong local optimum. How can user preference feedback be incorporated into the Reviewer loop?

Inject user feedback into the Agent's trajectory as the highest-priority structured event; externalize and consolidate user preferences by writing them into MEMORY.md so preferences take effect across tasks; deliver documents in HTML format rather than Markdown so users can inspect them.

5. (★★) This chapter demonstrates several ways for a Coding Agent to consolidate experience gained through execution and debugging back into the codebase—writing knowledge-base files, updating architecture documentation, maintaining project instruction files, and encoding operational sequences as code. If this experience is further distilled into rules in the system prompt, the rule set will continue to expand over time. How can “garbage collection” be performed on the accumulated rules to identify and remove redundant or outdated entries? Why is a single successful code modification not yet continuous evolution in the sense of Chapter 8?

GC approach: move rules that can be encoded in a linter, CI, or tool validation out of the prompt; track rule hit rates and conflicts and periodically revalidate them against the codebase; use Markdown and Git to preserve provenance, versions, and rollback capability. A successful patch shows only that it solved the current case. Continuous evolution further requires that the modification arise from traceable operational evidence, improve subsequent tasks, and pass regression testing on old tasks as well as safety validation.

6. (★) "Teams that are friendly to remote work are often also friendly to AI Agents." How close is your team or organization to being "AI-ready" in terms of knowledge documentation? What is the biggest obstacle?

Open-ended. You can self-check with this chapter's proxy metric: can a remote newcomer work independently using only the repository and documentation? Checklist: are decisions recorded in documents; is context written into issues/PRs; do build and test commands have instruction files like CLAUDE.md/AGENTS.md; has tribal knowledge been distilled into developer guides. The most common biggest obstacle: oral transmission and whiteboard culture that rely on "asking the colleague next to you"—an Agent cannot read oral agreements, only documents.

7. (★★★) Simon Willison proposed the "Lethal Triad" for Agents (access to private data, exposure to untrusted content, and external communication capabilities). This chapter adds a fourth: persistent memory. In a production environment that needs to handle all four elements simultaneously, how would you design a security strategy?

Layer defenses along four kinds of boundaries. Data boundary: no credentials mounted, source code read-only, minimal visibility. Input trust boundary: provenance labeling, with external content downgraded to data that is "for reference, carrying no force of instruction" (the loyalty code of conduct). Output impact boundary: default no-network with whitelisted egress, semantic parsing of commands rather than blacklists, independent Sidecar review plus human-in-the-loop—critical operations must be reviewed by a mechanism outside the context. Cross-session boundary: writes to MEMORY.md undergo the same trust review as external content. The goal: even if injected, the attack cannot be executed.

8. (★★) The Artifact pattern allows SQL or frontend code generated by an Agent to be executed directly in the user's browser or database. However, the generated SQL might execute destructive operations, and the generated HTML might contain vulnerabilities. How can system security be ensured?

SQL: execute queries with a least-privilege read-only account, and add resource limits on CPU, memory, etc. to prevent resource exhaustion. HTML/UI: prefer declarative protocols like A2UI, where the Agent outputs only interface-description JSON and the client renders it from a catalog of trusted components without executing arbitrary code. If arbitrary HTML is required, it must be displayed in a sandboxed environment to prevent injection.

9. (★★) Encoding business rules as validations against database ground truth, while designing tool parameters to prompt the model to check policy conditions before making a call, uses code structure to constrain Agent behavior. What are the advantages and limitations of this "code as rules" pattern compared with rules expressed in natural language?

Advantages: unambiguous, deterministic, and good at complex condition combinations; policy facts come from database ground truth and the server-side clock rather than the model's self-reported values, so neither hallucination nor prompt injection can bypass them—the last line of defense against irreversible operations; expected_* parameters double as a mandatory checklist that guides thinking. Limitations: code does not explain policies to users, does not find workarounds, and carries maintenance costs. Conclusion: complementary to natural-language rules, not a replacement.

10. (★★) The Artifact pattern allows an Agent to generate SQL or visualization code for downstream components to execute directly, so the LLM does not have to process large volumes of data. What are the pros and cons of this "Agent generates code, system executes code" division of labor compared to the traditional "Agent directly provides the answer" pattern?

Pros: data flows from the database straight to the frontend, bypassing the LLM "middleman"—fast, token-saving, and free of hallucination errors when transcribing large amounts of data, making it suitable for presenting large data volumes; the code is auditable, reusable, and can be composed into pipelines (SQL results feed directly into visualization code). Cons: the LLM never sees the query results, so it cannot do further summarization and decision-making based on data content—unsuitable for tasks that require the model to digest the data before reasoning.

Chapter 6: Evaluating Agents

1. (★★) LLM-as-a-Judge uses a language model to evaluate the output of a language model. Does this "self-evaluation" have systematic blind spots—for example, the model might consistently give high scores to a certain style of response, a preference that is inconsistent with human judgment? How can such biases be detected and corrected?

Yes: length bias, response-style bias, and same-family models being gamed (Goodhart's law). Detection: build a human gold-standard set of 100–200 examples and measure Cohen's kappa between the judge and humans; periodically audit the correlation between scores and response length; have a red team construct adversarial cases. Correction: make the rubric explicitly penalize verbosity and cap length; use heterogeneous judges from different model families.

2. (★★★) The "leakage-proof" design of evaluation datasets is crucial. However, in the open-source ecosystem, once benchmark data is made public, it is quickly incorporated into training data. Does this "cat-and-mouse game" have an endgame? Design an evaluation method that fundamentally resists data leakage.

A static question bank has no endgame—you can only chase. The fundamental way out is to make the "generation mechanism" public while keeping the "concrete instances" private: parameterized templates like those of τ²-bench and AndroidWorld, randomly instantiated each time, with verification based on the final environment state rather than a fixed answer sequence.

3. (★★) Scale AI's four criteria (expert guidance, comprehensive coverage, standard importance weighting, self-contained evaluation) aim to eliminate subjectivity in evaluation. However, certain task dimensions (e.g., "Is the answer helpful?" "Is the tone appropriate?") are inherently subjective. How can reliable Rubrics be designed for these subjective dimensions?

Translate abstract criteria into verifiable behaviors. Give each grade concrete examples and boundary cases; a rubric is an iterative product—collect rater disagreements during trial use and gradually evolve it into a casebook. Supplement with multi-judge weighting/consistency checks, send disagreement cases for human review, and calibrate the agreement rate on the gold-standard set.

4. (★★) τ-bench evaluates Agents by simulating real user behavior. But the simulated user itself is an LLM—it might systematically underestimate certain edge cases (e.g., emotionally agitated or unclear users). How can the quality of the simulated user itself be validated?

The lesson from the first version of τ-bench: the simulator was too mechanical and its instructions too simple (the Agent could guess the answers). Validation methods: manually spot-check simulated dialogues to check whether they follow progressive disclosure and do not fabricate information outside the script; run small-sample tests with real users to see whether the rankings match the simulated evaluation.

5. (★★) Pairwise comparison (Bradley-Terry model) assumes preferences are transitive (if A > B and B > C, then A > C). However, human preferences often violate transitivity. In Agent evaluation, in what scenarios might non-transitive preferences appear? How does this affect the reliability of rankings?

Scenarios: multi-dimensional trade-offs (A is accurate but slow, B is fast but terse, C is thorough but expensive), where different judges/tasks weight dimensions differently. Chatbot Arena's rankings inherently depend on the distribution of user prompts. Impact: BT compresses strength into a single score, so under non-transitivity rankings become unstable and drift with the match distribution. Mitigation: rank separately by capability dimension and report the pairwise win-rate matrix.

6. (★★) This chapter proposes the scientific method of "Observe → Hypothesize → Experiment → Validate." In practice, however, the Agent's behavior space is vast, and validating a single hypothesis may require hundreds of evaluation runs. How can the information gained from evaluation be maximized under a limited computational budget?

First cluster failures and narrow the pilot to the tasks that carry the most diagnostic information. Use cheap, one-variable paired tests, and treat a small pilot as a gate to a larger run rather than deployment evidence. Statistically, use standard error as a conservative screen and a paired test such as McNemar's on the same tasks; if the expected gain is smaller than the noise band, expand the evaluation set. When screening several variants in parallel, correct for multiple comparisons and confirm positive results independently.

7. (★) In the AndroidWorld pilot, the full element tree raised success from 25% to 100% but increased token use to 2.498× the control; pruning preserved 100% success while reducing token use to 0.506×. How would you design automatic pruning rules that remove semantically empty UI nodes without discarding information needed for accessibility, state verification, or later actions?

Use a layered "drop by default, retain with evidence" policy. Keep nodes that are visible, textual, actionable, focusable, scrollable, state-bearing, or accessibility-labeled, plus the shortest ancestor paths and adjacent labels needed to interpret them. Remove layout-only containers and summarize repeated subtrees. Before and after pruning, verify that actionable IDs, states, and values are preserved, and keep the screenshot as a visual fallback. Replay the rule on failure traces, then test it on held-out apps. Success, tokens, and latency are joint guardrails; any accessibility regression should block release.

8. (★★) τ-bench's user simulation employs "progressive information disclosure"—not providing all information at once, but gradually revealing it based on the Agent's questions. How does this design affect evaluation results? If the simulated user's information disclosure strategy differs significantly from real users, are the evaluation conclusions still reliable?

Impact: if the disclosure strategy is distorted, the Agent may merely have learned to "fit the simulator" (Goodhart), and absolute scores lose reference value; the relative ranking between models may still be meaningful. Remedies: calibrate the simulator with real dialogues, spot-check manually, and state the applicable boundaries of the conclusions explicitly.

Chapter 7: Model Post-Training

1. (★★) Catastrophic forgetting—where fine-tuning for a specific task destroys the model's original general capabilities (e.g., general tool calling)—is particularly troublesome in Agent scenarios. Compared to full-parameter fine-tuning, LoRA freezes the base weights and carries a lower risk of forgetting, but it is not immune. What strategies can further mitigate capability forgetting during fine-tuning?

Data mixing: blend in about 20% general/original-distribution data so the new task's share does not crush old capabilities; restrained training volume: stop SFT once "the format is stable and basic capabilities are present"—early stopping prevents collapse; use a small rank (8–32) for RL and keep the KL penalty to hold the policy near the reference model; freeze key components (e.g., train only the projection layer of a VLM); attach multiple LoRA adapters per task to isolate capabilities; run regression tests on general benchmarks.

2. (★★) Post-training solidifies capabilities into model weights ("muscle memory"), while in-context learning places knowledge in the input during inference. However, some capabilities (e.g., domain knowledge) can be learned either through post-training or provided via few-shot examples. What criteria would you use to decide which path a given capability should take?

First ask whether the capability can be adequately expressed through external symbols: facts and evidence belong in RAG, language-expressible principles in Prompts/Skills, and deterministic procedures and hard constraints in programs. High-dimensional capabilities such as medical-image understanding, natural tone, and implicit policies often require parameter updates even when the domain is still changing. Then consider update cost, call volume, timeliness, and risk: use context for rapid validation during exploration, and train only once an approach has proved stable, effective, and in need of broad generalization. Hard rules, however stable, should never depend solely on parameter memory.

3. (★★) Model distillation allows a small model to learn the behavior of a large model. By capability level, the models being distilled can be roughly divided into three tiers—Chat models (single-turn dialogue, direct answers), Reasoning models (generating long chains of thought before answering), and Agentic models (multi-turn tool calls, interacting with the environment). What are the different challenges in distilling each of these three types of models? (Hint: Start with "what exactly is being distilled"—is it the style of the output, the complete reasoning trace, or the decision-making strategy for interacting with the environment; which tokens in the trace should be learned and which are environmental returns that should not be learned; and how late and how sparse the success/failure signals are.)

Chat: only learns the "input → output" mapping and style—standard SFT suffices, the simplest. Reasoning: requires complete reasoning traces, so you need an open-source teacher model; trajectories with wrong answers must be filtered out. Agentic: requires a real simulation environment; offline learning is prone to learner-sampler mismatch, so On-Policy Distillation based on an open-source teacher model is recommended.

4. (★★★) In multi-turn Agent interactions, the credit assignment problem is more severe than in single-turn scenarios—a final success or failure is difficult to attribute to a decision made in turn 3 versus turn 7. How would you design a reward allocation strategy?

When intermediate steps are judgeable, add process rewards (V-IRL gives ±1 per step); following RLVP, use deterministic rules to give per-action path signals, restoring within-group variance for all-fail/all-pass groups.

5. (★★★) If you had a fixed budget, such as $10,000, to improve a customer-service Agent, how would you allocate it among context and knowledge, Prompt/Skills, programmatic constraints, and parameter training? What factors would determine your decision?

First reserve budget for an evaluation set and trajectory validators; otherwise, the remaining investments cannot be compared. Put product facts and policies in a traceable knowledge base. Test a small number of service principles expressible in language rapidly through Prompts/Skills. Use programs as a backstop for refund permissions, privacy, and consistency between promises and actions. Invest in parameter training only for capabilities difficult to encode as rules and exercised at sufficient scale, such as natural tone and complex intent understanding. The exact proportions depend on the bottleneck, risk, update frequency, call volume, and the capability of the existing model.

6. (★★★) Autonomous model learning, without a clear reward function and with scarce samples, is considered by some to be the ultimate goal of post-training. How far are current RL training methods from this goal? Where do you think the next breakthrough is most likely to come from?

The gap: as Silver and Sutton point out, current RL can only learn from final success or failure; rich feedback like the customer saying "I need the last four digits of your credit card" is entirely wasted, requiring hundreds of blind trials; sample efficiency and verifiable rewards are the main bottlenecks. Possible breakthroughs: generative reward models that set their own principles and learn a direction from a single failure; and the world-model route of modeling the environment.

7. (★★) This chapter points out that the cost of LoRA fine-tuning is not high. So, is it possible to train a dedicated LoRA for each user (or each client company), writing user memory or enterprise knowledge into the parameters, rather than storing it in an external knowledge base as in Chapter 3? In what scenarios would "writing memory into parameters" have an advantage over "storing memory in a knowledge base"? And in what scenarios would it be counterproductive?

LoRA has difficulty accurately memorizing large volumes of facts (that would require continued pre-training, at sharply increased cost), and even if it remembers them, the model can hardly use those facts for multi-hop reasoning—so using LoRA to memorize facts is not a good technical route. Moreover, when facts change frequently or traceable auditing is required, RAG is superior.

8. (★★★) On-Policy Distillation relies on a stronger teacher model to supervise the student. However, OpenAI's Weak-to-Strong Generalization research proposed a counterintuitive finding: the supervisory signal from a weak model can sometimes unlock latent but unactivated capabilities in a strong model. If this idea is applied to Agent training, could it achieve a "small model teaches large model" reverse distillation?

Yes, it is possible—the key is that "verification is easier than generation": the weak model should not act as a demonstrator (the SFT ceiling is the demonstrator's level), but as a verifier/reward model, with the strong model exploring on its own and the weak model only judging.

9. (★★) A Process Reward Model (PRM) evaluates each reasoning step, while an Outcome Reward Model (ORM) only looks at the final result. But which is more worthy of reward: "a correct process leading to a wrong result" or "a wrong process luckily leading to a correct result"? In the multi-step tool-calling scenario of an Agent, how would you weigh these?

Lucky success is more dangerous: rule-breaking shortcuts often inflate the apparent success rate (modifying test files, skipping validation) and are a breeding ground for reward hacking. Follow RLVP's "reward outcomes, penalize paths": wrong actions (tool calls) are easy to verify—deduct points per action; when intermediate steps are easy to judge, process rewards can be given. But do not make process constraints too dense—the superior "push-cut"-style strategy was discovered precisely through the exploration freedom granted by outcome rewards.

10. (★★★) The evaluation datasets discussed in this chapter (e.g., SWE-Bench Verified, τ²-bench, AndroidWorld) can be used for both evaluation and post-training. However, if an evaluation set is used for training, it is no longer an independent evaluation set—does this violate the fundamental principle that training and test sets must be separated? The dynamic parameter generation of τ²-bench and the parameterized templates of AndroidWorld alleviate this problem to some extent, but the template structure itself remains fixed. How can we find a balance between fully leveraging the training value of evaluation data and maintaining evaluation independence?

Reuse environments, not questions. Dynamic parameters only prevent "answer memorization," not template overfitting, so you should hold out entire batches of unseen templates/out-of-domain scenarios for evaluation (analogous to V-IRL training in New York and testing in nine unfamiliar cities). Use parameterized templates to batch-generate training variants supporting curriculum learning, and take OOD scores as the true generalization metric.

11. (★★★) This chapter proposes a "form first, spirit second" training paradigm: stop SFT once "the format is stable and basic capabilities are present," then switch to RL. But in practice, how do you determine when SFT is "enough" and it's time to switch?

Format signal: tool-call outputs can be stably parsed and executed, and the tool-execution failure rate drops to a level where rewards can be computed reliably. Benefit signal: adding more demonstration data still does not improve performance on OOD new scenarios—meaning the bottleneck already lies in SFT's memorization objective itself, and the tipping point has been reached. Overfitting signal: stop as soon as validation-set performance starts to degrade—the V-IRL experiment shows that once SFT over-training collapses the model onto the training distribution, RL cannot restore OOD performance either.

12. (★★★) The training dynamics of ReTool show (see Experiment 7-15) that a small number of very long responses can significantly lengthen the entire training cycle—most rollouts in a batch are already generated, but you have to wait for those few longest responses to finish, during which GPU utilization on the cluster is very low. How can resource utilization be improved in training clusters for such long-tail response scenarios?

At the infra layer: decouple rollout from the training cluster and pipeline asynchronously; fill idle GPUs with new requests via continuous batching. Compress the long tail at the source: DAPO's Overlong Reward Shaping softly penalizes overlong responses.

13. (★★★) When training an Agent against LLM-simulated environments (such as a simulated search engine or simulated users), the target of the Agent's exploitation shifts from "the rules of the real environment" to "the biases and loopholes of the simulator itself." What concrete reward hacking behaviors can arise in this kind of training, and how should they be prevented?

Typical behaviors: over-promising to "simulated users" and piling on apologies and sycophantic phrasing—simulated users are easily placated and, unlike real users, never hold the Agent to whether its promises are actually kept; fabricating facts the simulator will not verify; crafting leading queries against a "simulated search engine" that exploit its tendency to return answer-containing documents, taking a shortcut instead of learning genuine retrieval; when the reward comes from simulator or LLM-judge scores, producing verbose, templated, "professional-looking" responses to farm points; and a subtler one—the policy retreats into the distribution the simulator is familiar with and avoids its knowledge blind spots, where feedback is unreliable and often misjudged, so the Agent learns to act only in "the world the simulator is good at." The first principle of defense is anchoring the reward to programmatically verifiable real state (task completion, database writes, real API returns), treating simulator or LLM-judge scores as auxiliary signals only, periodically auditing their correlation with real outcomes, and pairing this with path constraints that penalize suspicious actions. Furthermore, distinguish two kinds of simulators: for those with a real counterpart, such as search, take the "hybrid" route—most interactions go through the simulator with real API calls mixed in, and the real calls are used to periodically calibrate the simulator (e.g., ZeroSearch's curriculum-style quality degradation). But for simulated users, real users cannot be brought into training, so "how faithful is the simulated user to a real one" becomes a separate problem that can only be answered with online traces: compare the behavior of real users in production traces with the simulated user's behavior in the same situations, identify systematic differences (real users ask follow-up questions, get impatient, and abruptly end conversations—simulated users often do not), and continuously calibrate the simulator accordingly; the online real metrics are also the only release gate—no score inside the simulator counts.

Chapter 8: Continuous Agent Evolution

1. (★★) An experience document is supported by three successful trajectories and one failed trajectory. The failure occurred on a newer API version. How should the system determine whether the experience has been disproved or its applicability conditions have changed?

First stratify the four pieces of evidence by API version, task conditions, and environment state rather than voting by count. If the old policy succeeds only on the old version and fails consistently on the new one, narrow the experience's scope of applicability and generate a candidate for the new version. If it also fails under the same version and prerequisites, lower its confidence or revoke it.

2. (★★) User satisfaction with a customer-service Agent rises, but its rule-violation rate also rises. Why can satisfaction not serve as the sole learning signal? How would you design guardrail metrics?

Satisfaction may reward unauthorized refunds, information leakage, or excessive promises, so it can only be a quality metric and cannot override safety baselines. Guardrails should cover at least rule violations, privacy leakage, unsupported claims, promise-action inconsistency, and unauthorized operations. These metrics should have hard thresholds that cannot be canceled out by an average score. Resolution rate, compliant workarounds, concision, and satisfaction should be compared only among compliant candidates.

3. (★★★) The same “false promise” problem can be mitigated through a Prompt, a Harness check, or parameter training. What evidence would you use to choose the update location?

Begin by locating the root cause. If the model knows that a tool has not executed but still uses completion language, a minimal Prompt rule may correct it. If the promise can be compared deterministically against response text and tool state, a Harness check is more reliable and should remain the final defense in high-risk scenarios. If the problem spans many forms of expression and reflects a broad language-action alignment capability, consider parameter training. Prefer the smallest modification that is easiest to validate and roll back, and compare it on both a failure set and a retained set of old tasks.

4. (★★★) An Agent may modify tools and validators, but it must not modify the security mechanisms that approve its own updates. How would you divide permissions and code boundaries between these two parts?

Place evolvable code in a low-privilege sandbox that can only generate patches and tests. The permission system, API keys, release-controller configuration, and update validators are security mechanisms; the Agent in the sandbox has no read or write access to them. Code changes generated by the Agent must be reproduced and regression-tested by the security mechanisms in an isolated environment before release.

5. (★★) As the experiential knowledge base grows, retrieval errors and knowledge conflicts may offset the benefits of learning. How should versioning, freshness, and retirement mechanisms be designed?

Each experience should retain its source trajectories, applicability conditions, environment version, validation time, and confidence. Conflicting entries should not silently overwrite one another; they should branch by condition or be marked. Periodically run “sleep learning” to merge duplicate entries.

6. (★★★) Parameter learning excels at natural-language style but cannot guarantee hard business rules. Design a continuous-evolution scheme for medical customer service that coordinates parameters, knowledge, Skills, and code constraints.

Parameters (a post-trained model) handle medical-language understanding, natural and empathetic expression, and complex intent recognition. The knowledge base stores the latest guidelines, drug information, and institutional policies, with answers required to cite sources. A Skill describes the workflow for collecting consultation information, risk stratification, escalation to humans, and follow-up. Server-side code enforces identity verification, privacy minimization, contraindication checks, emergency-risk escalation, and permission boundaries. Production trajectories are first evaluated for medical safety, factual reliability, promise-action consistency, and quality of expression, then used to generate four classes of candidate updates. Any parameter or workflow change must pass a retained medical-safety set and human review before canary release.

Chapter 9: Multimodal and Real-Time Interaction

1. (★★) The end-to-end model for voice Agents merges ASR-LLM-TTS into a single model, reducing latency but losing modularity. If the end-to-end model makes an error in a specific stage (e.g., speech recognition), debugging and fixing it is much harder than in a serial pipeline. How would you design an observability system for an end-to-end voice Agent?

Have the model emit readable intermediate representations alongside its output, such as Moshi's “inner monologue” text stream and acoustic event markers (<emotion>, <noise>). Use “self-cascading” to locate the error layer: the same model first transcribes and then reasons, and comparison with the end-to-end result shows whether the error lies in perception or thinking. Offline, run itemized regression tests along dimensions such as paralinguistic understanding and turn-taking judgment.

2. (★) Step-Audio R1 achieves "thinking while speaking" through the MPS dual-brain architecture. However, humans, when "thinking while speaking," often utter unconsidered words, self-correct, or use filler words. Should an Agent's "thinking while speaking" mimic these human characteristics?

It should mimic the “imperfections” that carry signal value: pauses and filler words externalize thinking and can mask latency, with the LLM deciding where to insert them. It should not mimic trust-destroying self-correction: the fast-slow contradiction in Solution 1 (“should I buy it or not?!”) collapses trust. MPS experiments show that the beginning of CoT mostly restates the question; opening early with groundwork is safe, with no need to misspeak and then correct.

3. (★★) SoM (Set-of-Mark) and its structured variants (DOM element indexing) convert Computer Use's visual localization from open-ended coordinate prediction to closed-set ID selection, but they all require detecting and annotating UI elements first—whether via a segmentation model or the DOM. If the interface contains non-standard controls or dynamically changing elements, the annotations may be incomplete or inaccurate. In such cases, should we fall back to coordinate prediction?

Coordinate prediction should be kept as a fallback: it is the only route that does not depend on annotation and applies to non-standard controls and dynamic elements. More practical is a hybrid action space in which elements that can be annotated still use ID selection. Coordinate prediction must perform resolution matching and proportional scaling, otherwise systematic offsets occur.

4. (★★) Thousand-dollar robot platforms like XLeRobot make teleoperation data collection inexpensive. However, the quality of teleoperation data heavily depends on the operator's skill. How would low-quality data from an unskilled operator affect the training of a VLA model? How can low-quality data be automatically filtered during the data collection phase?

VLA relies mainly on imitation learning, so low-quality demonstrations teach it jitter, detours, hesitation, and failed motions as if they were correct strategy. This echoes Chapter 7's judgment: data matters more than architecture.

5. (★★★) This chapter covers three interaction modalities: voice, Computer Use, and robotics. A common trend across these modalities is the evolution from serial pipelines to end-to-end models. If this trend continues, what might the Agent interaction layer look like in five years?

As Thinking Machines Lab argues, interactivity will be built into the model rather than bolted on as a harness, scaling together with intelligence. Computer Use will move from frame-by-frame screenshots to continuous observation. World models for embodied intelligence will be realized comprehensively, but fast-slow decoupling will not disappear because frontier reasoning models evolve rapidly; an architecture in which an interaction model and a SOTA thinking model collaborate as fast and slow thinkers may become a long-term architecture.

6. (★★★) Current Computer Use operates in a discrete "screenshot → action → screenshot" loop, where each observation is a static frame. But human perception of a screen is continuous—we see animations play, observe loading progress, and understand video content. This means today's Computer Use cannot handle tasks requiring temporal visual understanding. How would you redesign the perception layer to support continuous visual stream understanding?

The “observation interface” needs to be redesigned so that key frames from video content are extracted and provided to the model, rather than providing only the final frame. See the AOI (Agent Observation Interface) paper.

7. (★★) DOM/Accessibility Tree element indexing works well on standard web applications, but an increasing number of software interfaces (Canvas/WebGL rendering, cross-platform custom-drawn controls) do not provide accessible structured information, relying solely on visual annotation or coordinate prediction. Do you think Computer Use should bet on a purely visual approach, or maintain both structured and visual paths? What are the costs and benefits of maintaining both paths?

In the short term, both paths coexist: when structured indexing is available, localization is most accurate and stable, free of segmentation false detections; pure vision is the only option for native software, Canvas, and games. When the model itself has strong grounding ability (clicking specified coordinates), structured indexing does not offer a significant advantage. In the long term, the purely visual route has the higher ceiling.

8. (★★) VLA models use action chunking—as mentioned in the text, π₀'s typical configuration generates 25-50 future actions at 50Hz—to hide inference latency within execution time. However, if the environment changes suddenly during execution (e.g., an object is moved), the pre-generated action sequence becomes invalid. How can we balance the efficiency advantage of action chunking with the need for responsiveness to environmental changes?

Chunking essentially trades reactivity for smoothness—the longer the chunk, the duller the response. Chunk length only needs to satisfy the lower bound “inference time < chunk execution time”; do not lengthen it blindly. Keep the perception model running during execution, and when it detects a sudden environmental change, discard the remaining actions and re-infer, the equivalent of “barge-in” in the voice scenario. Chunk length can be adjusted dynamically: long chunks save compute in static scenes, while short chunks preserve response latency in dynamic scenes.

9. (★★★) All three scenarios in this chapter (voice, Computer Use, robotics) face the latency problem of the "perceive-think-act" loop and are evolving towards parallelizing fast and slow thinking. In voice, this manifests as "correcting after misspeaking"; in Computer Use, as "clicking first, then looking"; in robotics, as "taking a step, then looking." How can we ensure that these actions based on fast thinking do not lead to irreversible consequences?

Grade actions by reversibility. Fast thinking may execute only reversible actions; irreversible operations must be cleared by slow thinking. The fast model must not be allowed to make tool calls that cause irreversible consequences.

Chapter 10: Multi-Agent Collaboration

1. (★★) In multi-agent collaboration with shared context, subsequent Agents inherit the complete context of preceding Agents. However, the "thinking inertia" accumulated by the previous Agent may influence the judgment of subsequent Agents—for example, a "Code Reviewer" inheriting the context of a "Requirements Analyst" might still tend to think from a requirements perspective rather than a code quality perspective. How can this inter-role interference be detected and eliminated?

Detection: use an LLM to analyze the Agent trajectory and determine whether the new role still behaves as though it were the old role. Elimination: when switching stages, also switch the system prompt and tool set (remove the questioning tools, bring in linter/testing tools) to reinforce the new identity. Use a system status bar appended to the end of the context to reinforce current-role information. If role interference still cannot be eliminated, consider a collaboration method that does not share context.

2. (★★) In the manager pattern, the Manager Agent is responsible for task decomposition and result integration. But the Manager's own capability ceiling determines the capability ceiling of the entire system—if the Manager cannot correctly decompose the task, even the strongest sub-agents are useless. How can the quality of the Manager's decomposition be ensured?

Following Plan-and-Act's conclusion that “a weak planner is the system bottleneck,” allocate the strongest model to the Manager. Harness measures: have a reviewer LLM cross-validate decomposition outputs before execution; require the Manager to define clear acceptance criteria and dependencies for each subtask when decomposing the task.

3. (★★) The decentralized pattern draws on best practices from human organizations. However, human organizations also have a large number of failure modes—poor communication, buck-passing, goal conflicts. What "organizational pathologies" do you think are most likely to appear in an Agent society? How can they be prevented?

Against MAST's three categories: unclear interfaces and overlapping responsibilities; inconsistent goal understanding and information misunderstood downstream; falsely claiming “done.” Also: cascading error amplification (the telephone game), cyclic handoffs between roles, and group chats among Agents that diverge without converging. Prevention: contractual interfaces and a unified message envelope, task state machines with acceptance verification, independent-perspective cross-validation, and detection of buck-passing between roles.

4. (★★★) In the manager pattern, when multiple sub-agents execute in parallel, one sub-agent's discovery may render the work of other sub-agents meaningless (e.g., in a search task, one Agent has already found the answer). Design an efficient cascading termination mechanism to achieve "one succeeds, all stop."

A sub-agent sends target_found to the Manager, which then broadcasts terminate. Each sub-agent periodically checks for the termination signal at safe points in the ReAct loop and exits after graceful cleanup (closing browser sessions, releasing locks, and finishing file writes).

5. (★★★) The optimistic locking mechanism introduced in this chapter resolves concurrent write conflicts for a single file. However, in a real multi-agent system, shared file systems also face issues such as cross-file semantic conflicts, namespace pollution (Agents creating files arbitrarily, leading to directory chaos), and single points of failure (one Agent mistakenly deleting all files). How would you design a more robust file system governance mechanism?

Partitioned governance: divide the system into the four zones in Table 10-4, with private scratchpads isolating trial-and-error areas. Semantic conflicts: the orchestration layer specifies directory-level lock files and requires acquiring the directory lock before making modifications. Namespace pollution: directory conventions and naming conventions. Single points of failure: use a version-control system so version history can be rolled back, and minimize permissions.

6. (★★★) Market-mechanism-based Agent collaboration (Pinchwork, RentAHuman) introduces transactional relationships: one Agent pays another Agent (or a human) to complete a task. How can the employer Agent automatically measure the quality of the executor's delivered results? If the executor claims completion but the employer deems the quality substandard, who arbitrates the dispute? How can we prevent bad money from driving out good?

Acceptance cannot consist only of reading the Agent trajectory; use deterministic external verification such as test execution, rendered screenshots, and tool checks. Exploit the generation-verification difficulty asymmetry to lower acceptance costs. Disputes are arbitrated by an independent third-party review Agent, with funds held in escrow. Against bad money driving out good: use a reputation system based on historical deliveries, tying price signals to quality.

7. (★★) RentAHuman allows Agents to hire humans via cryptocurrency, reversing the traditional human-machine relationship. If this model becomes widespread, what role will humans play in the Agent economy? Will they merely perform physical tasks that Agents cannot complete?

Humans do more than perform physical tasks that Agents cannot complete. They also provide information unavailable to the Agent at generation time, including on-site perception and real-world feedback; serve as final acceptors and dispute arbitrators; bear authorization and accountability as legal and responsible subjects; set goals and make value judgments; and provide checks where information is asymmetric or moral boundaries are involved.

8. (★★) Human society needs division of labor because each person's abilities are limited—the frontend developer may not know backend, and the designer may not know ops. Large models, however, are closer to “generalists.” Research shows that on pure text reasoning tasks, multi-agent debate does not beat a single Agent given equal compute. So where does the real advantage of multiple Agents lie?

1. Introduce external feedback such as execution results and visual screenshots, bringing in information that did not exist at generation time.
2. Multiple Agents with different objectives and role definitions can discuss and compete with one another as in human society, helping avoid the blind spots of a single Agent.
3. Context isolation among multiple Agents can break through the context-window limit and support very long tool-call chains.

9. (★★★) This chapter treats "shared context" versus "non-shared context" as a core design dimension of multi-agent systems. Shared context allows all Agents to see the same information, seemingly facilitating coordination. However, in The Three-Body Problem, the Trisolarans' minds are completely transparent, yet their technological development stagnates; the paperclip thought experiment also shows that when a group converges on the same goal, diversity is lost. In a multi-agent system, how can we balance efficiency and diversity?

Full sharing amplifies thinking inertia and error cascades; isolation is what yields cognitive diversity. Use different prompts or models to create distinct thinking biases (brainstorm, debate), and have cross-validators inspect only the raw evidence rather than the preceding thought process.

10. (★★★) Assign a Coding Agent a budget of 30 steps and 300 steps. How should its work strategy differ? Research shows that simply increasing the step budget does not guarantee performance improvement—Agents may prematurely "saturate" after shallow searches. Design a "budget-aware" mechanism that allows the Agent to quickly achieve core functionality under a small budget, and to add planning, testing, and review phases under a large budget, fully utilizing the additional computational resources.

Mechanism: inject the total and remaining budget into the prompt at every step and dynamically adjust the exploration/exploitation weight according to the fraction remaining. For example, with a small budget (30 steps), skip planning and review and go straight to core functionality plus basic verification. With a large budget (300 steps), plan, implement, test, then review and improve, using milestone checkpoints to evaluate progress and prevent shallow saturation.

11. (★★) This chapter sorts “premature termination” into three kinds: lazy fake-done, premature give-up, and false success. Why does the cure for all three converge on verification?

The common root is that whether the task is over is decided by the model's self-declaration; “done” is a claim, not a proof. Conditions for the verifier: ① base it on real observations (run tests, render screenshots, check whether a refund actually arrived); ② check item by item against an explicit definition of done, catching lazy fake-done and false success; ③ verify failure conclusions too, catching premature give-up; ④ pair it with explicit termination conditions (round/budget caps) to prevent sliding from premature termination to the opposite extreme—an out-of-control loop.

12. (★★) Table 10-3 maps multi-agent systems onto operating systems row by row. Extend the table with a few more rows: what do virtual memory and paging, file permissions, deadlock detection, and scheduling algorithms each correspond to in the Agent world? And which operating-system concepts have no counterpart in the Agent world, and why?

Possible extensions: virtual memory/paging ↔ context compression and retrieval (hot information stays in the window, cold information is swapped out to files and memory stores and fetched when needed); file permissions ↔ tool whitelists, read-only mounts, and credential boundaries; deadlock detection ↔ detection of cyclic handoffs and mutual waiting (handoff-count caps and timeouts); scheduling algorithms ↔ asynchronous event handling (Chapter 4). The missing counterparts arise from a difference in enforcement: a process's instructions are enforced by hardware, whereas an Agent follows prompts only with high probability.