Mock Exam 1

60 items · 120 minutes · four scenarios

Domain weighting matches the real blueprint: D1 16 · D2 11 · D3 12 · D4 12 · D5 9.

Scenarios in this paper: 1 (Customer Support), 2 (Code Generation), 3 (Multi-Agent Research), 6 (Structured Extraction).

Instructions

Answer every item. There is no penalty for a wrong answer.
Each item states how many options to select. Unless stated, select one.
Read each scenario setup once, carefully, then work its fifteen items.
Target pace: 30 minutes per scenario. Flag anything you haven't converged on in 90 seconds and move.
Do not look at the answer key until you have finished all 60.

Answers: mock-exam-1-answer-key.md. Scoring: scoring-grid.md.

Scenario A — Customer Support Resolution Agent

A retail company is deploying a customer support agent built on the Claude Agent SDK. It handles order enquiries, returns, and refunds through four custom MCP tools: get_customer, lookup_order, process_refund, and escalate_to_human. The target is 80%+ first-contact resolution. Conversations frequently run 20+ turns. Refunds above $500 require human approval under company policy, and refunds are only permitted within 90 days of shipment.

1

Post-launch review finds that in a small number of conversations process_refund executed before any customer verification had occurred, despite a system prompt instruction requiring get_customer first. Which approach most reliably prevents this?

A. Move the instruction to the top of the system prompt and add emphatic wording.
B. Add a PreToolUse hook that blocks process_refund unless a verified customer ID is present in session state.
C. Add a few-shot example demonstrating the correct call ordering.
D. Add a PostToolUse hook that logs unverified refunds for daily audit.

2

The agent occasionally miscalculates whether an order falls inside the 90-day refund window. Investigation shows get_customer returns created_at as a Unix epoch integer, lookup_order returns shipped_at as an ISO 8601 string, and a third tool returns status as a numeric code. Best fix?

A. Document each format in the corresponding tool's description.
B. Instruct the agent in the system prompt to normalize all timestamps before reasoning about them.
C. Add a PostToolUse hook that normalizes timestamps and status codes before results reach the model.
D. Use a model with stronger arithmetic capability.

3

A customer requests a $780 refund. Company policy requires human approval above $500. Which implementation satisfies the policy most reliably?

A. A system prompt instruction stating that refunds above $500 must be escalated.
B. A PreToolUse hook that blocks process_refund above $500 and redirects to escalate_to_human.
C. Removing process_refund from the agent and escalating all refunds.
D. Adding a max_amount field to the refund tool's schema with a maximum of 500.

4

The agent's loop terminates prematurely in roughly 15% of conversations, usually right after the first tool call. The implementation exits when the assistant response contains any text content. What is the correct termination condition?

A. Exit when stop_reason is not "tool_use".
B. Exit when the response contains no tool_use blocks and the text is non-empty.
C. Exit after a maximum of 10 iterations.
D. Exit when the response text contains a completion phrase.

5

In one turn the agent calls both get_customer and lookup_order. How are the results returned?

A. Two separate user messages, in call order.
B. One user message containing two tool_result blocks, each with its tool_use_id.
C. One user message with both results concatenated into a single text block.
D. Two assistant messages, since the assistant made the calls.

6

A customer's name matches three accounts. What should the agent do?

A. Select the account with the most recent order activity.
B. Ask the customer for an additional identifying detail such as an email address or ZIP code.
C. Escalate to a human, since verification has failed.
D. Proceed with the most likely match and note the ambiguity in the audit log.

7

A refund is requested for an order shipped 139 days ago. What should process_refund return?

A. isError: true, errorCategory: "business", isRetryable: false, with the reason and the escalation alternative.
B. isError: true, errorCategory: "validation", isRetryable: true.
C. isError: false, with a message explaining the policy.
D. isError: true with the message "refund not permitted".

8

Which are legitimate escalation triggers for this agent? (Select three.)

A. The customer explicitly asks for a human.
B. The required action falls outside documented policy.
C. The agent cannot make progress.
D. The case is unusually complex.

9

A customer writes: "Honestly I've had enough of this — just put me through to a person." What should the agent do?

A. Escalate immediately.
B. Attempt one resolution first, since most requests can be handled automatically.
C. Ask the customer to summarize the problem so the handoff summary is complete.
D. Escalate only if the agent's confidence in resolving the issue is low.

10

What should the escalate_to_human payload contain?

A. The complete conversation transcript.
B. Customer ID, root cause, refund amount, and recommended action.
C. A one-line escalation reason.
D. The transcript plus a confidence score for each proposed action.

11

In long conversations the agent sometimes offers refund amounts inconsistent with what was agreed earlier, and fails to honour commitments the customer says they received from phone support. Earlier turns are summarized. Most effective fix?

A. Maintain a structured "case facts" block outside the summarized history, re-injected each turn.
B. Switch to a model with a larger context window.
C. Summarize less aggressively.
D. Have the agent re-read the full transcript before each response.

12

lookup_order returns 40 fields; the agent reasons over about 5. What is the effect and the fix?

A. Roughly 8× more context than needed on every call; trim the output, e.g. in a PostToolUse hook.
B. Slower tool execution; add pagination.
C. Increased hallucination risk; add an output schema.
D. No meaningful effect; unused fields are ignored.

13

The team adds nine more tools, bringing the total to thirteen. Tool-selection accuracy drops. Which two responses are appropriate? (Select two.)

A. Consolidate overlapping tools and remove rarely used ones.
B. Scope tools per agent so each has only what its role requires.
C. Add a system prompt section listing all thirteen tools and when to use each.
D. Force tool_choice: "any" so the model always commits to a tool.

14

A new check_return_eligibility tool is frequently used where lookup_order was intended. Both descriptions begin "Retrieve information about an order." Most effective first step?

A. Rewrite both descriptions with explicit boundaries, each referring to the other.
B. Merge them into a single tool.
C. Add a system prompt instruction distinguishing them.
D. Rename them.

15

Escalation logic is being designed. The agent currently escalates too rarely when a human is genuinely needed and too often otherwise. Most effective approach?

A. Define the three explicit situational triggers and reinforce each with 2–3 few-shot examples, including near-miss negatives.
B. Escalate whenever self-reported confidence falls below 0.7.
C. Escalate whenever sentiment analysis detects frustration.
D. Escalate whenever a case requires more than five tool calls.

Scenario B — Code Generation with Claude Code

A 40-developer platform team is standardizing on Claude Code. The monorepo contains a React frontend, a Python API, and Terraform infrastructure. They want consistent conventions, repeatable workflows, and effective use of plan mode. Some developers have personal preferences they don't want to impose on others. The team also runs Claude Code in CI.

16

Conventions that every developer on the repository should automatically pick up belong where?

A. ~/.claude/CLAUDE.md
B. CLAUDE.md at the repository root
C. .claude/commands/conventions.md
D. ~/.claude.json

17

One developer prefers exhaustive docstrings but the team hasn't adopted that standard. Where does the preference go?

A. Project CLAUDE.md
B. ~/.claude/CLAUDE.md
C. .claude/rules/docstrings.md
D. .claude/skills/docstrings/SKILL.md

18

React testing conventions must apply to test files, which sit beside their components across dozens of directories. Best mechanism?

A. A CLAUDE.md in each directory containing tests.
B. A .claude/rules/testing.md file with paths: ["**/*.test.tsx"].
C. A testing section in the root CLAUDE.md.
D. A /test-conventions slash command.

19

The team's agent needs a GitHub MCP server, shared across all developers, authenticating with a token. What is correct?

A. Commit .mcp.json at the repository root with the token referenced as ${GITHUB_TOKEN}.
B. Commit .mcp.json with the token value inline, and restrict repository access.
C. Have each developer add the server to ~/.claude.json.
D. Commit ~/.claude.json to the repository.

20

The team's API conventions are maintained in docs/api-conventions.md. How should CLAUDE.md incorporate them?

A. @import ./docs/api-conventions.md
B. Copy the contents into CLAUDE.md.
C. Reference the path in prose.
D. Symlink CLAUDE.md to the docs file.

21

A developer must locate every React component test file in the monorepo. Which tool?

A. Glob with **/*.test.tsx
B. Grep for describe(
C. Read on the test directory
D. Bash with find . -name "*.test.tsx"

22

The team wants a /review-pr workflow every developer can run. What do they create?

A. .claude/commands/review-pr.md, committed.
B. ~/.claude/commands/review-pr.md
C. An entry in a commands array in .claude/config.json.
D. .claude/skills/review-pr/SKILL.md

23

A database migration procedure is needed monthly and involves long, verbose exploration. Where should it live, and with what frontmatter?

A. In the project CLAUDE.md.
B. In .claude/skills/migrate-schema/SKILL.md with context: fork.
C. In .claude/rules/migrations.md with paths: frontmatter.
D. In ~/.claude/CLAUDE.md.

24

A developer must rename a variable that appears four times in one file. Which approach?

A. Edit with the variable name as the anchor.
B. Read the file, then Write the full updated contents.
C. Grep, then Edit.
D. Bash with sed -i.

25

Which situations call for plan mode? (Select three.)

A. A refactor touching 30 files across four modules.
B. Correcting a typo in a log message.
C. Choosing between three viable caching approaches.
D. Restructuring how authentication is layered.

26

A developer's discovery phase fills the conversation with search output and file contents that aren't needed afterwards. What addresses this?

A. Delegate discovery to the Explore subagent, which returns a summary.
B. Run /compact after discovery.
C. Switch to a larger context window.
D. Instruct the agent to read fewer files.

27

A developer reviewing generated code finds five separate problems. How should the feedback be delivered?

A. All five in one message.
B. One at a time, verifying each fix before the next.
C. Appended as a list to CLAUDE.md.
D. As five separate slash command invocations.

28

The CI pipeline runs Claude Code headlessly and needs machine-parseable, schema-constrained output. Which flags? (Select three.)

A. -p / --print
B. --output-format json
C. --json-schema
D. --ci

29

How does the CI run pick up the team's conventions?

A. From CLAUDE.md, which is read in headless mode too.
B. Conventions must be inlined in the CI prompt.
C. Via a --rules flag.
D. Via environment variables in the CI config.

30

A developer returns after three days to a session whose recorded decisions are useful, but several files have since been modified by teammates. What should they do?

A. --resume the prior session.
B. Start a fresh session and supply a structured summary of the prior decisions.
C. fork_session from the prior session.
D. /compact the prior session and continue.

Scenario C — Multi-Agent Research System

A market intelligence firm has built a research system in which a coordinator agent delegates to specialized subagents: web-search, document-analysis, synthesis, and report-generation. Clients submit broad research questions and expect reports with citations. Reports must be traceable to sources and must disclose any gaps in coverage.

31

For a question spanning product, pricing, distribution, regulation, and financials, the coordinator's four subtasks all concerned product features and pricing. The subagents returned accurate, well-sourced results within those areas. Root cause?

A. The subagents' tool access is too narrow.
B. The coordinator's task decomposition is overly narrow.
C. The synthesis agent is dropping dimensions when merging.
D. The subagents need larger context windows.

32

Fix for item 31?

A. Instruct the coordinator to partition the question across explicitly named dimensions and check coverage, re-delegating for gaps.
B. Increase the subagent count from four to eight.
C. Give the coordinator a fixed procedural checklist of research steps.
D. Have each subagent research the whole question independently and merge.

33

Three subagents given the identical instruction "research this company's competitive position" return heavily overlapping results citing many of the same sources. Which pattern addresses this?

A. Scope partitioning, with each prompt stating what the others cover.
B. Dynamic subagent selection.
C. Iterative refinement with a coverage check.
D. Reducing to a single subagent.

34

The coordinator does the research itself rather than delegating, despite prompt instructions to use subagents. What should be checked first?

A. Whether "Task" is in the coordinator's allowedTools.
B. Whether the subagent AgentDefinition prompts are detailed enough.
C. Whether the coordinator's model supports orchestration.
D. Whether parallel: true is set.

35

Four research subtasks are fully independent. How does the coordinator run them concurrently?

A. Set parallel: true in the configuration.
B. Emit four Task calls in a single assistant response.
C. Emit one Task call per turn.
D. Set concurrency: 4 in each AgentDefinition.

36

What is the description field of an AgentDefinition for?

A. Documentation for human maintainers.
B. The coordinator uses it to decide which subagent to route a task to.
C. It becomes the subagent's system prompt.
D. It is displayed to the end user.

37

A delegating prompt reads: "Research the regulatory position for the company we discussed, over the timeframe mentioned earlier." The subagent returns irrelevant results. Why?

A. It lacks web search tools.
B. The prompt relies on context the subagent does not have.
C. The prompt's key instruction is lost in the middle.
D. Its model is too small for regulatory analysis.

38

How should two subagents share intermediate findings?

A. Directly, over a shared channel.
B. Through the coordinator, which includes relevant findings in later delegating prompts.
C. Through a shared scratchpad file.
D. They cannot share findings.

39

A document-search tool returns [] with isError: false when the caller lacks permission on a collection. The final report states that no internal documents exist on the topic. What is wrong?

A. The synthesis agent should have verified the claim.
B. The tool reports an access failure as a valid empty result; it should return isError: true with a permission category.
C. The coordinator should have retried the search.
D. The report should have carried a confidence score.

40

A subagent can access three of four document collections. What should it return to the coordinator?

A. The three successful result sets plus a structured error for the fourth, with category, what was attempted, and alternatives.
B. A generic "search unavailable" error.
C. Only the three successful result sets.
D. An error that terminates the workflow.

41

Which two behaviours are anti-patterns when a component fails? (Select two.)

A. Returning success with partial results and no error indication.
B. Terminating the entire workflow.
C. Returning partial results with a structured error.
D. Annotating the coverage gap in the final report.

42

Which error categories should be marked non-retryable? (Select two.)

A. transient
B. validation
C. business
D. permission

43

Subagents return four paragraphs of reasoning followed by their findings. Over long runs the coordinator's output becomes vague. Which two changes help most? (Select two.)

A. Specify a structured return format — findings only, as {claim, source, date} records.
B. Increase the coordinator's context window.
C. Have the coordinator maintain a scratchpad file of accumulated findings.
D. Have subagents send findings directly to the synthesis agent.

44

The synthesis agent produces readable prose, but individual sentences can no longer be traced to sources. Requirement?

A. Preserve the claim→source mapping through synthesis in a structured representation.
B. Reattach citations after synthesis by matching claims to collected sources.
C. Append a bibliography of all consulted sources.
D. Cite only the highest-confidence sources.

45

Two credible sources report different figures for the same metric. What should the synthesis agent do?

A. Report both with attribution, including publication dates and any methodological difference.
B. Select the more authoritative source.
C. Use the most recent figure.
D. Report the average.

Scenario D — Structured Data Extraction

A financial services company extracts structured data from three document types — invoices, credit notes, and multi-currency statements — for downstream accounting integration. Output must validate against a JSON schema. The corpus is roughly 85% invoices, 11% credit notes, and 4% statements. Volume is 50,000 documents per month. A human review queue exists but leadership wants to reduce it.

46

All outputs are valid JSON with correct types and all required fields. In 6% of invoices the line item amounts do not sum to the extracted total. What should be added?

A. A validation step that recomputes the total from the line items and flags mismatches, with retry-and-feedback on failure.
B. A stricter JSON schema with tighter numeric constraints.
C. A larger max_tokens value.
D. tool_choice: "any" for more flexibility.

47

Which are guaranteed by a forced tool call with a JSON schema? (Select two.)

A. Valid JSON with all required fields present.
B. Correct types and enum values from the allowed set.
C. The correct value appearing in the correct field.
D. Internal arithmetic consistency across fields.

48

About 30% of invoices have no purchase order number. purchase_order is declared as a required string. What happens, and what is the fix?

A. The model fabricates PO numbers; make the field nullable with a description stating "null if absent; never infer."
B. Extraction fails schema validation; add a default of "N/A".
C. The model omits the field; add it to the required list.
D. The model returns an empty string; add a minimum-length constraint.

49

The document_type enum currently allows only invoice, credit_note, and statement. Which two values should be added? (Select two.)

A. "unclear"
B. "other"
C. "unknown_vendor"
D. "pending_review"

50

Why must those escape-hatch values be paired with a detail string field?

A. To satisfy schema validation.
B. So you can see what "other" actually was and evolve the enum.
C. To give the model a place to explain its reasoning.
D. To carry the classification confidence.

51

The schema declares issue_date as a string, but extractions still contain inconsistent formats and misread ambiguous dates like "3/4/25". What is needed?

A. Explicit format-normalization rules in the prompt: ISO 8601 output, a stated rule for day-first vs month-first ambiguity, and a null-plus-flag fallback.
B. A regex pattern constraint on the schema field.
C. A date parsing library applied to the output.
D. Retry with feedback until the format is correct.

52

What should a retry after validation failure include? (Select three.)

A. The original source document.
B. The model's previous failed extraction.
C. The specific validation errors.
D. An instruction to be more careful.

53

The pipeline retries up to five times when a required field is missing. For documents that genuinely lack the field, retries exhaust or eventually return a plausible-looking value. What should change?

A. Make the field nullable and route persistent absences to human review; do not retry for absent information.
B. Increase the retry limit to ten.
C. Add a stronger instruction not to fabricate values.
D. Lower the temperature.

54

Which three fields implement in-model self-correction for totals? (Select three.)

A. stated_total
B. calculated_total
C. conflict_detected
D. total_confidence

55

The pipeline reports 97% aggregate field-level accuracy. Leadership wants to remove human review. First step?

A. Measure accuracy per document type and per field, and automate only the combinations meeting the threshold.
B. Automate fully, since 97% exceeds the 95% target.
C. Continue reviewing a uniform random 5%.
D. Automate, and have the model flag low-confidence extractions.

56

Which sampling approach is correct for the remaining human review?

A. Stratified random sampling within each document type, vendor, and field.
B. Uniform random sampling across the corpus.
C. Reviewing the first 20 extractions of each batch.
D. Reviewing every statement, since statements are the hardest.

57

Under what condition does self-reported confidence become a usable routing signal?

A. When calibrated per field on a labeled validation set, with thresholds from measured accuracy.
B. When the model is instructed to be conservative in its estimates.
C. When two instances agree on the value.
D. Never.

58

Which statements about the Message Batches API are correct? (Select three.)

A. It offers roughly 50% cost savings.
B. Processing may take up to 24 hours with no latency SLA.
C. Requests are correlated to results by custom_id.
D. It supports multi-turn tool calling within a single request.

59

Extraction results must be available within 30 hours of a document's arrival. Documents arrive continuously and batch processing takes up to 24 hours. What is the maximum submission interval?

A. 30 hours
B. 24 hours
C. 6 hours
D. 1 hour

60

Some requests in a 50,000-document batch returned invalid output. What is the correct response?

A. Resubmit only the failed custom_ids, with modifications such as a corrected prompt or relaxed schema.
B. Resubmit the entire batch with the corrected prompt.
C. Process the failed documents synchronously at full price.
D. Accept the failures as within tolerance.

End of Mock Exam 1. Record your answers before checking the key.