Question Bank — Domain 4: Prompt Engineering & Structured Output

25 items. 20% of the exam (~12 items).

Block A — Prompt precision and false positives (items 1–11)

1

An automated code reviewer flags many correct comments as "stale documentation." The current instruction is "flag outdated comments." What is the most effective replacement?

A. "Flag comments only when the claimed behaviour contradicts the actual code behaviour."
B. "Be conservative when flagging comments."
C. "Only flag comments you are highly confident are outdated."
D. "Flag comments that appear to be older than the surrounding code."

Answer

An explicit, checkable criterion. The model can evaluate "does the comment's claim contradict the code?" consistently; it cannot evaluate "outdated."

B — The exam's canonical failing instruction. No operational definition.
C — Relies on uncalibrated self-reported confidence.
D — Comment age isn't observable from the code, and staleness isn't about age. Plausible-sounding and unimplementable.

Reflex 7.

2

Why does the exam treat false positives in automated code review as more damaging than false negatives?

A. They consume more tokens per finding.
B. They destroy developer trust, after which true positives are ignored too.
C. They cause CI pipelines to fail, blocking merges.
D. They are harder to measure than false negatives.

Answer

Trust is the operative concern. A tool developers ignore has zero value regardless of its recall.

A — Not the reasoning.
C — Depends on configuration, and a blocking failure is a symptom rather than the underlying cost.
D — False; they're the easier of the two to measure.

3

Analysis shows 70% of a reviewer's false positives come from one category: "possible null dereference." Which two responses are appropriate? (Select two.)

A. Temporarily disable that category while its criteria are rewritten.
B. Rewrite the category's criteria with explicit qualifying conditions and counter-examples.
C. Instruct the model to be more careful with null-dereference findings.
D. Lower the severity of that category from critical to informational and keep reporting it.

Answer

A and B.

Disabling stops the trust damage immediately; rewriting fixes the cause. Together they're the exam's expected pair.

C — Exhortation.
D — Still reports 70% noise, now labelled differently. Developers still learn to ignore the tool.

4

Severity criteria are being written for a review agent. What makes them operational rather than aspirational?

A. Concrete code examples of what qualifies at each level, including explicit "NOT critical" and "NOT major" counter-examples.
B. A numeric scoring rubric the model applies to each finding.
C. A instruction to match the severity conventions of the existing issue tracker.
D. A requirement to report a confidence score alongside each severity.

Answer

Examples at the boundary — especially negative ones — define the criterion's edges, which prose alone can't.

B — A rubric the model self-scores against is still judgement without a definition.
C — Points at conventions it can't see.
D — Adds uncalibrated confidence to an unfixed criterion.

5

How many few-shot examples does the guide recommend, and what should they demonstrate?

A. 2–4 targeted examples, including boundary and near-miss cases.
B. 1 canonical example of correct output.
C. 8–12 examples covering every input variation encountered.
D. As many as fit in the context window.

Answer

Few, targeted, boundary-focused.

B — Over-fitted to one shape.
C, D — Volume without signal; both lengthen context and risk lost-in-the-middle.

6

Which outcomes do well-chosen few-shot examples produce? (Select three.)

A. Greater consistency across similar inputs.
B. Better handling of ambiguous cases.
C. Reduced hallucination.
D. Elimination of semantic errors in structured output.

Answer

A, B, and C. (The fourth guide-listed benefit is generalization.)

D — Overclaims. Examples reduce semantic error rates; nothing eliminates them. Semantic correctness needs a computed validation step.

7

An extraction reviewer incorrectly flags negative line-item amounts (legitimate discounts) as errors. Most effective fix?

A. Add a few-shot example showing an invoice with a legitimate negative discount line and the correct output.
B. Add "negative amounts may be valid" to the prompt.
C. Add a schema constraint permitting negative numbers.
D. Disable amount validation entirely.

Answer

A demonstrated boundary case. The model sees the exact pattern and the correct handling.

B — Better than nothing, but a bare assertion doesn't show which negatives are valid.
C — The schema probably already permits them; this is a semantic judgement problem, not a type problem.
D — Removes a useful check to fix one false positive.

8

Which schema field best supports analysing why a reviewer produced a given finding?

A. detected_pattern — recording what the model matched on.
B. confidence — the model's self-assessed certainty.
C. severity — the finding's importance.
D. line_number — where the finding occurred.

Answer

Recording the reasoning trigger lets you cluster false positives by cause and fix the criteria that produced them.

B — Uncalibrated, and it tells you nothing about why.
C, D — Useful metadata; neither explains the decision.

9

When is strengthening the wording of a prompt instruction the appropriate fix?

A. When the instruction governs a financial action such as issuing a refund.
B. When the instruction governs formatting or style preferences.
C. When the instruction governs a compliance-mandated approval sequence.
D. When the instruction governs which tool the model should select.

Answer

Style has no irreversible consequence, so a probabilistic mechanism is acceptable.

A, C — Deterministic enforcement — hooks and gates.
D — Tool descriptions, not prompt emphasis.

10

An item asks for the "most effective first step" to reduce a reviewer's false positive rate. Which option shape is most likely correct?

A. Rewrite the criteria for the highest-volume false-positive category.
B. Replace the review architecture with a multi-agent consensus system.
C. Fine-tune a model on the team's historical review decisions.
D. Add a second review pass that filters the first pass's findings.

Answer

"Most effective first step" rewards the smallest high-leverage change at the root cause.

B — Over-engineering.
C — Fine-tuning is explicitly out of scope.
D — Adds cost and latency to filter noise rather than not generating it.

11

Order these three mechanisms from first to last as the exam prioritizes them for improving output quality: few-shot examples, explicit criteria, programmatic enforcement.

A. Explicit criteria → few-shot examples → programmatic enforcement.
B. Programmatic enforcement → explicit criteria → few-shot examples.
C. Few-shot examples → explicit criteria → programmatic enforcement.
D. Explicit criteria → programmatic enforcement → few-shot examples.

Answer

Criteria first (cheapest, highest leverage, defines the decision rule). Few-shot next, for boundaries criteria can't express. Enforcement last — reserved for consequences, and it doesn't improve judgement, only bounds it.

B — Enforcement can't substitute for an undefined criterion.
C — Examples without a criterion give the model shapes to imitate but no rule.
D — Skips the boundary-demonstration step where criteria are still ambiguous.

Block B — Structured output and validation (items 12–20)

12

All extraction outputs are valid JSON with correct types and all required fields. In 6% of invoices, the line item amounts do not sum to the extracted total. What should be added?

A. A validation step that recomputes the total from the line items and flags mismatches, with retry-and-feedback on failure.
B. A stricter JSON schema with tighter numeric constraints.
C. A larger max_tokens value.
D. tool_choice: "any" for more flexibility.

Answer

Schema-valid and semantically wrong. A schema cannot express a cross-field arithmetic relationship.

B — The named trap. No numeric constraint expresses a sum across array elements.
C — Would cause truncation, not arithmetic errors.
D — Loosens an existing guarantee and touches nothing semantic.

Reflex 8.

13

Which are guaranteed by a JSON schema with a forced tool call? (Select two.)

A. Valid JSON with all required fields present.
B. Correct types and enum values from the allowed set.
C. The correct value appearing in the correct field.
D. Internal arithmetic consistency across fields.

Answer

A and B. Syntax, not semantics. C and D are exactly what a schema cannot guarantee.

14

Roughly 30% of invoices have no purchase order number. The schema declares purchase_order as a required string. What happens and what is the fix?

A. The model fabricates PO numbers; make the field nullable with a description stating "null if absent; never infer."
B. The extraction fails schema validation; add a default value of "N/A".
C. The model omits the field; add it to the required list.
D. The model returns an empty string; add a minimum length constraint.

Answer

A required non-nullable field forces the model to produce something, and the something is invented. Nullability gives it a correct way to say "absent."

B — "N/A" is a sentinel string that pollutes downstream data and still doesn't distinguish "absent" from "unreadable."
C — It's already required; that's the cause.
D — A length constraint makes fabrication more likely by forbidding the empty escape.

15

Which two enum values does the guide recommend adding to a closed document-type enum? (Select two.)

A. "unclear"
B. "other"
C. "unknown_vendor"
D. "pending_review"

Answer

A and B.

"unclear" = can't determine from the document. "other" = determinate but not in the list. Both must be paired with a detail string field.

C, D — Not the pattern; D confuses an output category with a workflow state.

16

Why must the escape-hatch enum values be paired with a detail string field?

A. To satisfy schema validation requirements.
B. So you can see what "other" actually was and evolve the enum accordingly.
C. To give the model a place to explain its reasoning.
D. To carry the confidence score for the classification.

Answer

The detail field is the feedback loop: after a month, 200 other values turn out to be "proforma invoice," which becomes a proper enum member.

A — Not a validation requirement.
C — Reasoning isn't the purpose; the category name is.
D — Different field, different purpose.

17

A strict schema declares issue_date as a string. Extractions still contain inconsistent date formats and misread ambiguous dates. What is needed?

A. Explicit format-normalization rules in the prompt: ISO 8601 output, a stated rule for day-first vs month-first ambiguity, and a null-plus-flag fallback.
B. A regex pattern constraint on the schema field.
C. A date parsing library applied to the output.
D. Retry with error feedback until the format is correct.

Answer

A regex would catch a malformed shape, but "3/4/25" is well-formed and ambiguously interpreted. The schema constrains shape; the rules constrain interpretation.

B — Half-right. It would enforce ISO 8601 shape but not correct interpretation — and it can't rescue a genuinely ambiguous source.
C — Post-hoc parsing faces the same ambiguity with less context.
D — Retrying an ambiguous source produces a different guess, not a right answer.

18

What should a retry after a validation failure include? (Select three.)

A. The original source document.
B. The model's previous failed extraction.
C. The specific validation errors that occurred.
D. An instruction to try harder and be more careful.

Answer

A, B, and C.

D — Exhortation. Adds nothing the three concrete elements don't already supply.

19

An extraction pipeline retries up to five times when a required field is missing. For documents that genuinely lack the field, retries either exhaust or eventually return a plausible-looking value. What should change?

A. Make the field nullable and route persistent absences to human review; do not retry for absent information.
B. Increase the retry limit to ten.
C. Add a stronger instruction not to fabricate values.
D. Lower the temperature so retries are more deterministic.

Answer

Retry cannot recover information absent from the source. Continuing to retry actively invites fabrication.

B — More attempts at an impossible task; more chances to invent something.
C — Guidance against a structural pressure the schema creates. Fix the schema.
D — Makes the same wrong answer more reproducible.

20

Which three fields implement in-model self-correction for invoice totals? (Select three.)

A. stated_total
B. calculated_total
C. conflict_detected
D. total_confidence

Answer

A, B, and C.

The model reports what the document says, sums the line items itself, and flags disagreement — all while the document is still in context.

D — Self-reported confidence, uncalibrated. Not the mechanism.

Note: you still re-sum in your own code. Two independent checks.

Block C — Batching and review (items 21–25)

21

Which statements about the Message Batches API are correct? (Select three.)

A. It offers roughly 50% cost savings.
B. Processing may take up to 24 hours with no latency SLA.
C. Requests are correlated to results by custom_id.
D. It supports multi-turn tool calling within a single request.

Answer

A, B, and C.

D — The key limitation, stated backwards. Single-pass structured output with a forced tool works fine; an agentic loop does not.

22

A pipeline must guarantee extraction results within 30 hours of document arrival. Documents arrive continuously. Batch processing takes up to 24 hours. What is the maximum submission interval?

A. 30 hours
B. 24 hours
C. 6 hours
D. 1 hour

Answer

Worst case = wait for the next submission + the full processing window. A document arriving just after a submission waits the full interval, then up to 24 hours:

interval + 24 ≤ 30  →  interval ≤ 6

A, B — Forget the first term entirely.
D — Safe but not the maximum. (The guide's own variant uses 4-hour windows for margin — a choice, not the limit.)

23

Some requests in a 50,000-document batch returned invalid output. What is the correct response?

A. Resubmit only the failed custom_ids, with modifications such as a corrected prompt or relaxed schema.
B. Resubmit the entire batch with the corrected prompt.
C. Process the failed documents synchronously at full price.
D. Accept the failures as within tolerance.

Answer

custom_id correlation exists so you can target exactly the failures.

B — Pays for 50,000 successes twice.
C — Loses the discount unnecessarily; the failures aren't latency-sensitive.
D — Not an engineering answer.

24

Before a first-time batch run over 50,000 documents, what should be done?

A. Run a sample synchronously and refine the prompt against measured results.
B. Submit the full batch and inspect the results.
C. Split into ten batches of 5,000 to limit blast radius.
D. Set a lower max_tokens to control cost.

Answer

A flawed prompt costs the full amount and delivers 50,000 flawed extractions 24 hours later.

B — The failure mode being avoided.
C — Reduces blast radius without validating the prompt; you'd still discover the flaw after paying for 5,000.
D — Risks truncation and doesn't address quality.

25

An extraction pipeline's output is being reviewed for quality before automation. Which approach does the exam rank highest?

A. A separate reviewer instance with fresh context evaluating the extractions.
B. The extracting instance reviewing its own output.
C. The extracting instance with extended thinking enabled for a second look.
D. A rule-based post-processor checking for common error patterns.

Answer

Independence, not depth. A fresh context doesn't carry the assumptions that produced the error.

B — Self-review carries the contaminating assumptions.
C — More reasoning in the same contaminated context.
D — Valuable and complementary — deterministic checks belong in the pipeline — but the item asks which review approach ranks highest, and rule-based checks only catch the patterns you already anticipated.

Scoring

Score	Read
22–25	Strong.
18–21	Solid; re-read Chapter 10 §1 and §4 on the misses.
13–17	Re-read Chapters 09–10 fully.
< 13	Items 12, 14, 19, 21, 22 carry the domain's core. Re-study those first.