Architecture in Practice / Threat Detection and AI

Automated Containment: When Must a Human Approve?

A practical five-check architecture worksheet for threat detection and ai, covering trust boundaries, ownership, evidence and exception decisions.

This practitioner worksheet translates a security architecture concern into a short, repeatable decision process. It is deliberately vendor-neutral and should be adapted to local operational constraints.

Purpose and scope

Use this review when deciding whether a control protects the boundary between Threat signal and evidence and Reversible action or approver. The goal is a documented, testable decision rather than a checklist completed for its own sake. Confirm asset ownership and process impact before changing a production environment.

Conceptual architecture · Trust-boundary design
Exposure: Unreviewed automation disrupts critical service
Conceptual architecture, not an incident reconstruction. Flow moves from source through a controlled trust boundary to a destination. Trade-off: Containment speed versus business continuity. No client environment or sensitive topology is depicted.

Five decision checks

  1. Classify each action as reversible, service-impacting, safety-impacting or irreversible.
  2. Permit unattended actions only within pre-approved scopes and confidence thresholds.
  3. Require a human approver for high-blast-radius actions or uncertain evidence.
  4. Capture original alerts, actor context and rollback steps before taking action.
  5. Test false-positive recovery and define a kill switch for the automation itself.

Evidence to retain

Keep an inventory entry, a named business or technical owner, a documented authorization decision, and a record of test results. Capture evidence for Scope and rollback before automation and demonstrate that unreviewed automation disrupts critical service is either prevented, detected or constrained. If a system cannot be tested safely, note that limitation and obtain an operational risk decision rather than treating absence of evidence as success.

Escalation and exception rule

Escalate when the chosen control could interrupt critical services, expose sensitive data or depend on unverified assumptions. A valid exception specifies affected assets, compensating monitoring, owner, expiry and the action that closes it. The principal trade-off is containment speed versus business continuity; neither side should be treated as automatically dominant.

Suggested review cadence

At implementation, capture the baseline and the intended authorization boundary. At the first operational checkpoint, verify access logs and unexpected denials. At the next change window, re-evaluate exceptions and re-run the negative test. These are review prompts rather than universal intervals or service-level guarantees.

Sources and confidence

Primary source: Google Threat Intelligence Group, Evolution of Adversarial AI. These sources support the attributed signal. All architecture controls, comparisons and proposed review timelines are independent analytical recommendations, not claims that the cited organization implemented them.

Practical architecture worksheet; it is not a claim of measured outcomes, an incident advisory or a substitute for local engineering validation.