Data Exfiltration Scenario Design

Intermediate LAB teaching safe data-exfiltration scenario design: sensitive-data boundaries, identity authority, disclosure controls, uncertainty, reviewer-safe evidence, and non-access boundaries.

StatusIntermediate

DomainAI Security

TrackAI Red Team Scenario Design

RuntimeRead-only course

Overview

This LAB teaches how to design safe data-exfiltration scenarios that evaluate whether an AI workflow could expose sensitive information. The scenario uses synthetic data classes and bounded evidence, not real customer records, credentials, secrets, or production outputs.

Sensitive data boundary Synthetic evidence Disclosure controls No data access

Concept Deep Dives

Expand each concept when studying data-exfiltration scenario design fundamentals.

What is data exfiltration scenario design?

Data exfiltration scenario design is the safe planning of tests that evaluate whether an AI workflow could expose sensitive information. A safe scenario uses synthetic data classes and bounded evidence, not real customer records or secrets.

Why must sensitive data be represented synthetically?

Synthetic data allows learners to evaluate disclosure controls without accessing, storing, transmitting, or publishing real secrets, credentials, customer data, regulated data, or production outputs.

How do data boundaries and authority boundaries differ?

A data boundary defines what information is sensitive. An authority boundary defines who or what is allowed to request, view, summarize, transform, or disclose it. A strong scenario identifies both boundaries before evaluating behavior.

What controls should a data exposure scenario test?

Controls include sensitivity classification, identity and tenant scope, source authorization, redaction, refusal behavior, output filtering, tool restrictions, audit logging, and human approval for high-risk disclosure.

How should impact and uncertainty be recorded?

A safe finding separates observed behavior from inferred risk. It records what synthetic class was used, what control was expected, what was observed, what was not tested, and what remediation is recommended.

How should data exfiltration findings be documented safely?

A safe finding records objective, scope, synthetic data class, expected disclosure control, observed behavior, uncertainty, risk, and remediation without including real sensitive material.

Visual Data Exfiltration Scenario Design Model

A strong data-exfiltration scenario turns sensitive-data exposure risk into a scoped, evidence-backed control review.

AI Workflow Under Review Assistant, summarizer, retrieval flow, reporting tool, or tool-enabled workflow

→

Synthetic Data Class Secret-like token, customer-like record, credential-like field, or sensitive policy content

→

Authority Boundary Who can request, view, summarize, transform, or disclose the information?

Exposure Hypothesis Unsafe disclosure, overbroad summary, missing redaction, or identity boundary failure

→

Expected Control Refusal, redaction, minimization, approval gate, source check, or fail-closed behavior

→

Reviewer-Safe Finding Observed behavior, evidence, uncertainty, risk, and remediation

Learning rule: Data-exfiltration testing is safe only when it uses synthetic data and evaluates disclosure controls without accessing real customer data, secrets, credentials, or production outputs.

Example Scenario

An AI assistant summarizes account-support context and may encounter sensitive-looking fields. The learner must design a safe scenario to check whether the assistant minimizes, redacts, or refuses disclosure when the requester lacks authority.

Objective Evaluate whether the AI workflow prevents unsafe disclosure of synthetic sensitive information.

Scope Synthetic secret-like and customer-like records only. No real customer data, credentials, tokens, secrets, or production outputs.

Expected Control The assistant should minimize, redact, refuse, or require approval before disclosing sensitive-looking content.

Evidence Reviewer-safe record of data class, requester authority, expected control, observed behavior, uncertainty, and remediation.

Safe scenario handling:
define the AI workflow under review
define synthetic sensitive data classes
identify requester authority
state disclosure boundaries
define expected controls
do not use real customer data or secrets
observe whether redaction or refusal is preserved
record uncertainty and limits
write remediation tied to disclosure controls

Result:
The scenario becomes a data-boundary control review, not a real exfiltration exercise.

High-Risk Anti-Pattern

A dangerous pattern is using real customer data, credentials, secrets, or production outputs to prove that a data exposure scenario is realistic or valid.

Unsafe pattern:

real customer data
→ live secrets or tokens
→ credential material
→ production system interaction
→ live data movement
→ unsupported breach claims

Risk:

customer data exposure
credential leakage
regulatory impact
policy violation
audit evidence contamination
production incident
loss of trust in the learning platform

Secure alternative:
Use synthetic data classes.
Bound the scenario.
Do not use real secrets.
Do not access customer records.
Use expected controls.
Capture reviewer-safe evidence.
Record uncertainty.
Recommend remediation safely.

Governance Boundary

This LAB is read-only and deterministic. It teaches safe scenario design only. It does not access customer data, handle credentials, use secrets, move data, connect to production systems, mutate runtime systems, or claim production enforcement.

Runtime = read-only learning

Backend exposure = false
Public backend exposed = false
Live data exfiltration = false
Customer data access = false
Credential handling = false
Secret handling = false
Real sensitive data usage = false
Live model abuse execution = false
Live exploit execution = false
Live red-team execution = false
Runtime mutation = false
Production enforcement claim = false