AI safety, privacy and data governance

Decide what data may leave your systems, handle prompt injection and unsafe output, and log enough to investigate an incident without creating one.

What you send is a decision

Data	Default handling
Public documentation	Freely usable as context
Internal documentation	Allowed only under a reviewed provider agreement
Personal data	Minimise, redact where possible, and confirm the retention terms
Regulated data (health, finance, children)	Do not send to a general hosted API without explicit approval
Credentials, keys, tokens	Never. Filter them out before the request is built
Third-party confidential content	Check the contract before it leaves your boundary

import re

PATTERNS = [
    (re.compile(r"\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b"), "[CARD]"),
    (re.compile(r"\b[\w.+-]+@[\w-]+\.[\w.]+\b"), "[EMAIL]"),
    (re.compile(r"(?i)\b(api[_-]?key|secret|token)\s*[:=]\s*\S+"), "[SECRET]"),
]

def redact(text: str) -> str:
    for pattern, replacement in PATTERNS:
        text = pattern.sub(replacement, text)
    return text

payload = redact(user_message)      # redact before the request leaves the process

Redaction is a mitigation, not a guarantee. Patterns miss formats and invent confidence; the real control is not sending the data at all.
Check where the provider stores data, for how long, and whether it is used for training. Record the answer; the terms change.
Keep a data-flow diagram that names every provider that receives user content.

Prompt injection and output handling

import html

def safe_render(model_output: str) -> str:
    # treat model output as untrusted for the same reason you treat user input as untrusted
    return html.escape(model_output, quote=True)

def run_tool(name, args, allowed):
    if name not in allowed:
        raise PermissionError(f"tool {name} is not allowed")
    if not validate_args(name, args):        # never trust model-supplied arguments
        raise ValueError("invalid arguments")
    return TOOLS[name](**args)

Threat	Control
Direct injection from a user	Instructions in the system message; treat user text as data
Indirect injection via retrieved content	Delimit retrieved text; never let it call tools
Output rendered as HTML	Escape it, or render as text only
Model choosing a destructive tool	Allowlist tools, require confirmation for anything irreversible
Data exfiltration in a link	Block outbound requests from generated URLs
Model leaking another user's data	Filter retrieval by permission before ranking

⚠️

Anything the model can do, an attacker who controls its input may be able to make it do. Treat the model as a component with no privileges of its own: your code holds the permissions, validates the arguments, and decides what actually happens.

What to log

log_entry = {
    "request_id": rid,
    "actor": hashed_user_id,          # pseudonymous, not an email address
    "model": model,
    "prompt_version": "classify_v3",
    "input_tokens": usage.prompt_tokens,
    "output_tokens": usage.completion_tokens,
    "latency_ms": elapsed_ms,
    "outcome": "ok",
    # the raw prompt and completion are NOT logged here
}

# if you must retain content for debugging, keep it separate, encrypted,
# access-controlled and time-limited, with a documented deletion date

Log the metadata that lets you investigate an incident and answer a customer question: which prompt version ran, which model, how many tokens, how long it took and whether it succeeded. Content is a separate decision with separate controls, because a log is a copy of the data.

FAQ

Can I use a hosted model with customer data?

Only after checking the provider's terms, your own obligations and your customers' expectations. Prefer pseudonymising identifiers, sending only the fields the task needs, and using a provider or deployment with a contractual no-training guarantee.

What should an incident response plan cover?

How to disable the feature quickly, how to identify affected requests from logs, who to notify, and how to rotate any credential that may have been exposed. Write it before you need it, and test that the kill switch works.

Retrieval-augmented generation Designing an AI feature end to end

Last refreshed 2026-09-18.

AI safety, privacy and data governance

What you send is a decision

Prompt injection and output handling

What to log

FAQ

Related