Redacting Secrets and PII from AI Prompts: A Practical Client-Side DLP Method

The Disclosure Happens at Submit Time

A developer debugging a failing integration pastes a stack trace into an AI assistant. The trace includes a request header. The header includes a bearer token. The token is valid for another 40 days.

Nothing malicious happened, no attacker was involved, and no configuration was wrong. The credential is nonetheless outside the trust boundary, and rotating it is now the only remedy. This class of leak is why Sensitive Information Disclosure is ranked LLM02 in the OWASP Top 10 for LLM Applications (2025 edition), alongside Prompt Injection at LLM01.

The important property of the failure is its timing: the exposure completes the moment the request is sent. No provider-side control, retention setting, or contract clause can retroactively un-send it. The only control that acts before the boundary is crossed is a client-side pass over the text.

What you get from this guide: a concrete inventory of what to strip, detection logic that catches the structured 80% reliably, a placeholder scheme that keeps prompts useful, and an honest statement of what automated redaction cannot do.

What Actually Needs to Leave

A useful heuristic: if you would not paste it into a public forum post, do not paste it into a prompt without redaction. The relevant question is not whether the provider is trustworthy; it is whether the data was ever necessary for the task. Most of the time a stack trace is just as debuggable with API_KEY_1 in place of the real token.

Detection Layer 1: Anchored Patterns

Structured secrets are detectable because vendors gave them fixed shapes — deliberately, to make scanning possible.

const PATTERNS = [
  // Credentials — prefixed vendor tokens
  { type: "OPENAI_KEY",   re: /\bsk-[A-Za-z0-9_-]{20,}\b/g },
  { type: "ANTHROPIC_KEY",re: /\bsk-ant-[A-Za-z0-9_-]{20,}\b/g },
  { type: "GITHUB_PAT",   re: /\b(gh[pousr]_[A-Za-z0-9]{36,})\b/g },
  { type: "AWS_ACCESS_KEY", re: /\b(?:AKIA|ASIA)[0-9A-Z]{16}\b/g },
  { type: "SLACK_TOKEN",  re: /\bxox[abposr]-[A-Za-z0-9-]{10,}\b/g },
  { type: "STRIPE_KEY",   re: /\b(?:sk|rk)_(?:live|test)_[A-Za-z0-9]{16,}\b/g },
  { type: "PRIVATE_KEY",  re: /-----BEGIN (?:RSA |EC |OPENSSH )?PRIVATE KEY-----[\s\S]*?-----END[^-]*-----/g },
  { type: "JWT",          re: /\beyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\b/g },
  { type: "CONN_STRING",  re: /\b(?:postgres|postgresql|mysql|mongodb(?:\+srv)?|redis|amqp):\/\/[^\s"'<>]+/gi },

  // Personal data
  { type: "EMAIL",        re: /\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b/g },
  { type: "IPV4",         re: /\b(?:\d{1,3}\.){3}\d{1,3}\b/g },
  { type: "CARD",         re: /\b(?:\d[ -]*?){13,19}\b/g },  // validate with Luhn before flagging
  { type: "UPI_VPA",      re: /\b[\w.-]{2,}@(?:okhdfcbank|okicici|oksbi|okaxis|paytm|ybl|ibl|axl|upi)\b/gi },
];

Two refinements that decide whether the output is usable:

Validate card candidates with Luhn. Any 13-19 digit run matches the pattern, including order IDs and timestamps. The checksum removes most false positives:

function luhnValid(digits) {
  const d = digits.replace(/\D/g, "");
  if (d.length < 13 || d.length > 19) return false;
  let sum = 0, alt = false;
  for (let i = d.length - 1; i >= 0; i--) {
    let n = +d[i];
    if (alt) { n *= 2; if (n > 9) n -= 9; }
    sum += n; alt = !alt;
  }
  return sum % 10 === 0;
}

Do not blanket-flag IPv4. 127.0.0.1, 0.0.0.0, and documentation ranges are noise. Flag private ranges (10/8, 172.16/12, 192.168/16) and public addresses; skip loopback and unspecified.

Detection Layer 2: Entropy

Many real secrets carry no vendor prefix — a rotated internal token, a generated password, a session identifier. What they share is randomness. Shannon entropy over the character distribution separates them from ordinary words:

function shannonEntropy(s) {
  const freq = {};
  for (const ch of s) freq[ch] = (freq[ch] ?? 0) + 1;
  return -Object.values(freq).reduce((acc, n) => {
    const p = n / s.length;
    return acc + p * Math.log2(p);
  }, 0);
}

// Flag long alphanumeric runs whose per-character entropy is high.
function suspectHighEntropy(text, { minLen = 20, threshold = 3.5 } = {}) {
  return [...text.matchAll(/\b[A-Za-z0-9+/=_-]{20,}\b/g)]
    .filter((m) => shannonEntropy(m[0]) >= threshold)
    .map((m) => ({ value: m[0], index: m.index }));
}

Calibration in practice: English words sit near 2.5-3.2 bits per character; random Base64 sits near 4.5-6. A threshold around 3.5 with a 20-character minimum catches most unprefixed secrets while tolerating long identifiers. It will still flag Git commit SHAs and Base64 image fragments — acceptable for a review step, unacceptable for silent automatic replacement.

Replacement: Typed, Stable Placeholders

The instinct is to overwrite with asterisks. That is the wrong output, because it destroys the two properties that make the prompt worth sending.

Original : Charge card 4111111111111111 for alice@example.com, retry key sk-abc123XYZ...
Bad      : Charge card **************** for *****************, retry key ***********
Better   : Charge card CARD_1 for EMAIL_1, retry key API_KEY_1

Typed placeholders preserve type information (the model still knows it is reasoning about a card, an email, a key) and referential identity (the same original value maps to the same placeholder everywhere, so relationships across a long log stay intact). Rules that make this work:

1. Deterministic mapping per session. alice@example.com is always EMAIL_1; a second address becomes EMAIL_2.
2. Keep the map local. It is the sensitive artefact — never include it in the prompt, and clear it when done.
3. Preserve shape when shape is the question. For a format-validation task, 4111-11XX-XXXX-1111 is more useful than CARD_1; partial masking is the right choice only when structure is the subject.
4. Substitute back locally. When the model replies with send the receipt to EMAIL_1, restore the real value on your machine.

What This Method Cannot Do

Stating the limits is what separates a control from a comfort blanket.

• Unstructured personal data survives. “The customer, a 34-year-old teacher in Pune who called about her cancelled order,” contains no pattern to match and is still identifying in combination.
• Semantics are invisible to regex. A proprietary pricing formula written out in prose is a trade secret with no distinguishing byte pattern.
• Aggregation re-identifies. Individually harmless fields — postcode, birth date, employer — combine into an identity.
• Screenshots and attachments bypass text scanning entirely. Images carry text, and photographs carry EXIF metadata including GPS coordinates.
• The model’s own output can leak. LLM02 covers disclosure by the system as well as to it: a retrieval-augmented assistant can surface another tenant’s data through the answer path even when every prompt was clean.

Treat automated redaction as the last line before transmission, sitting under two stronger controls: not collecting the data into the prompt in the first place, and reviewing what you are about to send.

Step-by-Step: Sanitising a Prompt with Toolbox

1. Open the tool: visit the Toolbox Prompt Redactor.
2. Paste the raw material — the stack trace, log excerpt, config file, or draft prompt, unedited.
3. Review every finding. Confirm the true positives and dismiss noise such as commit hashes; an automated pass is a proposal, not a verdict.
4. Read the remaining prose yourself for the class no scanner catches: named individuals, internal project names, described business logic.
5. Copy the redacted text into your model client, and keep the placeholder mapping on your machine only.
6. Rotate anything already exposed. If a live key was in the original text and that text was ever sent anywhere, redacting the copy does not help — revoke and reissue the credential.

Outcome: prompts that keep enough structure for the model to be useful while removing the material that would turn a debugging session into an incident report — and a clear separation between what automation caught and what still needs your eyes.

Related guides: LLM token counting and cost · EXIF metadata and GPS stripping · JWT claims inspection