feat(anonymizer): strict anti-leak mode (default on), prefilter decoupling, boundary-aware substitution

Harden the core privacy guarantee: - Add `strict` mode (default true): after detection, verify no mapped value survives as a whole token in the output (ignoring placeholders, whose context may legitimately echo a value like `B+`). Catches a model that redacts one mention of a value but leaves another in clear — which the placeholder/mapping bijection check missed. Fail-closed: throws AnonymizationError naming only the non-secret placeholder key, and runs on the final result so it is not swallowed into the regex fallback (which can't fix a name leak). Set strict:false to opt out. - Add `prefilter` option (default true): decouple the PII pre-filter from the presence of a regex fallback. Set false to always consult the LLM while keeping the fallback for LLM failures (max recall + graceful degradation). - Boundary-aware value substitution: applyKnown and the leak check now match values only as whole tokens (Unicode letter/digit boundaries, regex-escaped), so "Ann" no longer replaces inside "Anna" and "jean@x.ch" no longer matches inside "jean@x.church"; accented/non-Latin names preserved. - deanonymize restores longest placeholder keys first (prefix-overlap defense). Restructures anonymize/anonymizeChunks/anonymizeTurn to a single exit so the leak check runs once on the final result. Behavior is unchanged for callers that were already leak-free. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 18:00:06 +01:00
parent 9941a21dd6
commit 7fb76e89f6
5 changed files with 268 additions and 27 deletions
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,6 +4,31 @@ All notable changes to this project are documented here. The format is based on
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to
 [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

+## [0.5.0] - Unreleased
+
+### Added
+
+- **Strict anti-leak mode (on by default).** New `strict` option on `Anonymizer`. When enabled, the anonymized
+  output is checked after detection to ensure no mapped value still appears as a whole token (ignoring
+  placeholders — a placeholder's context field may legitimately echo a value, e.g. `B+` in `[PER_1.SANG:B+]`).
+  This catches a model that redacts one mention of a value but leaves another in clear — a case the previous
+  bidirectional validation (placeholder ⇄ mapping-key) did not detect. On a suspected leak it throws
+  `AnonymizationError` naming only the non-secret placeholder key. Fail-closed: it runs on the final result
+  and is **not** swallowed into the regex fallback (which can't fix a name leak). **Defaults to `true`** — set
+  `strict: false` to restore the previous behaviour (or if a false positive rejects an otherwise-fine result).
+- **`prefilter` option** — decouples the cheap PII pre-filter (skip the LLM when no PII is heuristically
+  detected) from the presence of a regex fallback. Default `true`; set `false` to always consult the LLM
+  while still keeping the fallback for LLM failures (maximum recall with graceful degradation).
+
+### Changed
+
+- **Boundary-aware value substitution.** Known-value reuse (`applyKnown`, used by `anonymizeChunks` and
+  `anonymizeTurn`) now matches values only as whole tokens (Unicode letter/digit boundaries) instead of raw
+  substrings, so a short value like `"Ann"` is no longer replaced inside `"Anna"`, and `"jean@exemple.ch"`
+  no longer matches inside `"jean@exemple.church"`. Accented and non-Latin names are preserved. The strict
+  leak check uses the same boundary logic, so detection and substitution agree.
+- `deanonymize` now restores longest placeholder keys first (defensive against prefix overlaps).
+
 ## [0.4.0] - Unreleased

 ### Added
--- a/README.md
+++ b/README.md
@@ -152,10 +152,38 @@ new Anonymizer({
  patterns?,   // PatternDef[] — opt-in regex fallback; omit to fail closed
  nameHint?,   // RegExp flagging likely names so the LLM is consulted (has a default)
  logger?,     // { warn(msg) } — receives fallback warnings; defaults to no-op
+  strict?,     // boolean — verify no real value survives in the output (default true)
+  prefilter?,  // boolean — skip the LLM when no PII is heuristically detected (default true)
 });
 // At least one of `llm` or `patterns` must be provided, or the constructor throws.
 ```

+### Strict anti-leak mode
+
+Placeholder ⇄ mapping validation checks that every placeholder in the output has a mapping entry and
+vice-versa. On its own it does **not** catch a model that redacts the _first_ mention of a value but leaves a
+_second_ in clear. Strict mode (**on by default**) adds a final check that **no mapped value still appears**
+in the anonymized text — as a whole token, and ignoring placeholders (a placeholder's context may legitimately
+echo a value, e.g. `B+` in `[PER_1.SANG:B+]`):
+
+```ts
+const anonymizer = new Anonymizer({ llm, patterns: presets.swiss }); // strict: true by default
+// If a real value survives in the output, throws AnonymizationError (naming only the
+// placeholder key, never the value) instead of returning text that still leaks.
+```
+
+It is **fail-closed**: the check runs on the final result and is not swallowed into the regex fallback (which
+can't re-detect a leaked name). Set `strict: false` to disable it — e.g. if a false positive (a value that
+also occurs as a legitimate standalone token) rejects an otherwise-fine result, or to trade the guarantee for
+throughput.
+
+### Always consult the LLM (`prefilter: false`)
+
+By default a cheap regex/name pre-filter skips the LLM round-trip for text that clearly has no PII — but only
+when a `patterns` fallback exists to make "no PII" a sound verdict. That heuristic is Latin-script oriented
+and can miss unusual or non-Latin names. Set `prefilter: false` to **always** consult the LLM while still
+keeping the regex fallback for when the LLM fails — maximum recall with graceful degradation.
+
 ### Presets & custom patterns

 ```ts
@@ -238,7 +266,8 @@ trust boundary in identifiable form, while keeping the answer fully reversible o
   **legend** describing them.
 3. **Regex fallback** — optional, deterministic detection of structured identifiers; used if the LLM is
   unavailable. Omit it to **fail closed** (raise `AnonymizationError` rather than risk a leak).
-4. **Validation** — bidirectional check that every placeholder has a mapping entry and vice-versa.
+4. **Validation** — bidirectional check that every placeholder has a mapping entry and vice-versa. With
+   `strict: true`, additionally verify that no real value survives as a whole token in the output (fail-closed).
 5. **Streaming de-anonymization** — `makeStreamDeanonymizer` restores real values token-by-token, buffering
   any placeholder split across chunks so a partial `[PER_` is never emitted.

--- a/src/anonymizer.ts
+++ b/src/anonymizer.ts
@@ -54,6 +54,24 @@ function stateless(re: RegExp): RegExp {
  return re.global || re.sticky ? new RegExp(re.source, re.flags.replace(/[gy]/g, '')) : re;
 }

+/** Escape a literal string for safe interpolation into a `RegExp` source. */
+function escapeRegExp(s: string): string {
+  return s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
+}
+
+/**
+ * A global regex that matches `value` only as a whole token — not glued to
+ * additional letters/digits on either side (so a short value like `"Al"` never
+ * matches inside `"Alps"`). `\p{L}\p{N}` (with the `u` flag) keeps accented
+ * names and non-Latin scripts intact. The value is escaped: it may contain regex
+ * metacharacters (`.`, `+`, `@`, …). Shared by value→placeholder substitution
+ * ({@link Anonymizer.applyKnown}) and the strict leak check so the two agree on
+ * exactly which occurrences count.
+ */
+function boundedValue(value: string): RegExp {
+  return new RegExp(`(?<![\\p{L}\\p{N}])${escapeRegExp(value)}(?![\\p{L}\\p{N}])`, 'gu');
+}
+
 /**
 * Framework-agnostic anonymization engine.
 *
@@ -75,6 +93,8 @@ export class Anonymizer {
  private readonly nameHint: RegExp;
  private readonly logger: Logger;
  private readonly historyMaxTurns: number;
+  private readonly strict: boolean;
+  private readonly prefilter: boolean;

  constructor(config: AnonymizerConfig = {}) {
    this.llm = config.llm;
@@ -82,6 +102,8 @@ export class Anonymizer {
    this.nameHint = stateless(config.nameHint ?? DEFAULT_NAME_HINT);
    this.logger = config.logger ?? NOOP_LOGGER;
    this.historyMaxTurns = config.historyMaxTurns ?? 10;
+    this.strict = config.strict ?? true;
+    this.prefilter = config.prefilter ?? true;
    if (!this.llm && !this.fallback) {
      throw new AnonymizationError('Anonymizer requires an LlmProvider, regex patterns, or both');
    }
@@ -97,19 +119,19 @@ export class Anonymizer {

  async anonymize(text: string): Promise<AnonymizationResult> {
    // Pre-filter: most text has no PII → avoid the LLM round-trip. Only safe when
-    // a regex fallback exists to judge "no PII"; without one we always consult the
-    // LLM so structured PII (e.g. emails) can't slip through unanonymized.
-    if (this.fallback && !this.likelyHasPii(text)) return { anon: text, mapping: {}, legend: {} };
+    // a regex fallback exists to judge "no PII" (name-hint alone would miss e.g.
+    // emails); without one we always consult the LLM. Disable with prefilter:false
+    // to always consult the LLM even when a fallback is configured.
+    if (this.prefilter && this.fallback && !this.likelyHasPii(text)) {
+      return { anon: text, mapping: {}, legend: {} };
+    }

+    let result: AnonymizationResult | undefined;
    if (this.hasLlm()) {
      try {
-        const result = await this.llm!.anonymize(text);
-        this.validate(result); // throws if inconsistent → fall back
-        return {
-          anon: result.anon,
-          mapping: result.mapping,
-          legend: this.buildLegend(result.anon, result.legend),
-        };
+        const r = await this.llm!.anonymize(text);
+        this.validate(r); // throws if inconsistent → fall back
+        result = { anon: r.anon, mapping: r.mapping, legend: this.buildLegend(r.anon, r.legend) };
      } catch (err) {
        if (!this.fallback) {
          throw new AnonymizationError(`anonymize failed: ${(err as Error).message}`, { cause: err });
@@ -117,7 +139,11 @@ export class Anonymizer {
        this.logger.warn(`LLM unavailable/invalid → regex fallback: ${(err as Error).message}`);
      }
    }
-    return this.fallbackAnonymize(text);
+    if (!result) result = this.fallbackAnonymize(text);
+    // Strict anti-leak runs on the final result and throws (fail-closed) rather
+    // than being swallowed into the regex fallback, which can't fix a name leak.
+    this.assertNoLeak(result.anon, result.mapping);
+    return result;
  }

  /**
@@ -140,10 +166,13 @@ export class Anonymizer {
    if (!chunks.length) return { anon: [], mapping: { ...seed.mapping }, legend: { ...seedLegend } };

    const seeded = chunks.map((c) => this.applyKnown(c, seed.mapping));
-    if (this.fallback && !seeded.some((c) => this.likelyHasPii(c))) {
-      return { anon: seeded, mapping: { ...seed.mapping }, legend: { ...seedLegend } };
+    if (this.prefilter && this.fallback && !seeded.some((c) => this.likelyHasPii(c))) {
+      const out = { anon: seeded, mapping: { ...seed.mapping }, legend: { ...seedLegend } };
+      for (const a of out.anon) this.assertNoLeak(a, out.mapping);
+      return out;
    }

+    let out: { anon: string[]; mapping: Record<string, string>; legend: Record<string, string> } | undefined;
    if (this.hasLlm()) {
      try {
        const {
@@ -161,7 +190,7 @@ export class Anonymizer {
          }
        }
        const legend = this.buildLegend(anon.join('\n'), { ...seedLegend, ...gLegend });
-        return { anon, mapping, legend };
+        out = { anon, mapping, legend };
      } catch (err) {
        if (!this.fallback) {
          throw new AnonymizationError(`anonymizeChunks failed: ${(err as Error).message}`, { cause: err });
@@ -169,7 +198,9 @@ export class Anonymizer {
        this.logger.warn(`Chunk anonymization → regex fallback: ${(err as Error).message}`);
      }
    }
-    return this.fallbackChunks(seeded, seed.mapping, seedLegend);
+    if (!out) out = this.fallbackChunks(seeded, seed.mapping, seedLegend);
+    for (const a of out.anon) this.assertNoLeak(a, out.mapping);
+    return out;
  }

  /**
@@ -199,9 +230,10 @@ export class Anonymizer {
    const prev: AnonymizerSession = session ?? { mapping: {}, legend: {}, history: [] };
    const seeded = this.applyKnown(text, prev.mapping);

-    let turn: AnonymizationResult;
+    let merged: { anon: string; mapping: Record<string, string>; legend: Record<string, string> } | undefined;
    if (this.hasLlm()) {
      try {
+        let turn: AnonymizationResult;
        if (this.llm!.anonymizeInConversation) {
          turn = await this.llm!.anonymizeInConversation(seeded, {
            history: prev.history,
@@ -214,7 +246,7 @@ export class Anonymizer {
          if (b.segments.length !== 1) throw new Error('SEGMENT_COUNT_MISMATCH');
          turn = { anon: b.segments[0], mapping: b.mapping, legend: b.legend };
        }
-        return this.commitTurn(prev, this.mergeAndCheck(prev.mapping, turn));
+        merged = this.mergeAndCheck(prev.mapping, turn);
      } catch (err) {
        if (!this.fallback) {
          throw new AnonymizationError(`anonymizeTurn failed: ${(err as Error).message}`, { cause: err });
@@ -222,7 +254,9 @@ export class Anonymizer {
        this.logger.warn(`Conversation turn → regex fallback: ${(err as Error).message}`);
      }
    }
-    return this.commitTurn(prev, this.mergeAndCheck(prev.mapping, this.fallback!.anonymize(seeded)));
+    if (!merged) merged = this.mergeAndCheck(prev.mapping, this.fallback!.anonymize(seeded));
+    this.assertNoLeak(merged.anon, merged.mapping);
+    return this.commitTurn(prev, merged);
  }

  /** Merge a turn's result into the running mapping (de-collision) + anti-leak check. */
@@ -275,17 +309,43 @@ export class Anonymizer {
    };
  }

-  /** Replace known values (seed) by their placeholder, longest values first. */
+  /**
+   * Replace known values (seed) by their placeholder, longest values first (so a
+   * value that contains a shorter one is claimed first). Matching is boundary-aware
+   * ({@link boundedValue}): a value is only swapped when it stands as a whole token,
+   * so a short value like `"Al"` is never substituted inside `"Alps"`.
+   */
  private applyKnown(text: string, seed: Record<string, string>): string {
    let out = text;
    for (const [ph, val] of Object.entries(seed)
      .filter(([, v]) => v)
      .sort((a, b) => b[1].length - a[1].length)) {
-      out = out.split(val).join(ph);
+      out = out.replace(boundedValue(val), ph);
    }
    return out;
  }

+  /**
+   * Strict anti-leak guard: no mapped value may still appear as a whole token in
+   * the anonymized text, ignoring placeholders (a placeholder's context field can
+   * legitimately echo a value, e.g. the `B+` in `[PER_1.SANG:B+]`). Catches a model
+   * that emits the placeholder for one mention of a value but leaves another in
+   * clear. Fail-closed: throws {@link AnonymizationError} naming only the non-secret
+   * placeholder key (never the value). No-op unless `strict` is enabled.
+   */
+  private assertNoLeak(anon: string, mapping: Record<string, string>): void {
+    if (!this.strict) return;
+    const residual = anon.replace(PLACEHOLDER_RE, ''); // drop placeholders — their context may echo a value
+    for (const [ph, value] of Object.entries(mapping)) {
+      // Skip 1-char values: negligibly identifying and a magnet for false positives.
+      if (value.length >= 2 && boundedValue(value).test(residual)) {
+        throw new AnonymizationError(
+          `strict mode: the value behind ${ph} still appears in the anonymized output`,
+        );
+      }
+    }
+  }
+
  private applyRename(text: string, rename: Record<string, string>): string {
    let out = text;
    for (const [from, to] of Object.entries(rename)) out = out.split(from).join(to);
@@ -406,11 +466,13 @@ export class Anonymizer {
    }
  }

-  /** Restore the real values (replace every known placeholder). */
+  /** Restore the real values (replace every known placeholder), longest key first. */
  deanonymize(text: string, mapping: Record<string, string>): string {
    if (!text) return text;
    let out = text;
-    for (const [ph, value] of Object.entries(mapping)) {
+    // Longest placeholder first so a shorter key can't pre-empt a longer one that
+    // shares its prefix (defensive; the bracketed format already prevents most overlaps).
+    for (const [ph, value] of Object.entries(mapping).sort((a, b) => b[0].length - a[0].length)) {
      out = out.split(ph).join(value);
    }
    return out;
--- a/src/types.ts
+++ b/src/types.ts
@@ -113,8 +113,32 @@ export interface AnonymizerConfig {
   * Heuristic that flags likely proper names so the LLM is consulted. Has a
   * sensible default. Any `g`/`y` flags are stripped internally so the regex is
   * used statelessly — a global flag would otherwise alternate results.
+   *
+   * The default is Latin-script oriented (`A–Z` + Latin-1 accents); pass your own
+   * for other scripts, or set `prefilter: false` to always consult the LLM.
   */
  nameHint?: RegExp;
+  /**
+   * Strict anti-leak mode. When `true`, the anonymized text is checked after
+   * detection to ensure no mapped value still appears — as a whole token, and
+   * outside placeholders (a placeholder's context field may legitimately echo a
+   * value, e.g. `[PER_1.SANG:B+]`). This catches a model that redacts one mention
+   * of a value but leaves another in clear. On detection it throws
+   * {@link AnonymizationError} (fail-closed; the error names only the non-secret
+   * placeholder key, never the value). **Default `true`** (fail-safe). Set `false`
+   * to trade the guarantee for throughput, or if a false positive — a value that
+   * also occurs as a legitimate standalone token — rejects an otherwise-fine result.
+   */
+  strict?: boolean;
+  /**
+   * Whether to run the cheap PII pre-filter that skips the LLM round-trip when no
+   * personal data is heuristically detected. **Default `true`.** It only takes
+   * effect when a `patterns` fallback is configured — the structured detector is
+   * what makes a "no PII" verdict sound (name-hint alone would miss e.g. emails).
+   * Set `false` to ALWAYS consult the LLM while still keeping the regex fallback
+   * for when the LLM fails: maximum recall with graceful degradation.
+   */
+  prefilter?: boolean;
  /** Where fallback/diagnostic warnings go. Defaults to a no-op. */
  logger?: Logger;
  /**
--- a/test/anonymizer.test.ts
+++ b/test/anonymizer.test.ts
@@ -186,6 +186,107 @@ describe('Anonymizer (Swiss preset)', () => {
  });
 });

+describe('strict anti-leak mode', () => {
+  // A model that redacts the FIRST mention of a value but leaves a second in clear.
+  const leakyLlm = () =>
+    failingLlm({
+      anonymize: vi.fn().mockResolvedValue({
+        anon: 'Dossier de [PER_1.NOM:M] ; contactez Alain Jaccard directement.',
+        mapping: { '[PER_1.NOM:M]': 'Alain Jaccard' },
+        legend: {},
+      }),
+    });
+
+  it('throws AnonymizationError when a mapped value still appears in clear', async () => {
+    const s = new Anonymizer({ llm: leakyLlm(), strict: true });
+    await expect(s.anonymize('Dossier de Alain Jaccard')).rejects.toBeInstanceOf(AnonymizationError);
+  });
+
+  it('the leak error names the placeholder key, never the secret value', async () => {
+    const s = new Anonymizer({ llm: leakyLlm(), strict: true });
+    await expect(s.anonymize('Dossier de Alain Jaccard')).rejects.toMatchObject({
+      message: expect.stringContaining('[PER_1.NOM:M]'),
+    });
+    await expect(s.anonymize('Dossier de Alain Jaccard')).rejects.not.toMatchObject({
+      message: expect.stringContaining('Alain Jaccard'),
+    });
+  });
+
+  it('fails closed even with a fallback configured (does not silently degrade to regex)', async () => {
+    const s = new Anonymizer({ llm: leakyLlm(), patterns: presets.swiss, strict: true });
+    await expect(s.anonymize('Dossier de Alain Jaccard')).rejects.toBeInstanceOf(AnonymizationError);
+  });
+
+  it('with strict:false the leaky result is returned as-is', async () => {
+    const s = new Anonymizer({ llm: leakyLlm(), strict: false });
+    const r = await s.anonymize('Dossier de Alain Jaccard');
+    expect(r.anon).toContain('Alain Jaccard'); // leak check disabled → passes through
+  });
+
+  it('is enabled by default (no strict option → still throws on a leak)', async () => {
+    const s = new Anonymizer({ llm: leakyLlm() });
+    await expect(s.anonymize('Dossier de Alain Jaccard')).rejects.toBeInstanceOf(AnonymizationError);
+  });
+
+  it('does NOT flag a value that only appears inside a placeholder context (e.g. B+)', async () => {
+    const anonymize = vi.fn().mockResolvedValue({
+      anon: 'Groupe sanguin [PER_1.SANG:B+] confirmé.',
+      mapping: { '[PER_1.SANG:B+]': 'B+' },
+      legend: {},
+    });
+    const s = new Anonymizer({ llm: failingLlm({ anonymize }), strict: true });
+    const r = await s.anonymize('Groupe sanguin B+ confirmé.');
+    expect(r.anon).toBe('Groupe sanguin [PER_1.SANG:B+] confirmé.');
+  });
+
+  it('does NOT flag a value that is only a substring of a larger token', async () => {
+    // "Ann" is redacted; "Anna" (a different token) is left in clear → not a leak.
+    const anonymize = vi.fn().mockResolvedValue({
+      anon: '[PER_1.PRENOM:F] connaît Anna.',
+      mapping: { '[PER_1.PRENOM:F]': 'Ann' },
+      legend: {},
+    });
+    const s = new Anonymizer({ llm: failingLlm({ anonymize }), strict: true });
+    const r = await s.anonymize('Ann connaît Anna.');
+    expect(r.anon).toBe('[PER_1.PRENOM:F] connaît Anna.');
+  });
+
+  it('regex-only output never trips the check (every occurrence is replaced)', async () => {
+    const s = new Anonymizer({ patterns: presets.swiss, strict: true });
+    const r = await s.anonymize('Écris à jean@exemple.ch puis à jean@exemple.ch');
+    expect(r.anon).not.toContain('jean@exemple.ch');
+  });
+});
+
+describe('prefilter decoupling', () => {
+  it('prefilter:false consults the LLM even for PII-free text when a fallback exists', async () => {
+    const anonymize = vi.fn().mockResolvedValue({ anon: 'INNER JOIN', mapping: {}, legend: {} });
+    const s = new Anonymizer({ llm: failingLlm({ anonymize }), patterns: presets.swiss, prefilter: false });
+    await s.anonymize('Explique INNER JOIN'); // no structured PII, no name hint
+    expect(anonymize).toHaveBeenCalledTimes(1);
+  });
+
+  it('prefilter default (true) still skips the LLM for PII-free text', async () => {
+    const anonymize = vi.fn();
+    const s = new Anonymizer({ llm: failingLlm({ anonymize }), patterns: presets.swiss });
+    await s.anonymize('Explique INNER JOIN');
+    expect(anonymize).not.toHaveBeenCalled();
+  });
+});
+
+describe('boundary-aware applyKnown', () => {
+  it('reuses a known value only as a whole token, not inside a larger word', async () => {
+    const s = new Anonymizer({ llm: failingLlm(), patterns: presets.swiss });
+    const seed = { '[PER_1.PRENOM:F]': 'Ann' };
+    // "Anna" must stay intact; the old substring replace would mangle it.
+    const kept = await s.anonymizeChunks(['Anna arrive demain.'], { mapping: seed });
+    expect(kept.anon[0]).toBe('Anna arrive demain.');
+    // but a standalone "Ann" is reused deterministically (no LLM call).
+    const reused = await s.anonymizeChunks(['Ann arrive demain.'], { mapping: seed });
+    expect(reused.anon[0]).toBe('[PER_1.PRENOM:F] arrive demain.');
+  });
+});
+
 describe('multi-turn conversation (anonymizeTurn)', () => {
  it('keeps stable ids across turns and accumulates the session', async () => {
    const anonymizeInConversation = vi