feat(anonymizer): strict anti-leak mode (default on), prefilter decoupling, boundary-aware substitution

Harden the core privacy guarantee:

- Add `strict` mode (default true): after detection, verify no mapped value
  survives as a whole token in the output (ignoring placeholders, whose context
  may legitimately echo a value like `B+`). Catches a model that redacts one
  mention of a value but leaves another in clear — which the placeholder/mapping
  bijection check missed. Fail-closed: throws AnonymizationError naming only the
  non-secret placeholder key, and runs on the final result so it is not swallowed
  into the regex fallback (which can't fix a name leak). Set strict:false to opt out.
- Add `prefilter` option (default true): decouple the PII pre-filter from the
  presence of a regex fallback. Set false to always consult the LLM while keeping
  the fallback for LLM failures (max recall + graceful degradation).
- Boundary-aware value substitution: applyKnown and the leak check now match
  values only as whole tokens (Unicode letter/digit boundaries, regex-escaped),
  so "Ann" no longer replaces inside "Anna" and "jean@x.ch" no longer matches
  inside "jean@x.church"; accented/non-Latin names preserved.
- deanonymize restores longest placeholder keys first (prefix-overlap defense).

Restructures anonymize/anonymizeChunks/anonymizeTurn to a single exit so the
leak check runs once on the final result. Behavior is unchanged for callers that
were already leak-free.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Oussama Knouz
2026-07-02 18:00:06 +01:00
parent 9941a21dd6
commit 7fb76e89f6
5 changed files with 268 additions and 27 deletions

View File

@@ -148,14 +148,42 @@ raw values.
```ts
new Anonymizer({
llm?, // LlmProvider — omit for regex-only mode
patterns?, // PatternDef[] — opt-in regex fallback; omit to fail closed
nameHint?, // RegExp flagging likely names so the LLM is consulted (has a default)
logger?, // { warn(msg) } — receives fallback warnings; defaults to no-op
llm?, // LlmProvider — omit for regex-only mode
patterns?, // PatternDef[] — opt-in regex fallback; omit to fail closed
nameHint?, // RegExp flagging likely names so the LLM is consulted (has a default)
logger?, // { warn(msg) } — receives fallback warnings; defaults to no-op
strict?, // boolean — verify no real value survives in the output (default true)
prefilter?, // boolean — skip the LLM when no PII is heuristically detected (default true)
});
// At least one of `llm` or `patterns` must be provided, or the constructor throws.
```
### Strict anti-leak mode
Placeholder ⇄ mapping validation checks that every placeholder in the output has a mapping entry and
vice-versa. On its own it does **not** catch a model that redacts the _first_ mention of a value but leaves a
_second_ in clear. Strict mode (**on by default**) adds a final check that **no mapped value still appears**
in the anonymized text — as a whole token, and ignoring placeholders (a placeholder's context may legitimately
echo a value, e.g. `B+` in `[PER_1.SANG:B+]`):
```ts
const anonymizer = new Anonymizer({ llm, patterns: presets.swiss }); // strict: true by default
// If a real value survives in the output, throws AnonymizationError (naming only the
// placeholder key, never the value) instead of returning text that still leaks.
```
It is **fail-closed**: the check runs on the final result and is not swallowed into the regex fallback (which
can't re-detect a leaked name). Set `strict: false` to disable it — e.g. if a false positive (a value that
also occurs as a legitimate standalone token) rejects an otherwise-fine result, or to trade the guarantee for
throughput.
### Always consult the LLM (`prefilter: false`)
By default a cheap regex/name pre-filter skips the LLM round-trip for text that clearly has no PII — but only
when a `patterns` fallback exists to make "no PII" a sound verdict. That heuristic is Latin-script oriented
and can miss unusual or non-Latin names. Set `prefilter: false` to **always** consult the LLM while still
keeping the regex fallback for when the LLM fails — maximum recall with graceful degradation.
### Presets & custom patterns
```ts
@@ -238,7 +266,8 @@ trust boundary in identifiable form, while keeping the answer fully reversible o
**legend** describing them.
3. **Regex fallback** — optional, deterministic detection of structured identifiers; used if the LLM is
unavailable. Omit it to **fail closed** (raise `AnonymizationError` rather than risk a leak).
4. **Validation** — bidirectional check that every placeholder has a mapping entry and vice-versa.
4. **Validation** — bidirectional check that every placeholder has a mapping entry and vice-versa. With
`strict: true`, additionally verify that no real value survives as a whole token in the output (fail-closed).
5. **Streaming de-anonymization**`makeStreamDeanonymizer` restores real values token-by-token, buffering
any placeholder split across chunks so a partial `[PER_` is never emitted.