Oussama Knouz 948f95f4bc Merge feat/strict-anti-leak into main
Strict anti-leak mode (default on), prefilter decoupling, boundary-aware substitution.
2026-07-13 12:05:57 +01:00

@mobiletic/anonymizer

CI npm version License: MIT

Framework-agnostic PII anonymization & pseudonymization for TypeScript/JavaScript.

It replaces personal data in text with stable placeholders before the text leaves your trust boundary (e.g. before sending it to a third-party LLM, log sink, or analytics pipeline), and restores the real values afterwards — including across streamed tokens.

  • 🔌 Pluggable LLM detection — catch free-form PII (names, addresses) via any OpenAI-compatible endpoint, or your own provider.
  • 🧩 Optional regex fallback — structured identifiers (email, phone, IBAN, …) via configurable presets (swiss, generic) or your own. Opt in for graceful degradation, or omit it to fail closed.
  • 🏷️ Self-describing tokens — every result ships a legend (PERPersonne, MMasculin) you can hand to the downstream LLM so it understands the placeholders; the model may coin new abbreviations too.
  • 🔁 Deterministic coreference — the same person keeps the same id ([PER_1]) across a question and every retrieved chunk, with automatic de-collision.
  • 🌊 Streaming-safe — a placeholder split across two stream chunks ([PER_ + 1.NOM:M]) is never leaked partially.
  • 🪶 Zero runtime dependencies, ESM + CJS, fully typed.

Built by Mobiletic.

Install

npm install @mobiletic/anonymizer

Requires Node ≥ 18 (uses native fetch).

Quick start

Regex-only (no LLM, fully deterministic)

import { Anonymizer, presets } from '@mobiletic/anonymizer';

const anonymizer = new Anonymizer({ patterns: presets.swiss });

const { anon, mapping, legend } = await anonymizer.anonymize('Écris à jean@exemple.ch');
// anon    -> "Écris à [EMAIL_1]"
// mapping -> { "[EMAIL_1]": "jean@exemple.ch" }   (secret — keep on your side)
// legend  -> { "EMAIL": "Adresse e-mail" }        (safe to share downstream)

anonymizer.deanonymize(anon, mapping); // -> "Écris à jean@exemple.ch"

With an LLM (also catches names, addresses…)

import { Anonymizer, openAICompatibleProvider, presets } from '@mobiletic/anonymizer';

const anonymizer = new Anonymizer({
  llm: openAICompatibleProvider({
    baseUrl: process.env.LLM_BASE_URL!, // OpenAI, Infomaniak, vLLM, Ollama, …
    apiKey: process.env.LLM_API_KEY!,
    model: process.env.LLM_MODEL!,
    timeoutMs: 3000,
  }),
  patterns: presets.swiss, // OPTIONAL regex fallback if the LLM is down/misbehaves
});

const { anon, mapping, legend } = await anonymizer.anonymize('Le dossier de Alain Jaccard est complet.');
// anon   -> "Le dossier de [PER_1.NOM:M] est complet."
// legend -> { "PER": "Personne", "NOM": "Nom de famille", "M": "Masculin" }

Fallback is opt-in (fail-closed). If a patterns fallback is configured, a failed/timed-out/invalid LLM call degrades to the regex engine. If you omit patterns, there's nothing to degrade to, so the call throws an AnonymizationError (with the underlying error as .cause) — it never silently returns un-anonymized text. With no fallback the pre-filter is also bypassed, so every non-trivial call consults the LLM (more calls, no leaks). At least one of llm or patterns is required.

openAICompatibleProvider retries transient failures (network error, timeout, HTTP 429/5xx) before giving up; 4xx and malformed responses are not retried. Tune with timeoutMs (per attempt, default 3000), retries (default 1 → 2 attempts), and retryDelayMs (linear backoff, default 250). Worst-case latency is (retries + 1) × timeoutMs, so keep retries low on latency-sensitive paths.

It works with Infomaniak and other open-model endpoints out of the box: response_format is omitted by default (Infomaniak rejects the legacy { type: 'json_object' }), and responses are parsed leniently (a fenced JSON code block or surrounding prose is tolerated). For endpoints that support it, opt in with responseFormat — e.g. { type: 'json_object' } or a json_schema object.

Streaming de-anonymization

When you stream an LLM answer back to a user, restore real values without ever emitting a half-written placeholder:

const stream = anonymizer.makeStreamDeanonymizer(mapping);
for await (const token of llmTokens) process.stdout.write(stream.push(token));
process.stdout.write(stream.flush());

Anonymizing retrieved chunks consistently (RAG)

anonymizeChunks(chunks, seed) reuses the question's { mapping, legend } so the same person gets the same id across the question and every chunk, batches the LLM call, and de-collides genuinely new entities:

const q = await anonymizer.anonymize(question);
const { anon, mapping, legend } = await anonymizer.anonymizeChunks(retrievedChunks, q);
// `mapping`/`legend` are the full question  chunks tables;
// pass `mapping` to deanonymize()/makeStreamDeanonymizer(), and `legend` to the downstream LLM.

Multi-turn conversations

In a chat, anonymizing each message independently would renumber entities every turn (Adil could become PER_1 in turn 2 while Oussama was PER_1 in turn 1). anonymizeTurn threads a serializable AnonymizerSession so one entity keeps one id across the whole conversation:

let session; // persist this per conversation (Redis/DB); pass it back each turn
for (const message of userTurns) {
  const turn = await anonymizer.anonymizeTurn(message, session);
  session = turn.session; // { mapping, legend, history } — carries coreference forward
  send(turn.anon, turn.legend); // → the downstream chatbot (placeholders only)
}
// restore the bot's placeholder-bearing reply for the user:
anonymizer.deanonymize(botReply, session.mapping);

Or the stateful wrapper for in-memory use:

const conv = anonymizer.conversation();
await conv.anonymize('Bonjour, je suis Oussama'); // → Oussama = [PER_1…]
await conv.anonymize('Mon amie Adil …'); // → Oussama stays [PER_1…], Adil = [PER_2…]
conv.deanonymize(botReply);

Under the hood: known values are re-substituted locally (applyKnown), the provider is told which ids are taken, and new entities are de-collided into the session — so numbering never drifts.

Trust boundary & the mapping. The real boundary is "keep PII out of the downstream chatbot" — it only ever receives placeholders + legend. Your anonymizer endpoint already sees the cleartext it's asked to anonymize, so if (and only if) it's a trusted processor you may give it richer context — set includeMappingInContext: true on openAICompatibleProvider to also send the mapping for maximum cross-turn accuracy. It's off by default; keep it off for third-party endpoints you don't trust with raw values.

Configuration

new Anonymizer({
  llm?,        // LlmProvider — omit for regex-only mode
  patterns?,   // PatternDef[] — opt-in regex fallback; omit to fail closed
  nameHint?,   // RegExp flagging likely names so the LLM is consulted (has a default)
  logger?,     // { warn(msg) } — receives fallback warnings; defaults to no-op
  strict?,     // boolean — verify no real value survives in the output (default true)
  prefilter?,  // boolean — skip the LLM when no PII is heuristically detected (default true)
});
// At least one of `llm` or `patterns` must be provided, or the constructor throws.

Strict anti-leak mode

Placeholder ⇄ mapping validation checks that every placeholder in the output has a mapping entry and vice-versa. On its own it does not catch a model that redacts the first mention of a value but leaves a second in clear. Strict mode (on by default) adds a final check that no mapped value still appears in the anonymized text — as a whole token, and ignoring placeholders (a placeholder's context may legitimately echo a value, e.g. B+ in [PER_1.SANG:B+]):

const anonymizer = new Anonymizer({ llm, patterns: presets.swiss }); // strict: true by default
// If a real value survives in the output, throws AnonymizationError (naming only the
// placeholder key, never the value) instead of returning text that still leaks.

It is fail-closed: the check runs on the final result and is not swallowed into the regex fallback (which can't re-detect a leaked name). Set strict: false to disable it — e.g. if a false positive (a value that also occurs as a legitimate standalone token) rejects an otherwise-fine result, or to trade the guarantee for throughput.

Always consult the LLM (prefilter: false)

By default a cheap regex/name pre-filter skips the LLM round-trip for text that clearly has no PII — but only when a patterns fallback exists to make "no PII" a sound verdict. That heuristic is Latin-script oriented and can miss unusual or non-Latin names. Set prefilter: false to always consult the LLM while still keeping the regex fallback for when the LLM fails — maximum recall with graceful degradation.

Presets & custom patterns

import { presets } from '@mobiletic/anonymizer';

presets.swiss; // AVS, IBAN CH, EMAIL, Swiss phone, DATE
presets.generic; // EMAIL, IBAN, credit card, IPv4, phone, DATE

// Compose / extend:
const patterns = [
  ...presets.generic,
  { tag: 'TICKET', re: /\bJIRA-\d+\b/g }, // patterns must use the global flag
];

Each pattern may carry an optional validate(match) => boolean second stage — a match is only redacted if it passes. presets.generic uses it for a Luhn check so arbitrary long digit runs aren't mistaken for credit cards:

{ tag: 'CREDIT_CARD', re: /\b\d(?:[ -]?\d){12,18}\b/g, validate: luhnValid }

The generic preset is a best-effort starting point — broad patterns (phone, date, card) can overlap. For production use, prefer a locale-specific preset (presets.swiss) or your own patterns.

Custom LLM provider

Implement LlmProvider to use any backend (Anthropic, a local model, a rules engine…):

import type { LlmProvider } from '@mobiletic/anonymizer';

const myProvider: LlmProvider = {
  isConfigured: () => true,
  async anonymize(text) {
    /* return { anon, mapping, legend } */
  },
  async anonymizeBatch(texts, usedIds) {
    /* return { segments, mapping, legend } */
  },
};

Placeholder format

[PER_1.NOM:M]   entity PER #1, attribute NOM, context M (rich, from the LLM)
[EMAIL_1]       structured id from the regex fallback

PLACEHOLDER_RE is exported if you need to scan text for placeholders. The LLM may also coin new abbreviations (always uppercase [A-Z_]) for entities/attributes/context it discovers — every one it uses is described in the result legend.

How it works

The library does query-time pseudonymization: it rewrites text so that personal data never leaves your trust boundary in identifiable form, while keeping the answer fully reversible on your side.

                        ┌───────────────────────────── your trust boundary ──────────────────────────────┐
  raw text  ─▶  ① pre-filter ─▶ ② LLM detect ─▶ ③ regex fallback ─▶ ④ validate ─▶  anon + mapping + legend
                  (skip if            (names,         (optional;          (every                │       │
                   clearly            addresses,      structured ids;     placeholder    ┌───────┘       │
                   PII-free)          coins abbrevs,  fail closed if      is mapped)      ▼               ▼
                                      builds legend)  absent)                          mapping         legend
                                                                                     (SECRET,        (shareable:
   downstream LLM  ◀── anon + legend ─────────────────────────────────────────────  reversible)     PER=Personne…)
   answer (with [PER_1.NOM:M] tokens)
        │
        ▼
  ⑤ stream de-anonymize ──▶ real values restored for the end user (placeholders never leak, even if split)
  1. Pre-filter — a cheap regex/name check skips the LLM round-trip for text that clearly has no PII. (Bypassed when no regex fallback is configured, so nothing slips through.)
  2. LLM detection — finds free-form PII a regex can't (names, addresses), keeps coreference (the same person is always [PER_1]), coins uppercase abbreviations for anything new, and returns a legend describing them.
  3. Regex fallback — optional, deterministic detection of structured identifiers; used if the LLM is unavailable. Omit it to fail closed (raise AnonymizationError rather than risk a leak).
  4. Validation — bidirectional check that every placeholder has a mapping entry and vice-versa. With strict: true, additionally verify that no real value survives as a whole token in the output (fail-closed).
  5. Streaming de-anonymizationmakeStreamDeanonymizer restores real values token-by-token, buffering any placeholder split across chunks so a partial [PER_ is never emitted.

Three distinct outputs, with different sensitivities:

Output Example Sensitivity
anon Le dossier de [PER_1.NOM:M] Safe to send onward — no identifiers
mapping { "[PER_1.NOM:M]": "Alain Jaccard" } Secret — re-identifies people; keep it on your side
legend { "PER": "Personne", "M": "Masculin" } Safe to share — explains the tokens to a downstream LLM

Examples

Two examples produced live by Gemma 4, served through Infomaniak's Swiss AI API (the endpoint used for all results here), on fictional Swiss e-learning data. mapping is the secret re-identification key (kept by the operator, never sent downstream); legend is safe to share. The model even coins its own abbreviations for attributes not in the base prompt (passport, permit, blood type, …).

See all examples   5 single messages of rising complexity + 5 multi-turn conversations, with full mappings & legends.

A dense single message — 5 people + rare attributes

User message:

Note de dossier complète : le patient mineur Noah Baumann (né le 12.06.2013, groupe sanguin B+, numéro d'assuré maladie 756.2211.9988.77, allergique aux arachides) … Ses parents, Delphine Rieder (mère, permis de séjour C, 021 555 12 34) et Marco Baumann (père, passeport italien YA9087654), cosignent … Le Dr Farah Haddad (n° RCC V123456) a ouvert le dossier médical DM-2025-0417 … l'agent Kevin Zbinden de la caisse Helvetia … Noah utilise l'identifiant biométrique BIO-7729.

Anonymized — what the model sees (5 distinct people PER_1…PER_5 + ORG_1; Noah reused at the end):

Note de dossier complète : le patient mineur [PER_1.PRENOM:U] [PER_1.NOM:U] (né le [PER_1.DATE_NAISSANCE:2013], groupe sanguin B+, numéro d'assuré maladie [PER_1.ASSUR_ID:U], allergique aux arachides) … Ses parents, [PER_2.PRENOM:F] [PER_2.NOM:F] (mère, permis de séjour [PER_2.PERMIS:C], [PER_2.TELEPHONE:U]) et [PER_3.PRENOM:M] [PER_3.NOM:M] (père, passeport italien [PER_3.PASSPORT:U]), cosignent … Le Dr [PER_4.PRENOM:F] [PER_4.NOM:F] (n° RCC [PER_4.RCC:U]) a ouvert le dossier médical DM-2025-0417 … l'agent [PER_5.PRENOM:M] [PER_5.NOM:M] de la caisse [ORG_1.NOM:Assurance] … [PER_1.PRENOM:U] utilise l'identifiant biométrique [PER_1.BIO_ID:U].

mapping 🔒 (secret — kept on the operator's side, never sent downstream):

Placeholder Real value
[PER_1.PRENOM:U] / [PER_1.NOM:U] Noah / Baumann
[PER_1.DATE_NAISSANCE:2013] 12.06.2013
[PER_1.ASSUR_ID:U] 756.2211.9988.77
[PER_1.BIO_ID:U] BIO-7729
[PER_2.PRENOM:F] / [PER_2.NOM:F] Delphine / Rieder
[PER_2.PERMIS:C] / [PER_2.TELEPHONE:U] C / 021 555 12 34
[PER_3.PRENOM:M] / [PER_3.NOM:M] Marco / Baumann
[PER_3.PASSPORT:U] YA9087654
[PER_4.PRENOM:F] / [PER_4.NOM:F] Farah / Haddad
[PER_4.RCC:U] V123456
[PER_5.PRENOM:M] / [PER_5.NOM:M] Kevin / Zbinden
[ORG_1.NOM:Assurance] Helvetia

legend 🏷️ (shareable with the downstream model; 🆕 = coined by the model, not in the base prompt):

Abbreviation Meaning
PER Personne
PRENOM / NOM Prénom / Nom de famille
M / F / U Masculin / Féminin / n.d.
DATE_NAISSANCE Date de naissance
TELEPHONE Numéro de téléphone
ORG Organisation
ASSUR_ID Numéro d'assuré 🆕
PERMIS / C Permis de séjour / type C 🆕
PASSPORT Numéro de passeport 🆕
RCC Numéro RCC (médecin) 🆕
BIO_ID Identifiant biométrique 🆕

A multi-turn conversation — 4 people; blood group recalled turn 1 → turn 6

Turn User message Anonymized (model sees)
1 …mon fils mineur, Noah Baumann, né le 12.06.2013, groupe sanguin B+. …mon fils mineur, [PER_1.PRENOM:U] [PER_1.NOM:U], né le [PER_1.DATE_NAISSANCE:2013], groupe sanguin [PER_1.SANG:B+].
3 Moi, sa mère Delphine Rieder … permis de séjour de type C. Moi, sa mère [PER_2.PRENOM:F] [PER_2.NOM:F] … permis de séjour de type [PER_2.PERMIS:C].
4 Son père, Marco Baumann, passeport italien YA9087654, … Son père, [PER_3.PRENOM:M] [PER_1.NOM:U], passeport italien [PER_3.PASSPORT:ITA], …
5 Le Dr Farah Haddad a ouvert le dossier médical DM-2025-0417 pour Noah. Le [PER_4.TITRE:MED] [PER_4.PRENOM:F] [PER_4.NOM:F] a ouvert le dossier médical [PER_1.DOSSIER:MED] pour [PER_1.PRENOM:U].
6 Rappel : le groupe sanguin B+ de Noah doit figurer sur son badge… Rappel : le groupe sanguin [PER_1.SANG:B+] de [PER_1.PRENOM:U] doit figurer…

Noah stays PER_1 across all six turns, and his blood group [PER_1.SANG:B+] from turn 1 is reused in turn 6 — long-range coreference holds. (The father shares the surname, so [PER_1.NOM:U] is reused for it — the same value maps to the same token.)

mapping 🔒 (cumulative, secret):

Placeholder Real value
[PER_1.PRENOM:U] / [PER_1.NOM:U] Noah / Baumann
[PER_1.DATE_NAISSANCE:2013] 12.06.2013
[PER_1.SANG:B+] B+
[PER_1.AVS:SUI] 756.2211.9988.77
[PER_1.DOSSIER:MED] DM-2025-0417
[PER_2.PRENOM:F] / [PER_2.NOM:F] Delphine / Rieder
[PER_2.PERMIS:C] C
[PER_3.PRENOM:M] Marco
[PER_3.PASSPORT:ITA] YA9087654
[PER_4.TITRE:MED] Dr
[PER_4.PRENOM:F] / [PER_4.NOM:F] Farah / Haddad

legend 🏷️ (cumulative across the conversation; 🆕 = coined by the model):

Abbreviation Meaning
PER Personne
PRENOM / NOM Prénom / Nom de famille
M / F / U Masculin / Féminin / inconnu
DATE_NAISSANCE Date de naissance
AVS Numéro d'assuré
SANG Groupe sanguin 🆕
PERMIS / C Permis de séjour / établissem. 🆕
PASSPORT Numéro de passeport 🆕
SUI / ITA Suisse / Italie 🆕
TITRE / MED Titre professionnel / médical 🆕
DOSSIER Numéro de dossier 🆕

→ See all 10 examples, with full mapping and legend tables

How this helps with the nLPD

Switzerland's nLPD (and the GDPR) push for data minimisation and favour pseudonymisation when personal data is processed by third parties. This library is built around those principles:

  • The third party never sees raw PII. When you send text to an external LLM (or any external service), it receives only pseudonymised tokens like [PER_1.NOM:M] plus the non-identifying legend — never the real name, e-mail, AVS number, etc.
  • Pseudonymisation, not loss of meaning. The re-identification key (mapping) stays in your infrastructure; only you can reverse the tokens. The legend lets the downstream model still reason correctly ("a person", "male") without knowing who.
  • Fail-closed option. Omitting the regex fallback means that if detection can't run, the call errors instead of forwarding data that wasn't pseudonymised — no silent leak.
  • Coreference & minimisation. Re-using one id per entity avoids spreading extra distinguishing detail across a prompt, and the corpus itself can stay in clear text — pseudonymisation happens only at the boundary, at query time.

This is an engineering aid, not legal advice or a certification. You remain the data controller; assess it against your own obligations (see the disclaimer below).

Compliance note

This library is a best-effort pseudonymization aid, not a guarantee of legal compliance. LLM and regex detection can miss or mis-classify data. Validate against your own requirements (nLPD, GDPR, HIPAA, …) before relying on it for regulated data.

License

MIT © Mobiletic

Description
No description provided
Readme Apache-2.0 194 KiB
Languages
TypeScript 99.5%
JavaScript 0.5%