feat: initial release of @mobiletic/anonymizer

Framework-agnostic PII anonymization extracted from Mobiletic's chatbot.
Pluggable LLM detection + configurable regex fallback, deterministic
coreference, and streaming-safe de-anonymization.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Mobiletic
2026-06-20 21:58:38 +01:00
commit f09b917f1a
20 changed files with 3889 additions and 0 deletions

146
README.md Normal file
View File

@@ -0,0 +1,146 @@
# @mobiletic/anonymizer
Framework-agnostic **PII anonymization & pseudonymization** for TypeScript/JavaScript.
It replaces personal data in text with stable placeholders before the text leaves your trust boundary
(e.g. before sending it to a third-party LLM, log sink, or analytics pipeline), and restores the real
values afterwards — including across **streamed** tokens.
- 🔌 **Pluggable LLM detection** — catch free-form PII (names, addresses) via any OpenAI-compatible
endpoint, or your own provider. No LLM? It degrades gracefully to regex.
- 🧩 **Deterministic regex fallback** — structured identifiers (email, phone, IBAN, …) via configurable
pattern presets (`swiss`, `generic`) or your own.
- 🔁 **Deterministic coreference** — the same person keeps the same id (`[PER_1]`) across a question and
every retrieved chunk, with automatic de-collision.
- 🌊 **Streaming-safe** — a placeholder split across two stream chunks (`[PER_` + `1.NOM:M]`) is never
leaked partially.
- 🪶 **Zero runtime dependencies**, ESM + CJS, fully typed.
> Built by [Mobiletic](https://mobiletic.com) and extracted from a production Swiss-nLPD chatbot.
## Install
```bash
npm install @mobiletic/anonymizer
```
Requires Node ≥ 18 (uses native `fetch`).
## Quick start
### Regex-only (no LLM, fully deterministic)
```ts
import { Anonymizer, presets } from '@mobiletic/anonymizer';
const anonymizer = new Anonymizer({ patterns: presets.swiss });
const { anon, mapping } = await anonymizer.anonymize('Écris à jean@exemple.ch');
// anon -> "Écris à [EMAIL_1]"
// mapping -> { "[EMAIL_1]": "jean@exemple.ch" }
anonymizer.deanonymize(anon, mapping); // -> "Écris à jean@exemple.ch"
```
### With an LLM (also catches names, addresses…)
```ts
import { Anonymizer, openAICompatibleProvider, presets } from '@mobiletic/anonymizer';
const anonymizer = new Anonymizer({
llm: openAICompatibleProvider({
baseUrl: process.env.LLM_BASE_URL!, // OpenAI, Infomaniak, vLLM, Ollama, …
apiKey: process.env.LLM_API_KEY!,
model: process.env.LLM_MODEL!,
timeoutMs: 3000,
}),
patterns: presets.swiss, // regex fallback if the LLM is down/misbehaves
});
const { anon, mapping } = await anonymizer.anonymize('Le dossier de Alain Jaccard est complet.');
// anon -> "Le dossier de [PER_1.NOM:M] est complet."
```
If the LLM call fails, times out, or returns an invalid shape, the anonymizer automatically falls back to
the regex engine — it never throws on a provider failure.
### Streaming de-anonymization
When you stream an LLM answer back to a user, restore real values without ever emitting a half-written
placeholder:
```ts
const stream = anonymizer.makeStreamDeanonymizer(mapping);
for await (const token of llmTokens) process.stdout.write(stream.push(token));
process.stdout.write(stream.flush());
```
### Anonymizing retrieved chunks consistently (RAG)
`anonymizeChunks(chunks, seed)` reuses the question's mapping (`seed`) so the same person gets the same id
across the question and every chunk, batches the LLM call, and de-collides genuinely new entities:
```ts
const q = await anonymizer.anonymize(question);
const { anon, mapping } = await anonymizer.anonymizeChunks(retrievedChunks, q.mapping);
// `mapping` is the full question chunks table; pass it to deanonymize()/makeStreamDeanonymizer().
```
## Configuration
```ts
new Anonymizer({
llm?, // LlmProvider — omit for regex-only mode
patterns?, // PatternDef[] — defaults to presets.swiss
nameHint?, // RegExp flagging likely names so the LLM is consulted (has a default)
logger?, // { warn(msg) } — receives fallback warnings; defaults to no-op
});
```
### Presets & custom patterns
```ts
import { presets } from '@mobiletic/anonymizer';
presets.swiss; // AVS, IBAN CH, EMAIL, Swiss phone, DATE
presets.generic; // EMAIL, IBAN, credit card, IPv4, phone, DATE
// Compose / extend:
const patterns = [
...presets.generic,
{ tag: 'TICKET', re: /\bJIRA-\d+\b/g }, // patterns must use the global flag
];
```
### Custom LLM provider
Implement `LlmProvider` to use any backend (Anthropic, a local model, a rules engine…):
```ts
import type { LlmProvider } from '@mobiletic/anonymizer';
const myProvider: LlmProvider = {
isConfigured: () => true,
async anonymize(text) { /* return { anon, mapping } */ },
async anonymizeBatch(texts, usedIds) { /* return { segments, mapping } */ },
};
```
## Placeholder format
```
[PER_1.NOM:M] entity PER #1, attribute NOM, context M (rich, from the LLM)
[EMAIL_1] structured id from the regex fallback
```
`PLACEHOLDER_RE` is exported if you need to scan text for placeholders.
## Compliance note
This library is a **best-effort** pseudonymization aid, not a guarantee of legal compliance. LLM and regex
detection can miss or mis-classify data. Validate against your own requirements (nLPD, GDPR, HIPAA, …)
before relying on it for regulated data.
## License
[MIT](./LICENSE) © Mobiletic