feat: initial release of @mobiletic/anonymizer

Framework-agnostic PII anonymization extracted from Mobiletic's chatbot.
Pluggable LLM detection + configurable regex fallback, deterministic
coreference, and streaming-safe de-anonymization.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Mobiletic
2026-06-20 21:58:38 +01:00
commit f09b917f1a
20 changed files with 3889 additions and 0 deletions

24
.github/workflows/ci.yml vendored Normal file
View File

@@ -0,0 +1,24 @@
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [18, 20, 22]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: npm
- run: npm ci
- run: npm run typecheck
- run: npm test
- run: npm run build

28
.github/workflows/release.yml vendored Normal file
View File

@@ -0,0 +1,28 @@
name: Release
on:
push:
tags:
- 'v*'
permissions:
contents: read
id-token: write # required for npm provenance
jobs:
publish:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
registry-url: https://registry.npmjs.org
cache: npm
- run: npm ci
- run: npm run typecheck
- run: npm test
- run: npm run build
- run: npm publish --provenance --access public
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

7
.gitignore vendored Normal file
View File

@@ -0,0 +1,7 @@
node_modules/
dist/
coverage/
*.log
.DS_Store
.env
.env.*

17
CHANGELOG.md Normal file
View File

@@ -0,0 +1,17 @@
# Changelog
All notable changes to this project are documented here. The format is based on
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.1.0] - Unreleased
### Added
- Initial public release, extracted from Mobiletic's production Swiss-nLPD chatbot.
- `Anonymizer` — pre-filter → LLM → regex fallback, bidirectional validation, deterministic coreference,
and de-collision across question and retrieved chunks (`anonymize`, `anonymizeChunks`, `deanonymize`).
- `makeStreamDeanonymizer` — streaming-safe de-anonymization that never leaks a split placeholder.
- `openAICompatibleProvider` — pluggable LLM detection over any OpenAI-compatible Chat Completions API.
- `presets.swiss` and `presets.generic` regex pattern sets; fully configurable custom patterns.
- Regex-only mode (no LLM provider required).

31
CODE_OF_CONDUCT.md Normal file
View File

@@ -0,0 +1,31 @@
# Contributor Covenant Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our community a harassment-free
experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex
characteristics, gender identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and
healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment include demonstrating empathy and
kindness, being respectful of differing opinions, giving and gracefully accepting constructive feedback,
and focusing on what is best for the community.
Unacceptable behavior includes the use of sexualized language or imagery, trolling or insulting comments,
public or private harassment, publishing others' private information without permission, and other conduct
which could reasonably be considered inappropriate in a professional setting.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the project team at
`conduct@mobiletic.com`. All complaints will be reviewed and investigated promptly and fairly.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org),
version 2.1.

37
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,37 @@
# Contributing to @mobiletic/anonymizer
Thanks for your interest in improving this project! Contributions of all kinds are welcome — bug reports,
new pattern presets, additional LLM providers, docs, and tests.
## Getting started
```bash
git clone https://github.com/mobiletic/anonymizer.git
cd anonymizer
npm install
npm test # run the test suite (vitest)
npm run typecheck # tsc --noEmit
npm run build # tsup → dist/ (ESM + CJS + types)
```
## Guidelines
- **Tests first.** Every behavior change needs a test. The anonymization core is privacy-critical — we
keep coverage tight, especially around coreference, de-collision, and streaming.
- **No runtime dependencies.** The core must stay dependency-free. New providers should rely only on
platform APIs (e.g. `fetch`).
- **Keep it framework-agnostic.** No framework-specific code (NestJS, Express, React, …) in the package.
- **Patterns use the global flag.** Any `PatternDef.re` must be a global (`/…/g`) regex.
- **Conventional commits** are appreciated (`feat:`, `fix:`, `docs:`, `test:`, `chore:`).
## Reporting security issues
Please do **not** open public issues for vulnerabilities (e.g. a PII-leak path). Email
`security@mobiletic.com` instead.
## Pull requests
1. Fork and branch from `main`.
2. Add tests and update docs.
3. Ensure `npm test`, `npm run typecheck`, and `npm run build` pass.
4. Open the PR with a clear description of the change and its motivation.

21
LICENSE Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Mobiletic
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

146
README.md Normal file
View File

@@ -0,0 +1,146 @@
# @mobiletic/anonymizer
Framework-agnostic **PII anonymization & pseudonymization** for TypeScript/JavaScript.
It replaces personal data in text with stable placeholders before the text leaves your trust boundary
(e.g. before sending it to a third-party LLM, log sink, or analytics pipeline), and restores the real
values afterwards — including across **streamed** tokens.
- 🔌 **Pluggable LLM detection** — catch free-form PII (names, addresses) via any OpenAI-compatible
endpoint, or your own provider. No LLM? It degrades gracefully to regex.
- 🧩 **Deterministic regex fallback** — structured identifiers (email, phone, IBAN, …) via configurable
pattern presets (`swiss`, `generic`) or your own.
- 🔁 **Deterministic coreference** — the same person keeps the same id (`[PER_1]`) across a question and
every retrieved chunk, with automatic de-collision.
- 🌊 **Streaming-safe** — a placeholder split across two stream chunks (`[PER_` + `1.NOM:M]`) is never
leaked partially.
- 🪶 **Zero runtime dependencies**, ESM + CJS, fully typed.
> Built by [Mobiletic](https://mobiletic.com) and extracted from a production Swiss-nLPD chatbot.
## Install
```bash
npm install @mobiletic/anonymizer
```
Requires Node ≥ 18 (uses native `fetch`).
## Quick start
### Regex-only (no LLM, fully deterministic)
```ts
import { Anonymizer, presets } from '@mobiletic/anonymizer';
const anonymizer = new Anonymizer({ patterns: presets.swiss });
const { anon, mapping } = await anonymizer.anonymize('Écris à jean@exemple.ch');
// anon -> "Écris à [EMAIL_1]"
// mapping -> { "[EMAIL_1]": "jean@exemple.ch" }
anonymizer.deanonymize(anon, mapping); // -> "Écris à jean@exemple.ch"
```
### With an LLM (also catches names, addresses…)
```ts
import { Anonymizer, openAICompatibleProvider, presets } from '@mobiletic/anonymizer';
const anonymizer = new Anonymizer({
llm: openAICompatibleProvider({
baseUrl: process.env.LLM_BASE_URL!, // OpenAI, Infomaniak, vLLM, Ollama, …
apiKey: process.env.LLM_API_KEY!,
model: process.env.LLM_MODEL!,
timeoutMs: 3000,
}),
patterns: presets.swiss, // regex fallback if the LLM is down/misbehaves
});
const { anon, mapping } = await anonymizer.anonymize('Le dossier de Alain Jaccard est complet.');
// anon -> "Le dossier de [PER_1.NOM:M] est complet."
```
If the LLM call fails, times out, or returns an invalid shape, the anonymizer automatically falls back to
the regex engine — it never throws on a provider failure.
### Streaming de-anonymization
When you stream an LLM answer back to a user, restore real values without ever emitting a half-written
placeholder:
```ts
const stream = anonymizer.makeStreamDeanonymizer(mapping);
for await (const token of llmTokens) process.stdout.write(stream.push(token));
process.stdout.write(stream.flush());
```
### Anonymizing retrieved chunks consistently (RAG)
`anonymizeChunks(chunks, seed)` reuses the question's mapping (`seed`) so the same person gets the same id
across the question and every chunk, batches the LLM call, and de-collides genuinely new entities:
```ts
const q = await anonymizer.anonymize(question);
const { anon, mapping } = await anonymizer.anonymizeChunks(retrievedChunks, q.mapping);
// `mapping` is the full question chunks table; pass it to deanonymize()/makeStreamDeanonymizer().
```
## Configuration
```ts
new Anonymizer({
llm?, // LlmProvider — omit for regex-only mode
patterns?, // PatternDef[] — defaults to presets.swiss
nameHint?, // RegExp flagging likely names so the LLM is consulted (has a default)
logger?, // { warn(msg) } — receives fallback warnings; defaults to no-op
});
```
### Presets & custom patterns
```ts
import { presets } from '@mobiletic/anonymizer';
presets.swiss; // AVS, IBAN CH, EMAIL, Swiss phone, DATE
presets.generic; // EMAIL, IBAN, credit card, IPv4, phone, DATE
// Compose / extend:
const patterns = [
...presets.generic,
{ tag: 'TICKET', re: /\bJIRA-\d+\b/g }, // patterns must use the global flag
];
```
### Custom LLM provider
Implement `LlmProvider` to use any backend (Anthropic, a local model, a rules engine…):
```ts
import type { LlmProvider } from '@mobiletic/anonymizer';
const myProvider: LlmProvider = {
isConfigured: () => true,
async anonymize(text) { /* return { anon, mapping } */ },
async anonymizeBatch(texts, usedIds) { /* return { segments, mapping } */ },
};
```
## Placeholder format
```
[PER_1.NOM:M] entity PER #1, attribute NOM, context M (rich, from the LLM)
[EMAIL_1] structured id from the regex fallback
```
`PLACEHOLDER_RE` is exported if you need to scan text for placeholders.
## Compliance note
This library is a **best-effort** pseudonymization aid, not a guarantee of legal compliance. LLM and regex
detection can miss or mis-classify data. Validate against your own requirements (nLPD, GDPR, HIPAA, …)
before relying on it for regulated data.
## License
[MIT](./LICENSE) © Mobiletic

2791
package-lock.json generated Normal file

File diff suppressed because it is too large Load Diff

62
package.json Normal file
View File

@@ -0,0 +1,62 @@
{
"name": "@mobiletic/anonymizer",
"version": "0.1.0",
"description": "Framework-agnostic PII anonymization & pseudonymization: pluggable LLM detection for free-form PII (names, addresses) with a deterministic regex fallback, deterministic coreference, and streaming-safe de-anonymization.",
"license": "MIT",
"author": "Mobiletic",
"homepage": "https://github.com/mobiletic/anonymizer#readme",
"repository": {
"type": "git",
"url": "git+https://github.com/mobiletic/anonymizer.git"
},
"bugs": {
"url": "https://github.com/mobiletic/anonymizer/issues"
},
"keywords": [
"anonymization",
"pseudonymization",
"pii",
"redaction",
"privacy",
"gdpr",
"nlpd",
"llm",
"data-protection"
],
"type": "module",
"main": "./dist/index.cjs",
"module": "./dist/index.js",
"types": "./dist/index.d.ts",
"exports": {
".": {
"types": "./dist/index.d.ts",
"import": "./dist/index.js",
"require": "./dist/index.cjs"
}
},
"files": [
"dist",
"README.md",
"LICENSE"
],
"engines": {
"node": ">=18"
},
"scripts": {
"build": "tsup",
"test": "vitest run",
"test:watch": "vitest",
"typecheck": "tsc --noEmit",
"prepublishOnly": "npm run build"
},
"devDependencies": {
"tsup": "^8.0.0",
"typescript": "^5.4.0",
"vitest": "^1.6.0"
},
"publishConfig": {
"access": "public",
"provenance": true
},
"sideEffects": false
}

244
src/anonymizer.ts Normal file
View File

@@ -0,0 +1,244 @@
import { RegexFallback } from './fallback.js';
import { presets } from './presets.js';
import {
type AnonymizationResult,
type AnonymizerConfig,
type LlmProvider,
type Logger,
type StreamDeanonymizer,
PLACEHOLDER_RE,
} from './types.js';
const NOOP_LOGGER: Logger = { warn: () => {} };
// Likely proper-name heuristic (two capitalized words), in addition to the structured fallback.
const DEFAULT_NAME_HINT = /\b[A-ZÀ-Ý][a-zà-ÿ]+\s+[A-ZÀ-Ý][a-zà-ÿ]+\b/;
/**
* Framework-agnostic anonymization engine.
*
* `anonymize()`: pre-filter (skips the LLM when there is no PII) → LLM →
* regex fallback on failure/timeout → bidirectional validation.
* `deanonymize()` restores the real values. `makeStreamDeanonymizer()` handles
* the critical case of a placeholder fragmented across several streamed tokens.
*
* Pass an {@link LlmProvider} to detect free-form PII (names, addresses);
* omit it for deterministic regex-only mode.
*/
export class Anonymizer {
private readonly llm?: LlmProvider;
private readonly fallback: RegexFallback;
private readonly nameHint: RegExp;
private readonly logger: Logger;
constructor(config: AnonymizerConfig = {}) {
this.llm = config.llm;
this.fallback = new RegexFallback(config.patterns ?? presets.swiss);
this.nameHint = config.nameHint ?? DEFAULT_NAME_HINT;
this.logger = config.logger ?? NOOP_LOGGER;
}
private hasLlm(): boolean {
return !!this.llm && this.llm.isConfigured();
}
private likelyHasPii(text: string): boolean {
return this.fallback.hasPii(text) || this.nameHint.test(text);
}
async anonymize(text: string): Promise<AnonymizationResult> {
// Pre-filter: most text has no PII → avoid the LLM round-trip (latency + cost)
// and return the text unchanged.
if (!this.likelyHasPii(text)) return { anon: text, mapping: {} };
if (this.hasLlm()) {
try {
const result = await this.llm!.anonymize(text);
this.validate(result); // throws if inconsistent → fall back
return result;
} catch (err) {
this.logger.warn(`LLM unavailable/invalid → regex fallback: ${(err as Error).message}`);
}
}
return this.fallback.anonymize(text);
}
/**
* Anonymize retrieved chunks BEFORE sending them on, CONSISTENTLY with the
* question's mapping (`seed`). Returns the anonymized chunks + the COMPLETE
* mapping (question chunks). Strategy:
* 1. Deterministic reuse: replace any already-known value (from the question)
* by its placeholder → question↔chunk coreference with no LLM call.
* 2. Pre-filter: if no residual PII remains → return as-is (most corpus chunks
* have no PII → LLM cost avoided).
* 3. Otherwise: ONE batched LLM call, then de-collision merge (renumber the
* genuinely new entities). Failure/timeout → regex fallback per chunk.
*/
async anonymizeChunks(
chunks: string[],
seed: Record<string, string>,
): Promise<{ anon: string[]; mapping: Record<string, string> }> {
if (!chunks.length) return { anon: [], mapping: { ...seed } };
const seeded = chunks.map((c) => this.applyKnown(c, seed));
if (!seeded.some((c) => this.likelyHasPii(c))) return { anon: seeded, mapping: { ...seed } };
if (this.hasLlm()) {
try {
const { segments, mapping: gMap } = await this.llm!.anonymizeBatch(seeded, Object.keys(seed));
if (segments.length !== seeded.length) throw new Error('SEGMENT_COUNT_MISMATCH');
const { mapping, rename } = this.mergeMappings(seed, gMap);
const anon = segments.map((s) => this.applyRename(s, rename));
// Anti-leak: every placeholder present in a chunk must be mapped.
for (const s of anon) {
for (const ph of s.match(PLACEHOLDER_RE) ?? []) {
if (!(ph in mapping)) throw new Error(`unmapped placeholder ${ph}`);
}
}
return { anon, mapping };
} catch (err) {
this.logger.warn(`Chunk anonymization → regex fallback: ${(err as Error).message}`);
}
}
return this.fallbackChunks(seeded, seed);
}
/** Replace known values (seed) by their placeholder, longest values first. */
private applyKnown(text: string, seed: Record<string, string>): string {
let out = text;
for (const [ph, val] of Object.entries(seed)
.filter(([, v]) => v)
.sort((a, b) => b[1].length - a[1].length)) {
out = out.split(val).join(ph);
}
return out;
}
private applyRename(text: string, rename: Record<string, string>): string {
let out = text;
for (const [from, to] of Object.entries(rename)) out = out.split(from).join(to);
return out;
}
/** `[PER_1.NOM:M]` → { type:'PER', index:1, suffix:'.NOM:M' } ; `[EMAIL_2]` → suffix ''. */
private parsePlaceholder(ph: string): { type: string; index: number; suffix: string } | null {
const m = /^\[([A-Z]+)_(\d+)(\..*)?\]$/.exec(ph);
return m ? { type: m[1], index: Number(m[2]), suffix: m[3] ?? '' } : null;
}
/**
* Merge the chunks' LLM mapping into the question's (seed).
* - value already known → reuse the question's placeholder (rename).
* - new value without collision → added as-is.
* - collision (same placeholder, different value) → renumbered (next index).
*/
private mergeMappings(
seed: Record<string, string>,
gMap: Record<string, string>,
): { mapping: Record<string, string>; rename: Record<string, string> } {
const mapping: Record<string, string> = { ...seed };
const valueToPh = new Map<string, string>();
const maxIdx = new Map<string, number>();
for (const [ph, val] of Object.entries(seed)) {
valueToPh.set(val, ph);
const p = this.parsePlaceholder(ph);
if (p) maxIdx.set(p.type, Math.max(maxIdx.get(p.type) ?? 0, p.index));
}
const rename: Record<string, string> = {};
for (const [gph, val] of Object.entries(gMap)) {
const known = valueToPh.get(val);
if (known) {
if (known !== gph) rename[gph] = known; // same entity as the question
continue;
}
if (!(gph in mapping)) {
mapping[gph] = val;
valueToPh.set(val, gph);
const p = this.parsePlaceholder(gph);
if (p) maxIdx.set(p.type, Math.max(maxIdx.get(p.type) ?? 0, p.index));
} else {
// collision: placeholder reused for ANOTHER value → renumber.
const p = this.parsePlaceholder(gph);
const type = p?.type ?? 'PER';
const next = (maxIdx.get(type) ?? 0) + 1;
maxIdx.set(type, next);
const newPh = `[${type}_${next}${p?.suffix ?? ''}]`;
rename[gph] = newPh;
mapping[newPh] = val;
valueToPh.set(val, newPh);
}
}
return { mapping, rename };
}
/** Fallback (no LLM): regex per chunk, de-collision merged into the seed. */
private fallbackChunks(
seeded: string[],
seed: Record<string, string>,
): { anon: string[]; mapping: Record<string, string> } {
let mapping: Record<string, string> = { ...seed };
const anon: string[] = [];
for (const c of seeded) {
const r = this.fallback.anonymize(c);
const { mapping: merged, rename } = this.mergeMappings(mapping, r.mapping);
mapping = merged;
anon.push(this.applyRename(r.anon, rename));
}
return { anon, mapping };
}
/** Every placeholder in the text must be in the mapping AND vice-versa (anti-leak). */
private validate(result: AnonymizationResult): void {
const inText = new Set(result.anon.match(PLACEHOLDER_RE) ?? []);
for (const ph of inText) {
if (!(ph in result.mapping)) throw new Error(`placeholder ${ph} without mapping`);
}
for (const ph of Object.keys(result.mapping)) {
if (!inText.has(ph)) throw new Error(`orphan mapping ${ph}`);
}
}
/** Restore the real values (replace every known placeholder). */
deanonymize(text: string, mapping: Record<string, string>): string {
if (!text) return text;
let out = text;
for (const [ph, value] of Object.entries(mapping)) {
out = out.split(ph).join(value);
}
return out;
}
/**
* Streaming de-anonymizer. Buffers any unterminated `[...]` fragment until it
* completes, so a partial placeholder is never emitted and never leaks.
*/
makeStreamDeanonymizer(mapping: Record<string, string>): StreamDeanonymizer {
let buf = '';
const deanon = (s: string) => this.deanonymize(s, mapping);
return {
push: (chunk: string): string => {
buf += chunk;
const lastOpen = buf.lastIndexOf('[');
let emit: string;
if (lastOpen === -1) {
emit = buf;
buf = '';
} else if (buf.indexOf(']', lastOpen) === -1) {
// '[' opened without ']' → placeholder possibly cut: hold from there.
emit = buf.slice(0, lastOpen);
buf = buf.slice(lastOpen);
} else {
emit = buf;
buf = '';
}
return deanon(emit);
},
flush: (): string => {
const out = deanon(buf);
buf = '';
return out;
},
};
}
}

40
src/fallback.ts Normal file
View File

@@ -0,0 +1,40 @@
import type { AnonymizationResult, PatternDef } from './types.js';
/**
* Regex-based anonymizer for STRUCTURED identifiers (email, phone, IBAN, …).
* It is the deterministic fallback used when no LLM provider is available or a
* provider call fails. It does NOT detect proper names — that is the LLM's job.
*
* Patterns are injected (see {@link presets}) so the same engine serves any
* locale or regulation.
*/
export class RegexFallback {
constructor(private readonly patterns: PatternDef[]) {}
anonymize(text: string): AnonymizationResult {
const mapping: Record<string, string> = {};
const counters: Record<string, number> = {};
let anon = text;
for (const { tag, re } of this.patterns) {
anon = anon.replace(re, (match) => {
// Reuse the same placeholder for an identical value (consistency).
const existing = Object.entries(mapping).find(([, v]) => v === match);
if (existing) return existing[0];
counters[tag] = (counters[tag] ?? 0) + 1;
const ph = `[${tag}_${counters[tag]}]`;
mapping[ph] = match;
return ph;
});
}
return { anon, mapping };
}
/** Fast pre-check: is there any structured identifier worth anonymizing? */
hasPii(text: string): boolean {
return this.patterns.some(({ re }) => {
re.lastIndex = 0;
return re.test(text);
});
}
}

17
src/index.ts Normal file
View File

@@ -0,0 +1,17 @@
export { Anonymizer } from './anonymizer.js';
export { RegexFallback } from './fallback.js';
export { presets } from './presets.js';
export {
openAICompatibleProvider,
DEFAULT_SYSTEM_PROMPT,
type OpenAICompatibleOptions,
} from './providers/openai-compatible.js';
export {
PLACEHOLDER_RE,
type AnonymizationResult,
type AnonymizerConfig,
type LlmProvider,
type PatternDef,
type Logger,
type StreamDeanonymizer,
} from './types.js';

29
src/presets.ts Normal file
View File

@@ -0,0 +1,29 @@
import type { PatternDef } from './types.js';
/**
* Swiss preset — the structured identifiers most relevant to the Swiss nLPD.
* Order matters: specific patterns (AVS, IBAN) come before generic ones.
*/
const swiss: PatternDef[] = [
{ tag: 'AVS', re: /\b756\.\d{4}\.\d{4}\.\d{2}\b/g }, // Swiss social security (AVS/AHV)
{ tag: 'IBAN', re: /\bCH\d{2}[0-9A-Z]{17}\b/gi }, // Swiss IBAN
{ tag: 'EMAIL', re: /\b[\w.+-]+@[\w-]+\.[\w.-]+\b/g },
{ tag: 'TEL', re: /(?:\+41|0)(?:[\s.-]?\d){9}\b/g }, // Swiss phone number
{ tag: 'DATE', re: /\b\d{1,2}[./-]\d{1,2}[./-]\d{2,4}\b/g },
];
/**
* Generic preset — locale-agnostic identifiers useful as a starting point
* anywhere. Extend or compose with your own {@link PatternDef}s as needed.
*/
const generic: PatternDef[] = [
{ tag: 'EMAIL', re: /\b[\w.+-]+@[\w-]+\.[\w.-]+\b/g },
{ tag: 'IBAN', re: /\b[A-Z]{2}\d{2}[0-9A-Z]{11,30}\b/g },
{ tag: 'CREDIT_CARD', re: /\b(?:\d[ -]?){13,16}\b/g },
{ tag: 'IPV4', re: /\b(?:\d{1,3}\.){3}\d{1,3}\b/g },
{ tag: 'TEL', re: /\b(?:\+?\d{1,3}[\s.-]?)?(?:\(?\d{2,4}\)?[\s.-]?){2,4}\d{2,4}\b/g },
{ tag: 'DATE', re: /\b\d{1,2}[./-]\d{1,2}[./-]\d{2,4}\b/g },
];
/** Built-in pattern sets for the regex fallback. */
export const presets = { swiss, generic };

View File

@@ -0,0 +1,117 @@
import type { AnonymizationResult, LlmProvider } from '../types.js';
/** Default nLPD pseudonymization prompt (French). Override for other locales/regulations. */
export const DEFAULT_SYSTEM_PROMPT = [
'Tu es un moteur de pseudonymisation conforme à la nLPD suisse.',
'Identifie UNIQUEMENT les données personnelles (identifiants directs et indirects)',
'et remplace-les par des placeholders. Ne touche à RIEN dautre.',
'',
'FORMAT: [ENTITE_ID.ATTRIBUT:CONTEXTE]',
'Entités: PER (personne), ORG (organisation), LOC (lieu autonome).',
'Attributs PER: NOM, PRENOM, DATE_NAISSANCE, AGE, ADRESSE, EMAIL, TELEPHONE, AVS, IBAN, NSS.',
'Contexte = indice non-identifiant utile au raisonnement:',
' NOM:M|F|U · DATE_NAISSANCE:<année> · AGE:Mineur|Adulte',
' ADRESSE:Lieu|Rue|Ville|NPA|Pays · ORG:Entreprise|Ecole',
'',
'RÈGLES:',
'1. Coréférence: la MÊME personne garde le MÊME identifiant (PER_1) dans tout le texte.',
'2. Nanonymise JAMAIS les termes pédagogiques/techniques (langages, concepts, titres de cours, fonctions).',
'3. Si AUCUNE donnée personnelle: renvoie le texte original et "mapping": {}.',
'4. Nanonymise pas un placeholder déjà présent (idempotence).',
'5. Sortie STRICTEMENT JSON valide: {"texte_anonymise": "...", "mapping": {"[PER_1.NOM:M]": "..."}}.',
].join('\n');
export interface OpenAICompatibleOptions {
/** Base URL of an OpenAI-compatible API, e.g. `https://api.openai.com/v1`. */
baseUrl: string;
/** Bearer API key. */
apiKey: string;
/** Model id, e.g. `gpt-4o-mini` or `gemma-3-...`. */
model: string;
/** Per-request timeout in milliseconds (default 3000). */
timeoutMs?: number;
/** Override the system prompt (e.g. for another language or regulation). */
systemPrompt?: string;
/** Extra instructions appended to the system prompt in batch mode. */
batchInstructions?: (usedIds: string[]) => string;
}
const DEFAULT_BATCH_INSTRUCTIONS = (usedIds: string[]): string =>
'\n\nMODE LOT (segments) :' +
'\n- ENTRÉE : un objet JSON {"segments": ["…", "…"]}.' +
'\n- Anonymise CHAQUE segment ; coréférence GLOBALE entre segments.' +
(usedIds.length
? `\n- Identifiants DÉJÀ attribués (réutilise-les pour les mêmes valeurs, n'en duplique AUCUN pour d'autres valeurs) : ${usedIds.join(', ')}.`
: '') +
'\n- SORTIE STRICTEMENT JSON : {"segments": ["…anonymisé…"], "mapping": {"[PER_1.NOM:M]": "…"}}.' +
'\n- "segments" DOIT avoir la même longueur et le même ordre que lentrée ; "mapping" ne contient que les NOUVELLES entités.';
/**
* Build an {@link LlmProvider} backed by any OpenAI-compatible Chat Completions
* endpoint (OpenAI, Infomaniak, vLLM, Ollama, …). Uses `temperature: 0` and
* `response_format: json_object` for deterministic, parseable output, and throws
* on any failure so the {@link Anonymizer} falls back to its regex engine.
*/
export function openAICompatibleProvider(opts: OpenAICompatibleOptions): LlmProvider {
const timeout = opts.timeoutMs ?? 3000;
const systemPrompt = opts.systemPrompt ?? DEFAULT_SYSTEM_PROMPT;
const batchInstructions = opts.batchInstructions ?? DEFAULT_BATCH_INSTRUCTIONS;
const isConfigured = (): boolean => !!(opts.baseUrl && opts.apiKey && opts.model);
async function chat(system: string, user: string): Promise<string> {
if (!isConfigured()) throw new Error('LLM_NOT_CONFIGURED');
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), timeout);
try {
const res = await fetch(`${opts.baseUrl}/chat/completions`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${opts.apiKey}`,
},
body: JSON.stringify({
model: opts.model,
temperature: 0,
response_format: { type: 'json_object' },
messages: [
{ role: 'system', content: system },
{ role: 'user', content: user },
],
}),
signal: controller.signal,
});
if (!res.ok) throw new Error(`LLM_HTTP_${res.status}`);
const data = (await res.json()) as { choices?: Array<{ message?: { content?: string } }> };
return data?.choices?.[0]?.message?.content ?? '';
} finally {
clearTimeout(timer);
}
}
return {
isConfigured,
async anonymize(text: string): Promise<AnonymizationResult> {
const content = await chat(systemPrompt, text);
const parsed = JSON.parse(content) as { texte_anonymise?: string; mapping?: Record<string, string> };
if (typeof parsed.texte_anonymise !== 'string' || typeof parsed.mapping !== 'object' || !parsed.mapping) {
throw new Error('LLM_BAD_SHAPE');
}
return { anon: parsed.texte_anonymise, mapping: parsed.mapping };
},
async anonymizeBatch(
texts: string[],
usedIds: string[],
): Promise<{ segments: string[]; mapping: Record<string, string> }> {
const system = systemPrompt + batchInstructions(usedIds);
const content = await chat(system, JSON.stringify({ segments: texts }));
const parsed = JSON.parse(content) as { segments?: string[]; mapping?: Record<string, string> };
if (!Array.isArray(parsed.segments) || typeof parsed.mapping !== 'object' || !parsed.mapping) {
throw new Error('LLM_BATCH_BAD_SHAPE');
}
return { segments: parsed.segments, mapping: parsed.mapping };
},
};
}

67
src/types.ts Normal file
View File

@@ -0,0 +1,67 @@
/** Result of an anonymization: text with placeholders + the reverse-mapping table. */
export interface AnonymizationResult {
/** The text with every detected PII value replaced by a placeholder. */
anon: string;
/** Placeholder → original value, e.g. `{ "[PER_1.NOM:M]": "Alain JACCARD" }`. */
mapping: Record<string, string>;
}
/**
* Matches a single placeholder. Covers the rich LLM form `[PER_1.NOM:M]`
* and the plain regex-fallback form `[EMAIL_1]`.
*/
export const PLACEHOLDER_RE = /\[[A-Z]+_\d+(?:\.[A-Z_]+:[^\]]+)?\]/g;
/**
* Pluggable LLM backend. Implement this (or use {@link openAICompatibleProvider})
* to let the {@link Anonymizer} detect free-form PII such as proper names that
* regular expressions cannot reliably catch.
*/
export interface LlmProvider {
/** Whether the provider is ready to be called. When `false`, the Anonymizer skips it. */
isConfigured(): boolean;
/** Anonymize a single piece of text. */
anonymize(text: string): Promise<AnonymizationResult>;
/**
* Anonymize several segments in ONE call with GLOBAL coreference (same entity →
* same id across segments) and reuse of the ids already assigned upstream
* (`usedIds`). Returns the anonymized segments (same order/length) plus the
* mapping of the NEW entities only.
*/
anonymizeBatch(
texts: string[],
usedIds: string[],
): Promise<{ segments: string[]; mapping: Record<string, string> }>;
}
/** A structured-PII detector: a tag (e.g. `EMAIL`) and the global regex that finds it. */
export interface PatternDef {
tag: string;
re: RegExp;
}
/** Minimal logger sink. Defaults to a no-op; pass your own to capture fallback warnings. */
export interface Logger {
warn(msg: string): void;
}
/** Configuration for {@link Anonymizer}. */
export interface AnonymizerConfig {
/** LLM backend for free-form PII (names, addresses…). Omit for regex-only mode. */
llm?: LlmProvider;
/** Structured-PII patterns for the regex fallback. Defaults to {@link presets.swiss}. */
patterns?: PatternDef[];
/** Heuristic that flags likely proper names so the LLM is consulted. Has a sensible default. */
nameHint?: RegExp;
/** Where fallback/diagnostic warnings go. Defaults to a no-op. */
logger?: Logger;
}
/**
* Streaming de-anonymizer: buffers placeholders that get split across token
* boundaries so a partial `[PER_` is never emitted (and never leaks).
*/
export interface StreamDeanonymizer {
push(chunk: string): string;
flush(): string;
}

124
test/anonymizer.test.ts Normal file
View File

@@ -0,0 +1,124 @@
import { describe, it, expect, vi } from 'vitest';
import { Anonymizer, presets, type LlmProvider } from '../src/index.js';
/** An LLM provider that is configured but always fails → forces the regex fallback. */
const failingLlm = (overrides: Partial<LlmProvider> = {}): LlmProvider => ({
isConfigured: () => true,
anonymize: vi.fn().mockRejectedValue(new Error('LLM_NOT_CONFIGURED')),
anonymizeBatch: vi.fn().mockRejectedValue(new Error('LLM_NOT_CONFIGURED')),
...overrides,
});
describe('Anonymizer (Swiss preset)', () => {
const svc = new Anonymizer({ llm: failingLlm(), patterns: presets.swiss });
it('skips anonymization (no LLM call) when there is no PII', async () => {
const llm = failingLlm();
const s = new Anonymizer({ llm, patterns: presets.swiss });
const r = await s.anonymize('Explique la différence entre INNER JOIN et LEFT JOIN');
expect(r.mapping).toEqual({});
expect(r.anon).toContain('INNER JOIN');
expect(llm.anonymize).not.toHaveBeenCalled();
});
it('falls back to regex for structured PII (email/phone/AVS)', async () => {
const r = await svc.anonymize('Contact: jean@exemple.ch, +41 79 123 45 67, AVS 756.1234.5678.90');
expect(r.anon).not.toContain('jean@exemple.ch');
expect(r.anon).not.toContain('756.1234.5678.90');
expect(Object.values(r.mapping)).toContain('jean@exemple.ch');
expect(svc.deanonymize(r.anon, r.mapping)).toContain('jean@exemple.ch');
});
it('deanonymizes a placeholder split across stream tokens (critical)', () => {
const mapping = { '[PER_1.NOM:M]': 'Alain JACCARD' };
const d = svc.makeStreamDeanonymizer(mapping);
let out = '';
out += d.push('Bonjour [PER_');
out += d.push('1.NOM');
out += d.push(':M], ravi');
out += d.flush();
expect(out).toBe('Bonjour Alain JACCARD, ravi');
expect(out).not.toContain('[PER_');
});
it('streams plain text through unchanged', () => {
const d = svc.makeStreamDeanonymizer({});
expect(d.push('Un INNER JOIN ') + d.push('retourne...') + d.flush()).toBe('Un INNER JOIN retourne...');
});
describe('anonymizeChunks', () => {
it('PII-free chunks → no LLM call, returned as-is', async () => {
const llm = failingLlm();
const s = new Anonymizer({ llm, patterns: presets.swiss });
const r = await s.anonymizeChunks(['Un INNER JOIN combine deux tables.'], {});
expect(llm.anonymizeBatch).not.toHaveBeenCalled();
expect(r.anon[0]).toContain('INNER JOIN');
expect(r.mapping).toEqual({});
});
it('reuses the question placeholder for the same person (deterministic, no LLM)', async () => {
const llm = failingLlm();
const s = new Anonymizer({ llm, patterns: presets.swiss });
const seed = { '[PER_1.NOM:M]': 'Alain JACCARD' };
const r = await s.anonymizeChunks(['Le dossier de Alain JACCARD est complet.'], seed);
expect(llm.anonymizeBatch).not.toHaveBeenCalled();
expect(r.anon[0]).toContain('[PER_1.NOM:M]');
expect(r.anon[0]).not.toContain('Alain JACCARD');
expect(r.mapping).toEqual(seed);
});
it('renumbers a NEW person that collides with the question placeholder', async () => {
const anonymizeBatch = vi.fn().mockResolvedValue({
segments: ['[PER_1.NOM:M] a signé.'],
mapping: { '[PER_1.NOM:M]': 'Bob Martin' },
});
const s = new Anonymizer({ llm: failingLlm({ anonymizeBatch }), patterns: presets.swiss });
const seed = { '[PER_1.NOM:M]': 'Alain JACCARD' };
const r = await s.anonymizeChunks(['Bob Martin a signé.'], seed);
expect(anonymizeBatch).toHaveBeenCalledTimes(1);
expect(r.anon[0]).toContain('[PER_2.NOM:M]');
expect(r.mapping['[PER_1.NOM:M]']).toBe('Alain JACCARD');
expect(r.mapping['[PER_2.NOM:M]']).toBe('Bob Martin');
});
it('falls back to regex (structured ids) when the LLM is unavailable', async () => {
const s = new Anonymizer({ llm: failingLlm(), patterns: presets.swiss });
const r = await s.anonymizeChunks(['Contact : jean@exemple.ch'], {});
expect(r.anon[0]).not.toContain('jean@exemple.ch');
expect(Object.values(r.mapping)).toContain('jean@exemple.ch');
});
});
describe('regex-only mode (no LLM provider)', () => {
const s = new Anonymizer({ patterns: presets.swiss });
it('still anonymizes structured PII without any provider', async () => {
const r = await s.anonymize('Écris à jean@exemple.ch');
expect(r.anon).not.toContain('jean@exemple.ch');
expect(Object.values(r.mapping)).toContain('jean@exemple.ch');
});
it('leaves a bare proper name untouched (no LLM to catch it)', async () => {
const r = await s.anonymize('Alain Jaccard a réussi');
// The name-hint flags it, but with no LLM the regex fallback finds no structured id.
expect(r.anon).toContain('Alain Jaccard');
expect(r.mapping).toEqual({});
});
});
describe('configurable presets', () => {
it('generic preset anonymizes an IPv4 address', async () => {
const s = new Anonymizer({ patterns: presets.generic });
const r = await s.anonymize('Serveur 192.168.1.42 indisponible');
expect(r.anon).not.toContain('192.168.1.42');
expect(Object.values(r.mapping)).toContain('192.168.1.42');
});
it('accepts a fully custom pattern set', async () => {
const s = new Anonymizer({ patterns: [{ tag: 'TICKET', re: /\bJIRA-\d+\b/g }] });
const r = await s.anonymize('Voir JIRA-123');
expect(r.anon).toContain('[TICKET_1]');
expect(r.mapping['[TICKET_1]']).toBe('JIRA-123');
});
});
});

View File

@@ -0,0 +1,58 @@
import { describe, it, expect, vi, afterEach } from 'vitest';
import { openAICompatibleProvider } from '../src/index.js';
const opts = { baseUrl: 'https://api.example.com/v1', apiKey: 'k', model: 'm' };
function mockFetchJson(content: unknown) {
return vi.fn().mockResolvedValue({
ok: true,
json: async () => ({ choices: [{ message: { content: JSON.stringify(content) } }] }),
});
}
afterEach(() => {
vi.unstubAllGlobals();
});
describe('openAICompatibleProvider', () => {
it('isConfigured() reflects whether baseUrl/apiKey/model are present', () => {
expect(openAICompatibleProvider(opts).isConfigured()).toBe(true);
expect(openAICompatibleProvider({ ...opts, apiKey: '' }).isConfigured()).toBe(false);
});
it('anonymize() parses {texte_anonymise, mapping} and posts temperature 0 + json_object', async () => {
const fetchMock = mockFetchJson({ texte_anonymise: '[EMAIL_1]', mapping: { '[EMAIL_1]': 'a@b.ch' } });
vi.stubGlobal('fetch', fetchMock);
const r = await openAICompatibleProvider(opts).anonymize('a@b.ch');
expect(r).toEqual({ anon: '[EMAIL_1]', mapping: { '[EMAIL_1]': 'a@b.ch' } });
const [url, init] = fetchMock.mock.calls[0];
expect(url).toBe('https://api.example.com/v1/chat/completions');
const body = JSON.parse(init.body);
expect(body.temperature).toBe(0);
expect(body.response_format).toEqual({ type: 'json_object' });
expect(init.headers.Authorization).toBe('Bearer k');
});
it('anonymizeBatch() returns {segments, mapping} and includes used ids in the prompt', async () => {
const fetchMock = mockFetchJson({ segments: ['[PER_2.NOM:M]'], mapping: { '[PER_2.NOM:M]': 'Bob' } });
vi.stubGlobal('fetch', fetchMock);
const r = await openAICompatibleProvider(opts).anonymizeBatch(['Bob'], ['[PER_1.NOM:M]']);
expect(r.segments).toEqual(['[PER_2.NOM:M]']);
const system = JSON.parse(fetchMock.mock.calls[0][1].body).messages[0].content;
expect(system).toContain('[PER_1.NOM:M]');
});
it('throws on a non-OK HTTP status', async () => {
vi.stubGlobal('fetch', vi.fn().mockResolvedValue({ ok: false, status: 500 }));
await expect(openAICompatibleProvider(opts).anonymize('x')).rejects.toThrow('LLM_HTTP_500');
});
it('throws on a malformed response shape', async () => {
vi.stubGlobal('fetch', mockFetchJson({ wrong: true }));
await expect(openAICompatibleProvider(opts).anonymize('x')).rejects.toThrow('LLM_BAD_SHAPE');
});
});

18
tsconfig.json Normal file
View File

@@ -0,0 +1,18 @@
{
"compilerOptions": {
"target": "ES2021",
"module": "ESNext",
"moduleResolution": "Bundler",
"lib": ["ES2021", "DOM"],
"declaration": true,
"strict": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"noFallthroughCasesInSwitch": true,
"esModuleInterop": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"verbatimModuleSyntax": true
},
"include": ["src/**/*.ts", "test/**/*.ts"]
}

11
tsup.config.ts Normal file
View File

@@ -0,0 +1,11 @@
import { defineConfig } from 'tsup';
export default defineConfig({
entry: ['src/index.ts'],
format: ['esm', 'cjs'],
dts: true,
clean: true,
sourcemap: true,
target: 'es2021',
outExtension: ({ format }) => ({ js: format === 'cjs' ? '.cjs' : '.js' }),
});