anonymizer/CHANGELOG.md

# Changelog

All notable changes to this project are documented here. The format is based on
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.5.0] - Unreleased

### Added

- **Strict anti-leak mode (on by default).** New `strict` option on `Anonymizer`. When enabled, the anonymized
  output is checked after detection to ensure no mapped value still appears as a whole token (ignoring
  placeholders — a placeholder's context field may legitimately echo a value, e.g. `B+` in `[PER_1.SANG:B+]`).
  This catches a model that redacts one mention of a value but leaves another in clear — a case the previous
  bidirectional validation (placeholder ⇄ mapping-key) did not detect. On a suspected leak it throws
  `AnonymizationError` naming only the non-secret placeholder key. Fail-closed: it runs on the final result
  and is **not** swallowed into the regex fallback (which can't fix a name leak). **Defaults to `true`** — set
  `strict: false` to restore the previous behaviour (or if a false positive rejects an otherwise-fine result).
- **`prefilter` option** — decouples the cheap PII pre-filter (skip the LLM when no PII is heuristically
  detected) from the presence of a regex fallback. Default `true`; set `false` to always consult the LLM
  while still keeping the fallback for LLM failures (maximum recall with graceful degradation).

### Changed

- **Boundary-aware value substitution.** Known-value reuse (`applyKnown`, used by `anonymizeChunks` and
  `anonymizeTurn`) now matches values only as whole tokens (Unicode letter/digit boundaries) instead of raw
  substrings, so a short value like `"Ann"` is no longer replaced inside `"Anna"`, and `"jean@exemple.ch"`
  no longer matches inside `"jean@exemple.church"`. Accented and non-Latin names are preserved. The strict
  leak check uses the same boundary logic, so detection and substitution agree.
- `deanonymize` now restores longest placeholder keys first (defensive against prefix overlaps).

## [0.4.0] - Unreleased

### Added

- **Multi-turn conversation support.** New `Anonymizer.anonymizeTurn(text, session?) → { anon, mapping,
legend, session }` keeps one stable id per entity across a whole chat: it seeds each turn with a running,
  serializable `AnonymizerSession` (`{ mapping, legend, history }`), reuses known values via `applyKnown`,
  tells the model which ids are taken, and de-collides new ones. Persist the returned `session` and pass it
  back next turn.
- `Anonymizer.conversation(initial?)` — a stateful in-memory wrapper (`anonymize`, `deanonymize`,
  `session()`) over `anonymizeTurn`.
- Optional `LlmProvider.anonymizeInConversation(text, ctx)` — providers can use prior context (anonymized
  `history`, `legend`, `usedIds`, and optionally `mapping`) for better cross-turn coreference/attribution.
  `openAICompatibleProvider` implements it; providers that don't fall back to the batch path automatically.
- `openAICompatibleProvider` options: `includeMappingInContext` (**default false** — only send real values
  to a _trusted_ anonymizer endpoint) and `historyMaxTurns` via `AnonymizerConfig` (default 10).
- Exported the `AnonymizerSession` type.

## [0.3.1] - Unreleased

### Changed

- Default prompt hardening for lossless round-trips: (1) two **different** values of the same type never
  share a placeholder — they must be distinguished by context (e.g. `[PER_1.IBAN:Ancien]` vs
  `[PER_1.IBAN:Nouveau]`), preventing a within-message collision (old/new IBAN); (2) the model must not
  absorb adjacent **punctuation/separators** (commas, spaces, parentheses) into a placeholder. Together
  these took the live Gemma-4 e-learning demo from 4/5 to 5/5 exact round-trips.

## [0.3.0] - Unreleased

### Changed

- **`openAICompatibleProvider` now works with Infomaniak (and other open-model endpoints) out of the box.**
  `response_format` is **omitted by default** instead of hard-coding `{ type: 'json_object' }`, which
  current Infomaniak rejects (HTTP 422). Pass the new `responseFormat` option (e.g. `{ type: 'json_object' }`
  or a `json_schema` object) for endpoints that support/require it. **Breaking** for endpoints that relied
  on the previous forced `json_object`.
- JSON responses are now **parsed leniently** — a fenced JSON code block or surrounding prose is tolerated
  (the outermost `{ … }` is extracted), so models without an enforced `response_format` don't cause
  spurious parse failures.
- The default prompt gained a **COHÉRENCE** block (exact placeholder↔mapping-key identity, values are the
  original data never another placeholder, strict `[TYPE_N…]` format, mask the value not the adjacent
  label) to improve reliability across models.

## [0.2.0] - Unreleased

### Added

- **`legend`** on every result — abbreviation → French meaning (`PER`→`Personne`, `M`→`Masculin`), safe to
  forward to a downstream LLM so it understands the placeholder tokens. Backed by a built-in
  `DEFAULT_LEGEND` so coverage is guaranteed even if the model omits entries.
- The default prompt now lets the model **coin new uppercase abbreviations** for entities/attributes/
  context it discovers and return their meanings in `legende`.
- `PatternDef.meaning` — optional human label for a tag, surfaced in the `legend`. `presets.swiss`/
  `presets.generic` ship French meanings.
- `AnonymizationError` (exported) — thrown when anonymization can't complete and no fallback exists;
  carries the originating error in `.cause`.

### Changed (breaking)

- **Regex fallback is now opt-in.** `patterns` no longer defaults to `presets.swiss`. With no fallback,
  an LLM failure throws `AnonymizationError` (fail-closed) and the pre-filter is bypassed. At least one of
  `llm` or `patterns` is required, or the constructor throws.
- `AnonymizationResult` gained a required `legend` field; `LlmProvider.anonymizeBatch` returns `legend`.
- `anonymizeChunks(chunks, seed)` — `seed` is now `{ mapping, legend? }` (was the bare mapping) and the
  return includes `legend`.

## [0.1.0] - Unreleased

### Added

- Initial public release.
- `Anonymizer` — pre-filter → LLM → regex fallback, bidirectional validation, deterministic coreference,
  and de-collision across question and retrieved chunks (`anonymize`, `anonymizeChunks`, `deanonymize`).
- `makeStreamDeanonymizer` — streaming-safe de-anonymization that never leaks a split placeholder.
- `openAICompatibleProvider` — pluggable LLM detection over any OpenAI-compatible Chat Completions API.
- `presets.swiss` and `presets.generic` regex pattern sets; fully configurable custom patterns.
- Regex-only mode (no LLM provider required).
- `PatternDef.validate` — optional second-stage predicate to cut false positives; `presets.generic` uses
  it for a Luhn check on credit-card candidates, and de-overlaps its phone/date/IP patterns.
- `openAICompatibleProvider` retries transient failures (network/timeout/429/5xx) via `retries` and
  `retryDelayMs` options; non-transient 4xx and malformed responses are not retried.
- Hardened the `nameHint` heuristic: `g`/`y` flags are stripped internally so `.test()` is stateless.