Docs
PII redaction
This is an architecture diagram of how Foam would use NER behind the scenes.
PII redaction is not supported yet. It is optional and off by default — for support, contact perla@foam.ai to enable it for your telemetry.
Data flow
Redaction is opt-in per environment and per service, and fail-closed: if a value can’t be analyzed, it is masked as [REDACTED] rather than stored raw. Nothing is ever dropped.
How detection works
The collector runs Microsoft Presidio as a sidecar. Two mechanisms work together: deterministic pattern recognizers with checksums (emails, phone numbers, credit cards, SSNs, IBANs, IPs) and statistical NER (spaCy en_core_web_lg) for names. Domain identifiers like medical record numbers are covered by configurable patterns that only trigger near context words — MRN 4829102 is redacted, order 4829102 is not. Redacted items carry a foam.presidio.redactions marker for auditing.
Benchmarks
Measured on our own telemetry (500 adjudicated documents) and a 14-language test round:
- Latency: 12 / 81 / 92 ms (p50 / p95 / p99) per document; ~0 ms for unscanned traffic.
- Pattern entities: IPs 100%, phones 97.4% recall; emails, cards, and IBANs redact across all 14 tested languages, including right-to-left scripts, with zero text corruption.
- Names: 33.5% recall on production-domain text — the NER model is English-trained. Its published reference is ~0.85 F1 on OntoNotes 5 English news (see spaCy’s published benchmarks and the en_core_web_lg model card); the gap is domain and language shift, which is why we benchmark on our own data using the methodology from Presidio’s evaluation framework (presidio-research).