Dated, citable PII-accuracy stats for NeutralAI.
These are the exact numbers NeutralAI publishes for PII detection accuracy: what they measure, how they were produced, and how they compare to a Presidio-vanilla baseline. Vendor-published, not independently audited — methodology is documented below.
96.0%
PERSON-entity holdout F1
For the PERSON entity type specifically, NeutralAI measured a 96.0% F1 score on holdout data, last verified July 2026.
16 vs 3
entity families attempted (NeutralAI vs vanilla Presidio)
Of the 16 entity families in the NeutralAI benchmark, an uncalibrated vanilla Microsoft Presidio baseline is configured to attempt 3; it does not look for the other 13, including the 10 UK identity, financial and legal identifiers, last verified July 2026.
98.5% vs 74.1%
holdout F1 on the families both engines attempt
Restricted to the entity families both engines attempt (EMAIL_ADDRESS, PERSON, PHONE_NUMBER), NeutralAI scored 98.5% F1 on holdout data against 74.1% for a vanilla Presidio baseline, last verified July 2026.
14 vs 378
false positives on holdout (NeutralAI vs vanilla Presidio)
On the holdout set, NeutralAI produced 14 false positives against 378 for a vanilla Presidio baseline, whose PERSON detection ran at 34.2% precision, last verified July 2026.
Methodology
How the numbers were produced.
Dataset
A labeled benchmark set of prompt-style text. The published comparison run covers 1,587 cases across DE, EN, ES, FR, and TR (generated 2026-07-22), scored against 16 entity families — names, contacts, financial and account identifiers, and the UK identity, financial and legal pack.
Holdout discipline
“Holdout” means a sample kept separate from the data used to tune detectors and confidence thresholds. We lead with PERSON-holdout F1 (96.0%) and the like-for-like comparison, not a pooled overall score. A pooled figure across all 16 families is dominated by the pattern-matched identifiers, which sit at or near 1.0, so it rises whenever a new one is added — it moved from 98.4% to 99.9% when the UK pack was added, with no detection change. The tuned public set is at ceiling for the same reason: useful as a regression gate, worthless as a forecast.
Reproducibility
The methodology is described here: same test set and scorer used for both NeutralAI and the vanilla Presidio baseline, with dataset facts (case count, languages, generation date) published on this page. The labeled dataset and full scoring harness are not public today, so an external team cannot independently rerun this exact comparison. Detailed methodology is available on request through security or partnership review.
Per-entity results
What granularity actually exists today.
NeutralAI currently publishes overall and PERSON-specific F1 scores. It does not yet publish a full per-entity-type accuracy breakdown (email, phone, card, IBAN, UK NHS number, etc.) — the table below is honest about that gap rather than inventing precision that has not been measured yet.
| Entity / scope | Metric | Score |
|---|---|---|
| PERSON | Holdout F1 | 96.0% |
| PERSON | Holdout F1 — vanilla Presidio baseline | 49.4% |
| Both engines attempt these (EMAIL, PHONE, PERSON) | Holdout F1 | 98.5% |
| Both engines attempt these (EMAIL, PHONE, PERSON) | Holdout F1 — vanilla Presidio baseline | 74.1% |
| UK identity, financial and legal pack | Entity families measured, each CI-gated at a 0.95 F1 floor (2026-07-22) | 10 |
| UK NHS number | Supported entity type — no published per-entity F1 yet |
— = not currently published as a standalone score. See how entity coverage and confidence thresholds work for the supported entity list.
These were listed here as roadmap items without figures. As of 2026-07-22 all 10 are in the measured set, each held to a 0.95 F1 floor that fails the build on a regression. A vanilla Presidio baseline does not attempt any of them.
All 10 are also covered by the holdout split — the sample kept out of detector tuning — written as ordinary correspondence rather than templates, so the cue sits inside a real sentence instead of at the start of a fixed pattern.
- National Insurance number
- NHS number
- Sort code
- Bank account number
- HMRC UTR
- Companies House number
- Court case number
- SRA ID
- Land Registry title number
- Driving licence number (DVLA and DVA formats)
Scores are measured on synthetic, well-formed prompts, so treat them as a regression gate rather than a forecast of real-world accuracy: their job is to prove that what worked last release still works. UK postcodes are deliberately excluded — a postcode is often a legitimate search term rather than personal data, so it ships as an opt-in policy template instead of a default.
Comparison
NeutralAI vs Presidio vs OpenAI Privacy Filter.
Same benchmark set and scorer for the two measured rows. The OpenAI Privacy Filter row is left explicitly pending — no invented score, no placeholder zero.
FAQ
Honest answers, no spin.
How is it measured?
The headline numbers are F1 scores computed against a labeled benchmark set of prompt-style text, comparing detected PII against ground-truth annotations. The published comparison set covers 1,587 cases across German, English, Spanish, French, and Turkish. Holdout F1 is measured on a sample not used for tuning the detectors, so it is the figure we lead with. We do not publish a single pooled "NeutralAI vs Presidio" score: a baseline is not configured to attempt most of these entity families, so pooling coverage and accuracy into one number would widen the gap every time we added an entity type without anything getting more accurate. Coverage and shared-entity accuracy are reported separately instead.
Is this independently validated?
No. This is a vendor-published product benchmark, not an independent or third-party audit. NeutralAI generated and scored the benchmark itself. The methodology is documented and the comparison against a vanilla Presidio baseline uses the same test set and scorer so the delta is at least internally consistent.
Can I reproduce it?
Not fully, not yet. The methodology is described on this page, and dataset facts (case count, language split, generation date) are published, but the labeled dataset and full scoring harness are not public — so an external team cannot independently rerun this exact comparison today. Detailed methodology is available on request through a security or partnership review.
How often is this updated?
This page carries a "Last verified" dateline and a changelog. Numbers are refreshed when the underlying gateway benchmark is re-run against a new model or detector release, not on a fixed calendar schedule.
Changelog
Page history.
- July 2026— Initial publication of the standalone benchmark page. UK legal entity pack listed as in development, without figures.
- May 2026— Presidio-vanilla comparison benchmark generated (2026-05-08, 1,000 cases across DE, EN, ES, FR, TR).
- July 2026— UK identity, financial and legal pack added to the measured set: 10 entity families, each held to a 0.95 F1 floor that fails the build on a regression. Benchmark regenerated (2026-07-22, 1,587 cases).
- July 2026— Comparison reporting corrected. A single pooled "NeutralAI vs Presidio" F1 conflated coverage with accuracy: the baseline scores a structural zero on every family it is not configured to attempt, so the gap grew each time we added an entity type. Coverage and shared-entity accuracy are now reported separately, and every figure on this page is generated from the benchmark artifacts rather than typed in.
Try the masking flow yourself.
These numbers describe detection accuracy. The fastest way to see the product is to run a prompt through it.