The same protocol as PII-INDEXBENCH, applied to the most cited public PII corpus and scored per language and per kind of personal data. It also comes with the caveats that follow from a corpus this widely distributed.
Run by Hexagone AI, September 2026. Public dataset, reproducible.
47,566 synthetic emails, forms, chats and JSON payloads in six languages: English, French, German, Spanish, Italian and Dutch. It is the most widely cited public PII corpus, and that popularity cuts both ways, because it has been public long enough that any large model has probably read it during training.
Same protocol as the PII-INDEXBENCH run, same five systems, same scorer, over the whole validation split: 47,566 documents and 293,712 annotated values.
| System | Recallshare of personal data found | Precision | F1 | F1 · exact bounds |
|---|---|---|---|---|
| Hexagone AIapi.hexagone.ai, masking mode | 95.3% | 80.2% | 87.1% | 67.7% |
| OpenMed PIIOpenMed/OpenMed-PII-SuperClinical-Large-434M-v1 | 86.8% | 86.8% | 86.8% | 65.4% |
| OpenAI Privacy Filteropenai/privacy-filter, 1.5B, run locally | 82.9% | 89.5% | 86.1% | 54.0% |
| GLiNERurchade/gliner_multi-v2.1 | 50.8% | 60.1% | 55.1% | 47.9% |
| Microsoft Presidiopresidio-analyzer + spaCy en_core_web_lg | 49.0% | 65.5% | 56.1% | 31.8% |
| Language | Documents | Recallshare of personal data found | Precision |
|---|---|---|---|
| English | 7,942 | 95.9% | 79.9% |
| French | 8,390 | 95.9% | 81.9% |
| German | 8,100 | 95.2% | 78.9% |
| Spanish | 7,803 | 95.6% | 81.1% |
| Italian | 7,878 | 94.6% | 80.1% |
| Dutch | 7,453 | 94.7% | 79.4% |
French comes out ahead of English here, which is what you would expect from a system built in France, and which is the opposite of the usual pattern for tools in this category.
| Kind of personal data | Spans | Hexagone AI | OpenMed | OPF | Presidio | GLiNER |
|---|---|---|---|---|---|---|
| NamesPERSON | 52,298 | 97.8% | 93.7% | 95.2% | 36.3% | 83.4% |
| Street addressesADDRESS | 41,024 | 95.4% | 70.8% | 97.4% | no label | 56.9% |
| PlacesLOCATION | 36,318 | 91.9% | 94.0% | 40.2% | 37.3% | 41.5% |
| Phone numbersPHONE_NUMBER | 12,601 | 97.0% | 85.2% | 99.4% | 64.1% | 60.5% |
| Email addressesEMAIL_ADDRESS | 15,883 | 99.8% | 99.5% | 99.6% | 99.7% | 58.3% |
| Social security numbersSSN | 15,791 | 98.0% | 86.8% | 99.7% | 71.2% | 23.6% |
| Dates and timesDATE_TIME | 49,285 | 91.2% | 92.8% | 53.1% | 50.1% | 62.5% |
| National ID numbersNATIONAL_ID | 16,693 | 96.4% | no label | 95.5% | no label | no label |
| Passport numbersPASSPORT_NUMBER | 15,570 | 96.0% | no label | 95.9% | 46.6% | 29.7% |
| Driving licencesDRIVER_LICENSE | 15,207 | 97.4% | no label | 99.0% | 74.7% | 18.9% |
| IP addressesIP_ADDRESS | 14,110 | 99.0% | 99.4% | 99.1% | 99.6% | no label |
| Passwords, keys and tokensPASSWORD | 10,303 | 90.3% | 74.6% | 97.7% | no label | no label |
Best in each row is bold. « No label » means the system has no category for that kind of data at all, so it is neither scored nor penalised on that row, which is the generous reading of a missing vocabulary. The shading of each cell follows the value in it.
AI4Privacy has been on HuggingFace for years, so any large model has plausibly seen it during training, which flatters anything with a language model inside it, ours included. PII-INDEXBENCH is not distributed there, and where the two sets of numbers disagree we would trust the PII-INDEXBENCH figure.
Precision is 80.2% here against 93.5% on PII-INDEXBENCH, and the gap comes from the annotation scope rather than from anything different in the pipeline. This benchmark labels the values themselves without labelling the fields around them, and it does not label organisations at all.
The corpus is multilingual, and a hand-written list of field labels and salutations in six languages would amount to a classifier tuned until the answer flattered us. Every flagged span is therefore checked against the 210,529 distinct strings this benchmark annotates as personal data somewhere in its 47,566 documents, which is a check anyone can repeat from the published records.
The 94,924 spans that cost us precision
Every prediction on AI4Privacy that touched no annotated span, sorted against the benchmark's own annotations.
Rule. The exact string appears as an annotated value in none of the 47,566 documents.
State, Country, Street, Stadt, Straße, Rue, Calle, Postleitzahl, IP-Adresse. We mask the label as well as the value it introduces, which is over-masking on any reading, even if what it costs is readability rather than privacy. It is also the largest single reason this figure sits below the 93.5% we reach on PII-INDEXBENCH.
- Land: Deutschland - Gebäude: 543 - Straße: Amselreutehof - Stadt: Rosenfeld - Bundesland: BW
- Land: Deutschland - Gebäude: 543 - Straße: Amselreutehof - Stadt: Rosenfeld - Bundesland: BW
Same excerpt on both sides, from AI4Privacy record doc-27758, verbatim.
Rule. Our prediction is typed COMPANY. AI4Privacy annotates twelve kinds of personal data and organisations are not among them.
Every employer, hospital, university and platform we remove counts against us here, because there is no category in which it could have been marked correct: Universidad de Madrid, Université de Genève, Università di Zurigo. Setting those predictions aside lifts precision from 80.2% to 81.7%, and we report the uncorrected figure as the headline.
password reset has been requested for your EduPortal account
password reset has been requested for your EduPortal account
Same excerpt on both sides, from AI4Privacy record doc-11, verbatim.
Rule. The exact string is an annotated value somewhere in the corpus, just not at this occurrence.
Country names (Deutschland, Suisse, España, Nederland), salutations (Cher, Gentile, Herr), initials and bare numbers. The benchmark treats a country as personal data in one document and as ordinary prose in the next, and we have no way of telling the two apart from the text alone. A country alongside a profession and a date also happens to be the combination that narrows a person down, so we would rather remove it than guess.
die Gesundheitsversorgung in Deutschland zu optimieren und die Gesundheit der Bevölkerung nachhaltig zu fördern
die Gesundheitsversorgung in Deutschland zu optimieren und die Gesundheit der Bevölkerung nachhaltig zu fördern
Same excerpt on both sides, from AI4Privacy record doc-25142, verbatim.
A benchmark is someone else's data. One week free, no credit card, and the files never leave your machine, so there is nothing to risk in checking for yourself.