Perplexity publishes PII-TRACE for testing PII detectors in long AI chats

Quick answer

Perplexity has introduced PII-TRACE, a benchmark built from 13,148 synthetic user-assistant conversations across 13 languages, and PII-Tracer, a 0.6B detector for identifying recurring personal data before text reaches a cloud model. In Perplexity's 12-system evaluation, PII-Tracer recorded 0.629 character F1 and found every mention of 79.4% of recurring identifiers and 77.6% of identifiers repeated across turns. The same report shows that recall falls on very long conversations unless the input is processed with overlapping windows. A public MIT-licensed Hugging Face repository named pplx-pii-masking exposes a closely related 0.6B architecture, but Perplexity's currently visible research post does not explicitly identify that repository as the PII-Tracer release. Treat the results as company-reported evidence, test the exact artifact you deploy, and keep additional privacy controls in place.

Download Chat AI Opens the official App Store or Google Play for your device.

PII-TRACE measures whether a detector finds every repeat of personal data

Perplexity says PII-TRACE contains 13,148 synthetic user-assistant conversations across 13 languages and 10 writing systems, with 37,431 character-level identifier mentions spanning nine PII types. Of the 5,645 conversations containing labeled PII, 63.8% repeat at least one identifier and 28.7% repeat an identifier across turns. The benchmark's stricter consistency measure counts an identifier only when the detector covers every character in every mention, reflecting the risk that one missed repeat can still expose personal data.

Sources: Perplexity

Perplexity reports PII-Tracer led two conversation-level measures

Across the 12 systems Perplexity evaluated, PII-Tracer recorded the highest character F1 at 0.629. It also found every mention of 79.4% of recurring identifiers and 77.6% of identifiers recurring across turns; the report gives GPT-5.6-sol 57.0% and 55.1% on those two consistency measures. GPT-5.6-sol scored higher on both span-level F1 measures, so the reported lead is specific to character F1 and recurring-identifier consistency rather than every metric. These are Perplexity's evaluations, not independent results.

Sources: Perplexity

Long conversations need overlapping windows

With one 4,096-token window, Perplexity reports character recall of 0.975 for conversations under 1,000 characters, 0.955 from 1,000 to 10,000 characters, and 0.687 at 10,000 characters or longer. Reprocessing the same checkpoint with 50%-overlap sliding windows raised overall character recall from 0.830 to 0.965 and recurring-identifier consistency from 0.794 to 0.954 in the company's test. That result makes input handling part of the safety design: applications should preserve overlap, reconcile duplicate spans, and test boundary cases rather than silently truncating long chats.

Sources: Perplexity

PII-Tracer and pplx-pii-masking are not explicitly linked in the visible sources

Perplexity describes PII-Tracer as a 0.6B bidirectional Qwen3 encoder with a 37-label BIOES token head, nine PII types, a sensitivity head, and a 4,096-token window. The public pplx-pii-masking model card describes the same high-level architecture and provides an MIT-licensed checkpoint plus reference code. However, the research announcement says Perplexity plans to release PII-TRACE and PII-Tracer soon and does not explicitly point to the pplx-pii-masking repository. Until Perplexity publishes a direct mapping, benchmark results for PII-Tracer should not automatically be treated as verified results for every file in that repository.

Sources: Perplexity, Perplexity AI on Hugging Face, Perplexity AI on Hugging Face, Perplexity AI on Hugging Face

Deployment still requires artifact review and representative tests

The pplx-pii-masking reference implementation asks Transformers to load its backbone with trust_remote_code enabled. Before processing sensitive data, review that code, pin an approved revision, isolate execution, and control outbound network access. Build a test set that represents the languages, identifier formats, transcription errors, structured content, and message lengths in the real workflow. Compare misses and over-masking against deterministic checks, then define whether high-sensitivity or uncertain inputs stay local, are redacted, require consent, or are refused.

Sources: Perplexity AI on Hugging Face, Perplexity AI on Hugging Face, Perplexity

PII detection remains a filter, not a privacy guarantee

PII-TRACE uses synthetic conversations derived through a privacy-preserving rewriting pipeline, and only one of the five external benchmark sets in Perplexity's comparison uses real, human-labeled text. The reported scores also show false negatives and false positives, especially as context grows. Production systems should combine detection with data minimization, access controls, deterministic rules where appropriate, safe logs, and human review for high-risk uses. A benchmark score does not establish de-identification, regulatory compliance, or that no combination of remaining details can identify a person.

Sources: Perplexity

The exact model is not verified in Chat AI

Chat AI's current model directory does not list the exact pplx-pii-masking checkpoint, so its availability in Chat AI is unverified. The Hugging Face release is useful to teams building their own preprocessing or privacy controls, but it should not be interpreted as a Chat AI product integration or availability announcement.

Sources: Chat AI, Perplexity AI on Hugging Face

Frequently asked questions

What readers usually ask

What is Perplexity's PII-TRACE benchmark?

It is a test set of 13,148 synthetic user-assistant conversations across 13 languages designed to measure PII detection, including whether a detector finds every repeat of an identifier across a long conversation.

How did PII-Tracer perform in Perplexity's evaluation?

Perplexity reports 0.629 character F1, the highest among the 12 evaluated systems, plus complete coverage of 79.4% of recurring identifiers and 77.6% of identifiers repeated across turns. GPT-5.6-sol scored higher on the two span-level F1 measures.

Does PII-Tracer handle very long conversations?

Its window is 4,096 tokens. Perplexity reports that recall fell for conversations of at least 10,000 characters with a single window, while 50%-overlap sliding windows substantially improved recall in its test.

Is the pplx-pii-masking repository definitely the PII-Tracer checkpoint?

The public repository describes the same high-level 0.6B bidirectional Qwen3 architecture, but Perplexity's visible research announcement does not explicitly link the two names. Treat them as related until Perplexity publishes a direct mapping.

Does PII detection guarantee that personal data stays private?

No. A detector can miss identifiers, over-mask harmless text, or leave combinations of details that still identify someone. It should be one control within a broader privacy and security process.

Is pplx-pii-masking available in Chat AI?

The exact checkpoint is not listed in Chat AI's current model directory, so availability is unverified. The release should not be treated as a Chat AI integration announcement.

Evidence