Best PII and PHI Redaction Tools for LLM Gateway Pipelines

A support agent built on retrieval just handed a customer someone else's government-issued identity number, word for word. The number didn't come from what the customer typed. It came from a support ticket an employee had pasted into the internal wiki three years earlier, sitting quietly in the vector store until the retrieval step pulled it up as "relevant context." The input guardrail never saw it, because the input guardrail only watches input. That gap is the whole problem this piece is about: where redaction sits in an LLM pipeline decides whether it actually works.

Most teams building on top of a gateway assume the danger zone is the user's message box. It isn't, or at least it's not the only one. Sensitive data gets into a pipeline through four doors, and guarding one of them while leaving three open isn't a privacy control so much as a false sense of one.

• User input. The one door everyone locks. A customer types their SSN or a phone number into a chat box, and most gateways catch it.
• RAG-retrieved context. Internal tickets, wiki pages, CRM notes, all indexed into a vector store at some point, often without anyone scrubbing them first. Whatever an employee pasted in years ago is now searchable, and retrievable, and quotable by the model.
• Tool results. Agent frameworks call out to customer-lookup APIs, internal databases, MCP servers. Whatever comes back gets fed straight into the model's context window, unfiltered, because nobody wrote a guardrail for tool output.
• Model output. Rarer, but real: the model recombines fragments from context, or in edge cases surfaces something closer to memorized training data.

And "sensitive" is a wider category than most teams plan for. Names and government-issued identity numbers get the attention, but a large share of what employees actually paste into chatbots includes sensitive non-PII material such as API keys, internal code, and proprietary information. None of that looks like PII on paper. All of it is a liability once it's sitting in a third-party model's context window.

The scale of this is bigger than most privacy programs account for. Cyberhaven's AI Adoption and Risk Report found that 39.7% of all enterprise AI interactions involve sensitive data. 58.2% of Claude interactions and 32.3% of ChatGPT interactions happen through personal accounts that never touch corporate controls. A gateway sitting in front of sanctioned enterprise traffic can't do anything about the other path. It just means the redaction strategy for the traffic you do control needs to actually work, because it's covering less ground than it looks like.

Why the location of redaction is a compliance boundary, not a technical preference

A provider stripping PII out of a response after the fact is not the same as PII never reaching that provider. Only the second one keeps the data inside your own network, and only the second one gives you a compliance boundary you can actually defend, and only the second one gives you a compliance boundary you can actually defend. This isn't a technical nuance. Either you controlled this, or you hoped the vendor controlled this."

Regulators treat the two paths very differently, and the differences aren't small:

Under HIPAA, sending PHI to a hosted model without a signed vendor data-handling contract counts as an impermissible disclosure, and it's presumed to be a reportable breach unless a four-factor risk assessment can document a low probability of compromise. That's a heavy burden to carry after the fact. The redaction logic needs to run inside your own VPC, before the data ever leaves.

GDPR treats any personal data sent to an LLM as processing, full stop. Cumulative fines under a major data protection law have topped €5.88 billion since 2018, and regulators have consistently held that pseudonymized data still counts as personal data if it can be re-linked to an individual. Swapping a name for a token doesn't get you out of scope if the mapping still exists somewhere.

PCI DSS works the same way. Cardholder data reaching any external service pulls that service into your PCI assessment scope, whether or not the service ever intended to touch payment data.

Some teams try to route around this with cloud DLP, sending traffic through AWS Comprehend or Google DLP to sanitize it before it hits the LLM. That pattern doesn't solve the problem, it just moves it: the unredacted regulated data has already crossed an external trust boundary by the time DLP touches it. The sanitization happens one step too late to matter. Provider-side filters run into the identical wall. A service like Nightfall has to receive unredacted PHI or special-category data on its own infrastructure before it can detect and redact anything, and contractual terms don't fully close that gap, per predictionguard.com. The data transited somewhere it shouldn't have, contract or no contract.

Retention policy makes this concrete. OpenAI's API retains data for 30 days by default for abuse monitoring. Zero-data-retention exists, but it requires prior approval through OpenAI's sales team for eligible customers, and it's not available on pay-as-you-go plans. Anthropic reduced its standard API log retention from 30 days to 7 days in September 2025. So the honest thing to ask about any tool under evaluation is simple: does redaction happen before the data crosses the boundary, or after? That answer is the first filter, before latency, before accuracy, before price.

Detection methods and their accuracy-latency tradeoffs

Not all PII looks the same to a machine, and the method that finds a credit card number will never find a person's name. There are three rough tiers of difficulty here, and each one demands a different tool:

Structured PII, things like government-issued identity numbers, credit cards, emails, IBANs, and phone numbers, follow a predictable format. Regex with checksum validation catches these in sub-millisecond time with high precision, as long as the input is well formatted. What it can't do is recognize a name. There's no regex for "is this a person."

Semi-structured data, like API keys and access tokens, needs entropy analysis layered on top of pattern matching, since the format alone isn't distinctive enough.

Unstructured PII, names, addresses, clinical terms, needs actual language understanding: named entity recognition (NER) models or an LLM acting as a judge. There's no shortcut here, and no format to match against.

The production stack that's shaped up by 2026 reflects that split: regex and checksum first, then a NER model, then an LLM-judge as the final check. Each tier costs more than the last, so teams only add a layer where the compliance requirement justifies the expense.

Latency is where the tradeoffs get uncomfortable. Presidio deployed as a gateway sidecar adds 150 to 400 milliseconds of p99 latency, which is a real tax on a synchronous request. A fine-tuned Mistral 7B model hit 0.876 accuracy on noisy text in one arxiv.org study, well ahead of T5's 0.788, but it took 15.6 seconds per inference against T5's 1.46 seconds. That's not a tool you put in the critical path of a live chat response. On the other end, TrueFoundry's own guardrails benchmark reports around 10 milliseconds of latency at 350-plus requests per second on a single vCPU, though that figure is vendor-reported and worth testing against your own traffic before trusting it.

Redaction, masking, tokenization, and anonymization get used interchangeably in casual conversation, and that's a mistake, because they're not the same operation and they carry different regulatory weight:

• Redaction replaces the value with something like [REMOVED]. No mapping survives. Use it when the downstream task never needs the real value back.
• Masking (j***@example.com) keeps the original somewhere else, and works fine for UI display where a human just needs to recognize the field, not read it.
• Tokenization, sometimes called pseudonymization, swaps the value for something like PERSON_001 and keeps a separate key to reverse it. This is required whenever a pipeline needs to track the same entity across multiple turns or steps without exposing the real value at every hop.
• True anonymization is the hardest to actually achieve. It requires a real re-identification risk assessment, because a rare job title, an exact birth date, and a small town can identify a specific person even after every obvious string has been removed.

Minimum entity coverage for 2026 is around ten baseline classes: government ID, email, phone, postal address, full name, credit card, IBAN, IP address, generic ID, and date of birth, according to futureagi.com. GDPR's Article 4(1) definition of personal data, CCPA, and HIPAA's 18 PHI categories all stack additional requirements on top of that baseline. No single detection method hits high precision and high recall across every one of those categories at once. That's not a gap to be embarrassed about, it's just the physics of the problem, and it's why every serious production deployment ends up layering multiple tools instead of picking one and calling it done.

Clinical PHI benchmark results and what they reveal about tool selection for healthcare

Clinical text is where these tradeoffs stop being theoretical. John Snow Labs ran a 2026 benchmark against a public, expert-annotated set of 1,479 PHI chunks pulled from 48 clinical documents, with the scoring code published alongside the results. The spread between tools is not subtle:

• John Snow Labs Healthcare NLP: 0.96 PHI F1, with 0.98 micro F1 on the official i2b2 2014 test set.
• Prompted frontier models in the GPT-4 class: 0.86 to 0.91.
• Databricks' ai_mask(): 0.71.
• Microsoft Presidio: 0.60 to 0.85, a wide range driven by how sensitive it is to dataset, domain, and configuration.
• OpenAI Privacy Filter: 0.55, the lowest score in the benchmark.

These numbers are dataset-specific, not universal laws. TrueFoundry's own writeup on the benchmark cautions that any team relying on these figures should re-run the comparison against its own traffic before setting a threshold. A tool that scores well on a public i2b2 test set can behave differently against a hospital's actual intake notes.

Still, a 40-point F1 gap between the top and bottom performer isn't a rounding error, it's a different category of risk. A missed PHI entity in a supposedly de-identified clinical record isn't a bad metric, it's a regulatory incident with a patient's name attached to it. That distinction should change how healthcare teams weigh accuracy against everything else.

The uncomfortable tradeoff is that the most accurate tool here is also the most specialized and the heaviest to operate. The line between "good enough for general SaaS redaction" and "good enough for clinical notes" turns out to be wide enough that it deserves its own decision, made separately from whatever tool handles PII in a customer support chatbot.

Microsoft Presidio

Presidio is an open-source Python framework for detecting and redacting PII, built originally at Microsoft and, as of 2026, in the process of moving to the independent Data Privacy Stack organization. The license stays MIT, the APIs stay the same, and container images are relocating to the new organization's registry.

Detection runs through custom recognizers that combine regex, deny lists, checksums, rules, and NER models. The operators are extensible, so teams can fine-tune models against their own domain vocabulary instead of relying on a generic one.

On the John Snow Labs clinical benchmark, Presidio scored 0.60 to 0.85 PHI F1, and that range says almost as much as either endpoint: performance depends heavily on how it's configured and what domain it's pointed at.

On the gateway integration side, Presidio shows up as the detection engine behind more than one commercial guardrails product, and it can also be wired in as an external sidecar sitting alongside other gateway plugins on the same AI route. But when deployed that way, it adds 150 to 400 milliseconds of p99 latency, the highest figure in this comparison. That makes it a poor fit for a synchronous, inline guardrail at real scale unless the architecture around it is built carefully, with async processing or caching absorbing some of that cost.

Presidio makes sense when a team needs deep NLP customization and has the in-house capability to run and calibrate the underlying models, and when either latency tolerance is generous or the workload can run asynchronously. It's the wrong pick for teams that need guaranteed low-latency inline redaction, or that don't have the Python ML operations bench to keep NER models tuned. On whether redaction happens inside the data boundary, though, it clears the bar cleanly: redaction runs on infrastructure you control, and nothing leaves the perimeter to get scrubbed.

LLM Guard

LLM Guard is an open-source Python toolkit built around 35+ scanners, covering PII, toxicity, bias, code detection, and general output quality. Each scanner is its own model or analysis step, run independently and toggled on or off as needed.

The usage pattern is a chain: prompts and outputs pass through a sequence of scanners, each doing one job. That modularity is the appeal, and also the latency cost. Model-based scanners add anywhere from 100 milliseconds to 5 seconds each, and stacking five or more scanners can push total latency into multi-second territory, which rules out synchronous inline use unless the scanner list is trimmed hard.

LLM Guard supports reversible redaction through its Anonymize scanner, which tokenizes PII into a Vault, paired with a Deanonymize scanner that reverses the process on the way out. That's a genuinely useful pattern for pipelines that need entity continuity without exposing raw values mid-flight.

It's Python-only. GPU compute is recommended for the model-based scanners but not strictly required, since CPU inference is supported through ONNX Runtime. Some sources note the project has seen fewer updates in recent periods than in its earlier years, which matters for any team planning a long-term production dependency. Pricing is straightforward: the software is free and open source, so the real cost appears in GPU infrastructure, not licensing.

Data stays on infrastructure you control. LLM Guard fits best for teams that want broad coverage across input and output safety, not just PII but toxicity, code leakage, and bias too, and that can tolerate async or batch processing. It's not the right primary tool if low-latency inline PII redaction is the main requirement.

OpenAI Privacy Filter

Released in April 2026 under an Apache 2.0 license, OpenAI's Privacy Filter is a local, bidirectional token classifier: 1.5 billion total parameters, only 50 million of them active at inference time, with a 128,000-token context window.

It covers eight fixed categories: private persons, addresses, emails, phones, URLs, dates, account numbers, and secrets. That's a tight, opinionated list, and it doesn't flex.

On the John Snow Labs clinical benchmark, it scored 0.55 PHI F1, the lowest of every tool tested. The model card itself flags the relevant caveats: the taxonomy is fixed at those eight categories, performance can degrade on non-English or otherwise out-of-distribution text, and any policy change down the road may require fine-tuning rather than a config toggle.

On the data boundary, it runs locally, so nothing leaves the perimeter. That makes it a reasonable choice for teams needing a fast, local filter over general-purpose English text where the sensitive categories are secrets and common PII types, and where HIPAA or clinical PHI obligations don't apply. It's the wrong tool once clinical accuracy matters, once multilingual coverage is a requirement, or once the eight fixed categories don't cover what the business actually needs flagged.

John Snow Labs Healthcare NLP

This one is purpose-built for healthcare from the ground up, with de-identification models trained specifically on clinical language rather than general web text.

The numbers back that specialization up. On the John Snow Labs 2026 benchmark, it posted 0.96 PHI F1 on expert-annotated clinical notes, 0.98 micro F1 on the official i2b2 2014 test set, and 0.98 recall measured at a scale of 381,959 tokens. That's not a marginal edge, it's a 40-point F1 gap over OpenAI's Privacy Filter and a gap of 10 to 35 points over Presidio, depending on how Presidio is configured.

The gap exists because the model was trained on the actual language clinicians write in. Clinical notes have their own shorthand, abbreviations, and structure, and a model trained on general text simply doesn't have exposure to that pattern.

The tradeoff is operational weight. This isn't a lightweight sidecar a team drops into an existing pipeline over an afternoon. It's a fuller platform, heavier to run and maintain than Presidio or LLM Guard, and that footprint needs to be budgeted for, not treated as an afterthought.

It fits when the pipeline actually touches clinical notes, medical records, or anything falling under HIPAA's 18 PHI categories, where a missed entity isn't a bug ticket but a compliance incident. It doesn't fit general-purpose SaaS PII use cases, or teams working under tight latency or infrastructure constraints that rule out standing up a specialized NLP platform.

Philter (Philterd)

Philter is a self-hosted redaction engine exposed through an HTTP API, and it's part of a broader suite under the Philterd name. The suite includes Philter itself as the core engine, PhEye (trained AI and NLP models that locate PII and PHI in text), Arbiter (a human-in-the-loop review layer), Philter Scope (for evaluating model performance), and Philter AI Proxy, which sits inline in front of OpenAI or Anthropic traffic.

That proxy is the piece most relevant to gateway architecture. It sits between the client and the AI gateway, and pointing traffic at it is a one-line config change on the client side. A guide published on philterd.ai in August 2026 walks through where the proxy belongs relative to an AI gateway and how it handles streaming responses, which is a detail that trips up a lot of naive redaction implementations, since streaming tokens arrive incrementally and can't just be scanned as one finished block.

The redaction model itself follows a policy-as-code approach: redaction rules live as versioned, tested files rather than settings buried in a console somewhere, according to philterd.ai's documentation. That's a meaningful difference for audit purposes. A policy file can be reviewed in a pull request, tested against sample data, and rolled back like any other piece of code, instead of living as a setting someone changed six months ago with no record of why.

Published deployment guides cover contexts most vendors don't bother writing about, including a walkthrough for standing up Philter, PhEye, and MongoDB inside an air-gapped government enclave, with pinned container images and no outbound network access. That's a narrow but real use case: environments where "the data never leaves" isn't a preference, it's a hard requirement enforced by the network itself.