Clinical Audio Still Carries Identifiers After Text De-Identification
Clinical audio remains a privacy problem even after transcripts are cleaned. Spoken encounters can preserve names, places, dates, and other sensitive health information in the original waveform, creating a gap between text-based de-identification and real-world reuse of recorded visits, dictations, and care-team exchanges.
This preprint addresses that gap by establishing the SREDH-AICUP 2025 benchmark for recording-level sensitive health information detection. The goal is not simply to transcribe speech, but to determine whether a system can identify privacy-relevant content directly from audio or from transcript-derived pipelines well enough to support safer secondary use of clinical recordings.
Pipeline Systems Outperformed End-to-End Audio Models
Twenty-two qualifying systems entered the benchmark, and every one of them used automatic speech recognition followed by downstream sensitive health information detection. The study also evaluated an end-to-end approach that attempted to detect sensitive content directly from audio.
The performance gap was immediate. Pipeline baselines produced stronger transcription performance, with MER ranging from 0.123 to 0.155 compared with 0.155 for the end-to-end baseline. The same pattern appeared in privacy detection: pipeline systems reached macro-F1 values between 0.507 and 0.653, while the end-to-end baseline reached 0.422.
A simple operational lesson follows from those numbers: when clinical audio must be screened for privacy risk, the transcription stage still carries major weight.
The finding also reflects a familiar technical reality in speech medicine. De-identification depends on how well a system hears the words before it reasons about the words. If recognition misses a name, location, or other identifier, downstream filtering cannot recover what was never captured. In a setting where the audio itself is the protected object, transcription quality is not a side metric; it is part of the privacy barrier.
Mandarin Augmentation Changed What the Model Could Hear
Language coverage mattered in a very specific way. Synthetic Mandarin speech-text augmentation enabled the end-to-end model to detect Mandarin sensitive health information that it failed to identify without augmentation, moving macro-F1 from 0 to 0.171.
That shift is clinically relevant because multilingual clinical environments rarely behave like monolingual benchmarks. Acoustic cues, pronunciation patterns, and text normalization all shape whether protected information is recognized. When a model has not been exposed to enough language-specific examples, privacy leakage can survive even if the architecture appears robust on paper. Augmentation did not make the task easy, but it created a detectable path for the model to learn Mandarin-specific sensitive content rather than leaving it blind to the category altogether.
Foundation Models Helped Some Categories More Than Others
The benchmark also examined foundation models, and their performance was not uniform across sensitive information categories. Some categories were easier for the models to recognize than others, which suggests that privacy detection in speech is not a single-task problem with a single bottleneck.
Category-specific behavior matters because clinical audio does not contain one type of identifier. A system may be competent at recognizing one class of sensitive information while missing another, and those failures can have different downstream consequences depending on the use case. In a hospital transcription workflow, missing a patient name has a different privacy profile than missing a location, medication clue, or other embedded identifier. The benchmark’s category-level variation makes clear that broad overall scores can hide uneven protection.
Adaptive Routing Improved the Final Detection Score
A type-aware adaptive routing strategy improved macro-F1 from 0.656 to 0.697. The gain was modest in absolute terms, but in a privacy-sensitive task, even incremental improvement can matter when the objective is to reduce residual exposure in recordings that may be reused for research, quality improvement, or model development.
Adaptive routing is appealing because it treats detection as a matching problem rather than a single generalized decision. Different input types can be directed toward components better suited to handle them, which is especially useful when speech data includes uneven language coverage, different acoustic quality, and multiple categories of sensitive content. In practice, that sort of system design may reduce the mismatch between what the model sees and what the privacy policy requires.
Standardized Reference Performance Now Exists
The benchmark establishes a reference point for recording-level clinical speech de-identification. That matters because privacy-preserving audio research has often advanced with limited comparability across datasets, tasks, and model pipelines. A shared evaluation structure gives developers a way to judge whether a new system is truly better or merely tuned to a narrow dataset.
The work also frames the field in a more operational way. Hospitals and research teams do not just need transcription accuracy; they need confidence that spoken sensitive information can be found before recordings are stored, shared, or analyzed. A transcript-only approach may be sufficient for some workflows, but the benchmark shows why direct audio-level evaluation is still necessary when the source recording itself remains accessible.
The most important implication is practical: privacy protection in clinical speech cannot rely on text cleaning alone. Spoken identifiers survive the recording even when the transcript looks safe, and the benchmark provides a structured way to measure how well current systems confront that problem.
What the Benchmark Still Leaves Unresolved
Several questions remain open. The benchmark shows that pipeline systems currently outperform the end-to-end baseline, but it does not settle which deployment model will be best in routine clinical settings, where latency, cost, language diversity, and error tolerance all shape implementation. The improvement from augmentation and adaptive routing is encouraging, yet neither result removes the need for broader testing across more languages, more recording conditions, and more sensitive information categories.
The current evidence also comes from a preprint, so the findings should be treated as an emerging reference rather than a finalized standard. Even so, the data already make one point hard to ignore: clinical audio privacy requires measurement at the recording level, not just after transcription, if healthcare systems want trustworthy reuse of speech data.