In the Name of Fairness:
Assessing the Bias in Clinical Record De-identification

Abstract.

Data sharing is crucial for open science and reproducible research, but the legal sharing of clinical data requires the removal of protected health information from electronic health records. This process, known as de-identification, is often achieved through the use of machine learning algorithms by many commercial and open-source systems. While these systems have shown compelling results on average, the variation in their performance across different demographic groups has not been thoroughly examined. In this work, we investigate the bias of de-identification systems on names in clinical notes via a large-scale empirical analysis. To achieve this, we create 16 name sets that vary along four demographic dimensions: gender, race, name popularity, and the decade of popularity. We insert these names into 100 manually curated clinical templates and evaluate the performance of nine public and private de-identification methods. Our findings reveal that there are statistically significant performance gaps along a majority of the demographic dimensions in most methods. We further illustrate that de-identification quality is affected by polysemy in names, gender context, and clinical note characteristics. To mitigate the identified gaps, we propose a simple and method-agnostic solution by fine-tuning de-identification methods with clinical context and diverse names. Overall, it is imperative to address the bias in existing methods immediately so that downstream stakeholders can build high-quality systems to serve all demographic parties fairly.

1. Introduction

The increased availability of clinical datasets (Johnson et al., 2023, 2016; Fleurence et al., 2014) plays a significant role in the recent advancements in machine learning (ML)-aided healthcare systems (Shailaja et al., 2018; Ahmad et al., 2018; Chen et al., 2021; Qayyum et al., 2020). In order to share clinical trial data legally, stakeholders must adhere to the Health Insurance Portability and Accountability Act (HIPAA) Safe Harbor provisions by masking 18 types of protected health information (PHI). If done appropriately, clinical data sharing adds significant value to scientific reproducibility (McDermott et al., 2021) at low risk to patient privacy (Seastedt et al., 2022; Lehman et al., 2021). In this regard, various open-source software (Kayaalp, 2017; Meystre et al., 2010) and commercial companies provide services to de-identify electronic health records (EHRs). Named entity recognition (NER) tools (Li et al., 2020; Song et al., 2021; Yadav and Bethard, 2018) in natural language processing (NLP) libraries (Honnibal et al., 2020; Qi et al., 2020; Akbik et al., 2019; Manning et al., 2014) are commonly used in this space.

Despite the compelling performance of many ML-empowered healthcare systems (Topol, 2019), models have been shown to underperform in minorities and minoritized populations, and naive applications can extend and increase existing biases (Williams and Wyatt, 2015; Ghassemi et al., 2020; Chen et al., 2021; Seyyed-Kalantari et al., 2021; Ghassemi and Nsoesie, 2022; Ghassemi et al., 2020; Webster et al., 2022). Disparities in performance between demographic groups can lead to real harm (Zestcott et al., 2016; Marcelin et al., 2019; Hall et al., 2015). For instance, a state-of-the-art early warning model for acute kidney injury (Tomašev et al., 2019) failed to extend to female patients due to its male-dominated training data (Cao et al., 2022). In de-identification specifically, failing to remove the PHI of certain demographic groups would violate the Safe Harbor regulations. This failure could exacerbate the known misuse of data from minorities (Browne, 2015; Fagan et al., 2016; Ghassemi and Mohamed, 2022) and expose these groups to targeted attacks such as identity theft (Anderson, 2005, 2006).

Figure 1. Workflow of our empirical study. We identify (a) four demographic dimensions and prepare (b) 16 name sets with diverse settings. For each name set, we duplicate each of the (c) 100 clinical templates ten times and populate the copies with randomly generated names. We then use these (d) 16,000 evaluation notes to assess (e) nine de-identification methods.

In this paper, we audit the performance of de-identification methods on a specific PHI category—names—from clinical notes. We focus on names because they are correlated with demographic features and are disproportionately identifiable amongst the defined PHI categories. To date, existing studies (Mansfield et al., 2022; Mehrabi et al., 2020; Mishra et al., 2020) have compared a limited number of baselines on short sentence templates that are much simpler than the real clinical notes. In contrast, we conduct a large-scale empirical evaluation of nine commercial and open-source de-identification methods based on 16 name sets that vary along four demographic dimensions—gender, race, name popularity, and the decade of popularity—and 100 note templates (Lim et al., 2023) curated from real-world clinical records. We adopt the gender and racial categories in the U.S. Social Security (ssa, nd) and Census (cen, nd) datasets and calculate popularity from name frequency over three selected decades. While we acknowledge the inherent limitation of using standardized racial categorization and binary gender groups, our work is a first step toward the evaluation of de-identifying names in EHRs, capturing the real harm that gaps could incur.

First, we investigate whether demographic bias exists in clinical de-identification methods. While some methods attain an overall competitive recall, a majority of the examined methods exhibit statistically significant performance gaps along most demographic dimensions. For instance, we note that these methods are, on average, significantly better at recognizing “rare” names in White people than “popular” names in Asian people.

Second, we assess potential factors contributing to the observed underperformance. We find that names with polysemy—other meanings in English—are disproportionately unrecognized, regardless of the associated races. Most methods suffer when the gender inferred from a name disagrees with the gender suggested by the semantic context. Certain note characteristics, such as length and the number of unique names included, also reduce performance.

Third, we perform fine-tuning on two of the open-source de-identification methods (spaCy (Honnibal et al., 2020) and NeuroNER (Dernoncourt et al., 2017, 2016)) with clinical context and diverse names. We find that this significantly improves the methods’ overall performance and reduces their demographic bias, especially along the dimensions of race and popularity. We advise that this simple, method-agnostic solution should be a minimal first step for practitioners in the de-identification space.

We contribute a comprehensive analysis of the bias in de-identif-ying names from clinical notes, with insights into the existence of the bias and the cause of the underperformance, and provide a simple mitigation option. We emphasize that asymmetric de-identification by existing methods could violate legal regulations and is a serious socio-technical ethical issue. We encourage future work to build upon our results, balancing both de-identification performance and demographic fairness.

2. Related Work

De-identification

The HIPAA Safe Harbor regulations require clinical trial data to be properly anonymized before being shared for various purposes (Lo, 2015; Tucker et al., 2016). Toward this goal, the de-identification of EHRs has drawn long-lasting attention from both clinical practitioners and the NLP community (Kayaalp, 2017; Meystre et al., 2010). Traditional de-identification methods use rule-based pattern matching (Beckwith et al., 2006; Friedlin and McDonald, 2008; Thomas et al., 2002; Norgeot et al., 2020) or ML algorithms (Uzuner et al., 2008; Dernoncourt et al., 2016; Aberdeen et al., 2010; Yang and Garibaldi, 2015) for sequence tagging and attain competitive results in the i2b2 (Informatics for Integrating Biology and the Bedside) de-identification challenges (Uzuner et al., 2007; Stubbs and Uzuner, 2015). Several companies, like Google and Amazon, also provide commercial services to detect and obscure PHI data in plain text. Along a related line of research, many NLP systems (Honnibal et al., 2020; Qi et al., 2020; Akbik et al., 2019; Manning et al., 2014) can fulfill a similar goal by treating de-identification as an NER problem (Li et al., 2020; Song et al., 2021; Yadav and Bethard, 2018).

Bias in NLP Systems

Existing work reports the prevalence of systematic bias in NLP frameworks (Shah et al., 2020; Blodgett et al., 2020). Unfairness in text representations (Papakyriakopoulos et al., 2020; Bolukbasi et al., 2016; Caliskan et al., 2017; Kurita et al., 2019; Zhang et al., 2020) or language models (Nadeem et al., 2021; Nangia et al., 2020) can be escalated in downstream applications such as sentiment analysis (Bhaskaran and Bhallamudi, 2019; Kiritchenko and Mohammad, 2018), machine translation (Savoldi et al., 2021; Stanovsky et al., 2019), and coreference resolution (Rudinger et al., 2018; Zhao et al., 2018). Gender (Chaloner and Maldonado, 2019; Sun et al., 2019; Maudslay et al., 2019) and racial (Blodgett and O’Connor, 2017; Davidson et al., 2019) bias in NLP systems may bring about catastrophic social consequences (Hutchinson et al., 2020; Sap et al., 2019). In response, researchers have proposed metrics (Borkan et al., 2019; Czarnowska et al., 2021; Jacobs et al., 2020) and methods (Prost et al., 2019; Shin et al., 2020; Huang et al., 2020) to mitigate bias in NLP models.

Bias in Healthcare and Other High-Stakes Applications

Demographic bias exists in healthcare systems (Williams and Wyatt, 2015; Webster et al., 2022), typically in an implicit and unconscious way (Zestcott et al., 2016; Marcelin et al., 2019; Hall et al., 2015). For instance, when medical assistance leverages biased artificial intelligence (Mehrabi et al., 2021; Hutchinson and Mitchell, 2019), the unfairness is usually carried forward to subsequent healthcare practice (Ganz et al., 2021; Gianfrancesco et al., 2018). Hence, addressing the bias here demands joint efforts from both ML researchers (Chouldechova and Roth, 2020; Beutel et al., 2019) and healthcare professionals (Byrne and Tanesini, 2015; Ochs, 2022). Bias could also occur in other high-stakes domains such as job applications (De-Arteaga et al., 2019; Bertrand and Mullainathan, 2004; Hannák et al., 2017) and law enforcement (Buolamwini and Gebru, 2018; Drozdowski et al., 2020; Eisenman, 1995). We leave a detailed discussion of the bias in those areas to future work.

Name Set	Gender	Race	Popularity	Decade	First Name Examples	Last Name Examples
1	Male	White	Top	2000s	Jacob, Ethan, Tyler, …	Smith, Davis, Brown, …
2	Female	White	Top	2000s	Emily, Emma, Olivia, …	Smith, Davis, Brown, …
3	Male	White	Medium	2000s	Wade, Ted, Brien, …	Waldon, Clapp, Bogle, …
4	Female	White	Medium	2000s	Mabel, Liz, Terressa, …	Waldon, Clapp, Bogle, …
5	Male	White	Bottom	2000s	Nicki, Leslee, Marti, …	Lofft, Lyna, Tamaro, …
6	Female	White	Bottom	2000s	Glenn, Lyle, Heath, …	Lofft, Lyna, Tamaro, …
7	Male	Black	Medium	2000s	Cedric, Marlon, Ollie, …	Booker, Grier, Spikes, …
8	Female	Black	Medium	2000s	Aisha, Ebony, Jamila, …	Booker, Grier, Spikes, …
9	Male	Asian	Medium	2000s	Zhi, Nguyen, Rajeev, …	Ngo, Mao, Ahmed, …
10	Female	Asian	Medium	2000s	Neha, Priya, Xin, …	Ngo, Mao, Ahmed, …
11	Male	Hispanic	Medium	2000s	Leonel, Camilo, Cruz, …	Ceja, Amaro, Recinos, …
12	Female	Hispanic	Medium	2000s	Celina, Rebeca, Luisa, …	Ceja, Amaro, Recinos, …
13	Male	White	Top	1970s	Patrick, Brian, Eric, …	Smith, Davis, Brown, …
14	Female	White	Top	1970s	Amy, Lisa, Laura, …	Smith, Davis, Brown, …
15	Male	White	Top	1940s	Jerry, George, Frank, …	Smith, Davis, Brown, …
16	Female	White	Top	1940s	Linda, Carol, Nancy, …	Smith, Davis, Brown, …

Table 1. 16 name sets of diverse demographic backgrounds and examples of first and last names for each set. Name Sets $1\sim 6$ are names with top, medium, and bottom popularity in the 2000s that are also exclusive to the White racial group. Name Sets $7\sim 12$ are names with medium popularity in the 2000s that are also exclusive to the Black, Asian, and Hispanic racial groups. Name Sets $13\sim 16$ are names with top popularity in the 1970s and 1940s that are also exclusive to the White racial group.

Bias in Clinical De-identification

In light of the discussion above, it is crucial to carefully examine the bias in de-identification methods, given the pivotal role of de-identification in healthcare pipelines. Previous work (Mansfield et al., 2022; Mehrabi et al., 2020; Mishra et al., 2020) has only compared a small set of baselines based on template sentences that are much simpler than realistic clinical de-identification challenges. There lacks a holistic analysis that explores the bias in de-identification methods of different categories, the factors leading to the methods’ underperformance, and the solution to alleviate the bias. Therefore, our paper aspires to fill this gap via extensive empirical studies based on 16 name sets with diverse demographic backgrounds, 100 real-world clinical note templates, and nine public and private de-identification methods.

3. Experiment Setup

In this paper, we focus on assessing the bias in de-identifying a specific type of PHI data—people’s names—from clinical records. We choose names amongst the defined PHI types because they are commonly associated with specific demographic features and are particularly identifiable.

As illustrated in Figure 1, we first identify (a) four demographic dimensions (i.e., gender, race, name popularity, and the decade of popularity) and prepare (b) 16 name sets with diverse demographic settings in Table 1. Each name set consists of 20 first and 20 last names, which can be paired to produce 400 full names in total. We then curate (c) 100 clinical templates from hospital discharge records (Lim et al., 2023). For each name set, we duplicate each of the 100 templates ten times and fill in full names randomly generated from that name set. This creates a total of (d) 16,000 notes with 116,160 name mentions for evaluation. We use these notes to conduct a large-scale empirical analysis of (e) nine de-identification baseline methods to inspect the bias along the four demographic dimensions.

3.1. Definition of Demographic Dimensions

To measure the demographic information associated with a name, we define the following four demographic dimensions.

• The gender of a name refers to the sex assigned at birth to someone with that name, because the phonological property of a name suggests the associated gender (Cassidy et al., 1999). We examine two groups for gender: male and female.

• The race of a name refers to the expected racial or ethnic identity of someone with that name, reflecting the variation in prevalence that exists between different self-reported racial or ethnic groups (Harris, 2015). We consider four racial or ethnic groups: White, Black, Asian, and Hispanic. Other groups are skipped due to prohibitively small community sizes.

• The popularity of a name refers to the size of the population of a gender within a decade having that name. We compare three groups here: top, medium, and bottom popularity.

• The decade of popularity refers to the decade in which a name is popular in the U.S. in terms of babies being given the name, as name trends change over time (Hahn and Bentley, 2003). We assess three decade groups: 2000s, 1970s, and 1940s.

Limitations of Standardized Demographic Categories. We acknowledge the limitation of using standardized self-reported racial categorization and binary gender groups when composing the name sets. More fine-grained racial categorizations are possible in future work, and there could be variety in the linguistic norms and naming traditions even within each racial group we consider. Transgender and non-binary gender groups are also important to consider in future work, as these groups may use gender-neutral names or have variations in name usage between records.

We use standardized self-reported racial categorization and binary gender groups because it is important to evaluate the performance of de-identification methods on data that is routinely collected in EHRs (Bergdall et al., 2012). We emphasize that we do not perform any demographic inference as part of a classification system or training set in this work. We do not believe that these categories should be viewed as scientific truth and recognize the larger critical interrogation surrounding whether gender and ethnicity should be discerned from names in such systems (Lockhart et al., 2023). Instead, we use these categories in the spirit in which they were created by the U.S. Office of Management and Budget to “monitor and redress social inequality” (Bliss, 2012). The examination of the impact of more fluid categorizations of gender, race, and religion is important for future work in this space.

3.2. Construction of Name Sets

In this study, we compute the popularity of first names for each gender based on the U.S. Social Security dataset (ssa, nd) across the entire population, rather than for each racial group. We then select names that are primarily associated with a self-identified racial group with a margin over 10% based on the mortgage application dataset in (Tzioumis, 2018). We note that this is different from picking the most popular names for each racial group independently.

In the U.S. setting, all top popularity names, as evaluated by absolute frequency ranking, are identified with the White racial group. For this reason, we consider names associated with the Black, Asian, or Hispanic groups that are of medium popularity. First names of medium popularity for each race and gender (i.e., Name Sets 3, 4, 7, 8, 9, 10, 11, and 12) are randomly sampled from those with a frequency ranking between 400 and 8,000 in the entire population in the 2000s. First names of bottom popularity for the White group (i.e., Name Sets 5 and 6) are randomly sampled from those occurring exactly five times in the 2000s. We set each name set to 20 names since based on the procedure described above, there are only 20 names that are of medium popularity in the 2000s and primarily used by Black males. We also ensure that first names of top popularity within each gender and decade are mutually exclusive (i.e., no shared first names in Name Sets 1, 2, 13, 14, 15, and 16).

We prepare last names in a similar fashion based on the 2000 Census dataset alone (cen, nd), because we assume that the last name popularity is relatively fixed. Specifically, this means that the most popular last names for the White racial group in the 1970s and 1940s are assigned to be the same as those in the 2000s.

Limitations of the Datasets. We acknowledge that our datasets are limited to the U.S., and therefore, our findings need to be reproduced in other contexts with distinct name distributions. Furthermore, our use of the mortgage application dataset for self-reported racial matching is limited to those who have the financial security to apply for a loan. As we do not have access to other sources of names and self-reported races, we use the available data to demonstrate that—even in this presumably more privileged subset of the population—there are de-identification gaps.

3.3. Group Pooling for Demographic Performance Comparisons

To evaluate model performance along each demographic dimension, we design experiments that control for other dimensions as follows.

• We assess the impact of gender by pooling and comparing the results of male (i.e., 1, 3, 5, 7, 9, 11, 13, and 15) and female Name Sets (i.e., 2, 4, 6, 8, 10, 12, and 16). Race, popularity, and decade of popularity all vary within these two groups.

• We compare performance along race by pooling Name Sets 3 and 4 for the White group, Name Sets 7 and 8 for the Black group, Name Sets 9 and 10 for the Asian group, and Name Sets 11 and 12 for the Hispanic group. These are the male and female names of medium popularity in the 2000s across the four racial groups.

• We examine the influence of popularity by forming and comparing names with varying levels of popularity within the White group, where top popularity is based on Name Sets 1 and 2, medium popularity is based on Name Sets 3 and 4, and bottom popularity is based on Name Sets 5 and 6.

• We evaluate the difference in performance among the three decade groups by comparing the male and female names of top popularity for the White group in each decade: Name Sets 1 and 2 for the 2000s, Name Sets 13 and 14 for the 1970s, and Name Sets 15 and 16 for the 1940s.

3.4. Preparation of Clinical Templates

We manually curate 100 clinical note templates based on hospital discharge records from Beth Israel Lahey Health between 2017 and 2019. We follow the HIPAA Safe Harbor provisions by marking the occurrence of names in the templates and replacing other PHI classes with realistic, synthetic values. We note that our templates (Lim et al., 2023) are more complex than those used in existing benchmark datasets (Mansfield et al., 2022; Mehrabi et al., 2020; Mishra et al., 2020), with an average of 12,893 characters and 3.5 unique names per template and each unique name appearing an average of 2.1 times per template. This design is more analogous to real-world de-identification challenges and more likely to expose flaws in less effective methods.

Method	Overall Performance ($\uparrow$)			Bias along Dimensions ($\downarrow$)
	Precision	Recall	F1	Gender	Race	Popularity	Decade
spaCy	0.917$\pm$0.001	0.629$\pm$0.001	0.746$\pm$0.001	0.002${}^{*}\pm$0.001	0.013${}^{*}\pm$0.002	0.028${}^{*}\pm$0.002	0.007${}^{*}\pm$0.002
Stanza	0.678$\pm$0.001	0.881$\pm$0.001	0.766$\pm$0.001	0.002${}^{*}\pm$0.001	0.016${}^{*}\pm$0.002	0.011${}^{*}\pm$0.001	0.005${}^{*}\pm$0.001
flair	0.920$\pm$0.001	0.974$\pm$0.000	0.946$\pm$0.000	0.003${}^{*}\pm$0.000	0.006${}^{*}\pm$0.001	0.008${}^{*}\pm$0.001	0.002${}^{*}\pm$0.000
Amazon	0.923$\pm$0.001	0.925$\pm$0.001	0.924$\pm$0.001	0.005${}^{*}\pm$0.001	0.022${}^{*}\pm$0.001	0.032${}^{*}\pm$0.001	0.001$\pm$0.001
Microsoft	0.664$\pm$0.001	0.960$\pm$0.001	0.785$\pm$0.001	0.003${}^{*}\pm$0.001	0.023${}^{*}\pm$0.001	0.010${}^{*}\pm$0.001	0.006${}^{*}\pm$0.001
Google	0.609$\pm$0.001	0.869$\pm$0.001	0.716$\pm$0.001	0.009${}^{*}\pm$0.001	0.025${}^{*}\pm$0.001	0.014${}^{*}\pm$0.002	0.010${}^{*}\pm$0.001
NeuroNER	0.946$\pm$0.001	0.944$\pm$0.001	0.945$\pm$0.000	0.001$\pm$0.001	0.045${}^{*}\pm$0.001	0.026${}^{*}\pm$0.001	0.002$\pm$0.001
Philter	0.227$\pm$0.001	0.794$\pm$0.001	0.353$\pm$0.001	0.000$\pm$0.001	0.000$\pm$0.001	0.003${}^{*}\pm$0.002	0.000$\pm$0.001
MIST	0.474$\pm$0.001	0.751$\pm$0.001	0.581$\pm$0.001	0.013${}^{*}\pm$0.001	0.022${}^{*}\pm$0.002	0.017${}^{*}\pm$0.002	0.003${}^{*}\pm$0.002

Table 2. Overall performance (higher is better), bias along demographic dimensions (lower is better), and the associated bootstrapped standard error of the examined de-identification methods. We measure the bias with recall equality difference and bold the best two scores in each column. In particular, flair achieves the highest recall and F1 and the lowest bias for race and popularity. Moreover, the asterisk next to a bias score indicates a statistically significant difference in performance at an adjusted significance level ($5\%$ for gender, $0.833\%$ for race, $1.667\%$ for popularity and decade). A majority of the examined methods exhibit statistically significant performance gaps along most demographic dimensions.

3.5. De-identification Baseline Methods

In our large-scale empirical analysis, we examine nine popular de-identification methods of three different categories. For packages that offer multiple model options, we report the option with the highest performance in our experiments.

Three off-the-shelf NLP libraries with the NER function:

• spaCy (Honnibal et al., 2020) (25.9k GitHub Stars) is widely adopted for industrial information extraction. We choose RoBERTa-base (Liu et al., 2019), which is pre-trained on a massive general-purpose corpus, as the backbone of its NER pipeline.

• Stanza (Qi et al., 2020) (6.6k GitHub Stars) is a natural language analysis package. We apply its 18-class NER model variant based on the contextual string representations (Akbik et al., 2018) and pre-trained on the OntoNotes corpus (Weischedel et al., 2013).

• flair (Akbik et al., 2019) (12.7k GitHub Stars) is a powerful NLP framework. We employ its large four-class NER model variant built on XLM-R embeddings (Conneau et al., 2020) and document-level features (Schweter and Akbik, 2020) and pre-trained on the CoNLL03 corpus (Sang and De Meulder, 2003).

Three commercial services for PHI detection:

• Amazon Comprehend Medical DetectPHI Operation (ama, nd) is a HIPAA-eligible NLP service. We segment input notes into pieces shorter than 20,000 characters, the maximum allowed input length, when making the API calls.

• Microsoft Azure Cognitive Service for Language PHI Detection (mic, nd) de-identifies PHI information in unstructured texts. We divide notes into slices shorter than 5,120 characters to obey the input length threshold.

• Google Cloud Data Loss Prevention De-identification API (goo, nd) inspects and redacts sensitive data intelligently. We select the outputs for the class PERSON_NAME and remove the titles before the recognized full names.

We note that both Amazon Comprehend Medical DetectPHI Operation and Microsoft Azure Cognitive Service for Language PHI Detection are intended to be used for our specific case of free-text medical note de-identification. Google Cloud Data Loss Prevention De-identification is intended for the general text. We use this service because other medically-focused services operated by Google do not operate on free-text notes. Specifically, Google Cloud Healthcare API for de-identification (ggH, nd) only operates on FHIR JSON embeddings and DICOM images, and Google Cloud Healthcare Natural Language API (gHe, nd) only recognizes medical knowledge categories.

Three open-source de-identification toolkits:

• NeuroNER (Dernoncourt et al., 2017, 2016) (212 citations) is an NER tool based on the long short-term memory model (Hochreiter and Schmidhuber, 1997). We use the model pre-trained on the 2014 i2b2 de-identification corpus (Stubbs and Uzuner, 2015) with GloVe word embeddings (Pennington et al., 2014) and collect the outputs for PATIENT and DOCTOR as the set of recognized names.

• Philter (Protected Health Information filter) (Norgeot et al., 2020) (31 citations) leverages the Python NLTK module and regular expressions for rule-based de-identification.

• MIST (MITRE Identification Scrubber Toolkit) (Aberdeen et al., 2010) (156 citations) is a suite of tools for identifying and redacting PHI in free-text medical records. We pre-train the model supplied by the Carafe engine, a conditional random field-based (Lafferty et al., 2001) sequence tagger, on the 2006 i2b2 de-identification corpus (Uzuner et al., 2007) as instructed and view the outputs for the classes PATIENT and DOCTOR as the set of recognized names.

3.6. Evaluation of Bias

To quantify the bias of each method along each dimension, we follow (Mansfield et al., 2022) by introducing the recall equality difference: the average absolute difference between the recall of each demographic group and that of all the groups along the corresponding demographic dimension. More specifically, for dimension $D$ and its entailed set of demographic groups $\mathcal{G}^{D}=\{\mathcal{G}_{1}^{D},\mathcal{G}_{2}^{D},\dots\}$, recall equality difference $=\frac{1}{|\mathcal{G}^{D}|}\sum_{\mathcal{G}_{i}^{D}\in\mathcal{G}^{D}}|\text{Recall}(\mathcal{G}_{i}^{D})-\text{Recall}(\mathcal{G}^{D})|$. We use the recall equality difference as the fairness metric since it demonstrates the difference in recall each demographic group would experience while expecting the reported average performance. We also explore another fairness metric—recall maximum difference—and report the results in Appendix A.3.

We carry out the Wilcoxon signed-rank test (Woolson, 2007) for the dimension of gender and the Friedman test (Friedman, 1937) for the dimensions of race, popularity, and decade to assess the null hypothesis that a de-identification method treats all the groups equally well along a demographic dimension. After applying the Bonferroni correction (Weisstein, 2004), the adjusted significance levels for gender, race, popularity, and decade are $5\%$, $0.833\%$, $1.667\%$, and $1.667\%$, respectively.

Figure 2. Recall and 95% bootstrapped confidence interval of the demographic groups along each dimension by each examined de-identification method. Disparities in performance between different groups are more obvious along the dimensions of race and popularity than along the dimensions of gender and decade.

4. Q1: Is There Demographic Bias?

Toward the first question of whether demographic bias exists in de-identification methods, we obtain two key takeaways. First, the tested de-identification methods perform differently, with some achieving a relatively high recall. Second, a majority of the methods exhibit statistically significant performance gaps along most demographic dimensions. Such disparities call for urgent review and action to address bias in existing de-identification methods.

4.1. Overall Performance Varies

We present the overall performance of the nine de-identification methods in Table 2. The performance varies across the methods with some methods obtaining a relatively high recall. In particular, flair performs rather well, especially in recall and F1, probably due to its use of large pre-trained language models and document-level features. NeuroNER also achieves competitive scores, especially in precision and F1, possibly because it is pre-trained on clinical corpora. In contrast, spaCy gives the lowest recall, which suggests a high risk of information leakage, albeit its popularity in the NLP community (it has the most GitHub Stars among the three NLP libraries we consider). Interestingly, Google dramatically underperforms compared to the other two commercial platforms (i.e., Amazon and Microsoft). As a rule-based method, Philter outputs highly imprecise predictions in the complicated clinical context.

4.2. Significant Demographic Gaps in De-identification Performance

We find that a majority of the examined methods demonstrate statistically significant differences in performance along most of the four demographic dimensions. Table 2 exhibits the recall equality difference and the hypothesis test results, where an asterisk next to a score indicates a statistically significant difference at the corresponding significance level. In particular, Amazon and Google give the highest recall equality difference for name popularity and the decade of popularity, respectively, which should be a call for action for these commercial services. Although NeuroNER delivers an overall competitive de-identification performance, its recall equality difference is rather high, especially along the dimensions of race and popularity. We note that the rule-based Philter has very low bias and that flair achieves not only the highest recall but also relatively low recall equality differences along all four dimensions.

At a fine-grained level, we plot in Figure 2 the recall of the demographic groups along each dimension by each method. Along the dimension of gender, all the methods score better or equally well for female names than male names. Nevertheless, these methods act very differently in the four racial groups. More specifically, Stanza and NeuroNER attain very low recall in the Asian racial group, while MIST scores much higher. The three commercial services—Amazon, Microsoft, and Google—all perform better in the White and Hispanic racial groups than in the Black and Asian racial groups. Moreover, the performance of most methods deteriorates with the popularity of names, with the exceptions of Stanza and MIST. Finally, the disparity in recall among the three groups with different decades of popularity is more moderate. Stanza, Microsoft, and Google are more capable of recognizing popular names from more recent decades, while spaCy behaves oppositely.

We visualize in Figure 3 the recall of the 16 name sets averaged across the examined methods to further examine performance disparities. We observe that the average recall of the name sets with top popularity (i.e., Name Sets 1, 2, 13, 14, 15, and 16) outperforms the other sets. In addition, we note that the least popular names associated with the White racial group (i.e., Name Sets 5 and 6) score higher on average recall than the more popular names associated with the Asian racial group (i.e., Name Sets 9 and 10).

Figure 3. Average recall and standard error of each name set by the examined de-identification methods, ordered by decreasing recall. The average recall on name sets with top popularity exceeds the other sets by a clear margin. Moreover, the methods are, on average, more capable of recognizing less popular names associated with the White racial group compared to more popular names associated with the Asian racial group.

Figure 4. Recall and 95% bootstrapped confidence interval on polysemy first names associated with three racial groups by each examined de-identification method. The recall ranking among the three groups remains relatively consistent for most methods as that based on the original setting in Figure 2 (b). The increase in recall illustrated by the lighter color bar refers to the partially correct de-identification of non-polysemy last names.

5. Q2: What Leads to De-Identification Underperformance?

For the second question of what factors contribute to the underperformance, we draw three critical findings.

• Polysemy names account for methods’ underperformance but not necessarily their demographic bias.

• Most methods are better at recognizing names in agreement with the gender suggested by the local context.

• Longer templates with more unique names and medication injection histories make de-identification harder.

5.1. Polysemy Names Cause Underperformance

To understand what names are the hardest to recognize, we calculate the recall of each sampled name. We observe that names with the lowest recall usually have other meanings in English (i.e., polysemy). For instance, “An” in An Dizon and An Son—the two names with the lowest recall—is both a prevalent determiner in English and a first name of medium popularity associated with the Asian female group. “Cleveland” in Cleveland Spikes—the fifth hardest name by recall—is both a large city in the U.S. and a first name of medium popularity for the Black male group.

Therefore, we prepare five polysemy first names for each of the White, Black, and Asian racial groups as follows:

• White: Sydney, Faith, Forest, Cliff, June

• Black: Quincy, Cleveland, Kenya, Prince, Ivory

• Asian: Asian, Thai, King, Long, Young, Can

These sets share the same gender, name popularity, and the decade of popularity and only differ in race. Since we can only find five polysemy first names from the Black racial group and not enough polysemy first names from the Hispanic group that meet this requirement, we limit all sets to five names and omit the Hispanic group here. We then follow the procedure in Sec 3 and evaluate the methods on the polysemy first names listed above.

Figure 5. Difference in recall and 95% bootstrapped confidence interval between names that are consistent and inconsistent with the genders suggested by the local context. A positive recall difference means that performance was best when there was gender consistency, while a negative recall difference means that performance was best when there was gender inconsistency. Methods leveraging the gender context for name recognition are expected to see a positive recall difference.

Figure 6. Relationship between template characteristics and template recall averaged across the examined methods. With statistically significant p-values, a template’s average recall decreases with its length and the number of unique names included.

Figure 7. Average recall and snippets of three templates. Unlike usual templates (e.g., Template 50), templates with a low average recall (e.g., Templates 86 and 67) usually include medication injection histories that offer little semantic context for name recognition.

As shown in Figure 4, although we utilize polysemy first names for all three racial groups, the variation in performance persists. In addition, for all the methods except Stanza, the recall ranking of the three racial groups assessed on polysemy first names remains relatively consistent as that based on the original setting in Figure 2 (b). We also consider the scenario when a method can correctly recognize the non-polysemy last names and plot the increased recall above the original bar in lighter colors in Figure 4. In this case, most methods can see a significant increase in recall, especially for Google, NeuroNER, and Philter. Hence, names with overlap** meanings in English only explain the underperformance of the de-identification methods, but not necessarily their bias across demographic groups.

Method	Fine-tuning		Overall Performance ($\uparrow$)			Bias along Dimensions ($\downarrow$)
	Context	Name	Precision	Recall	F1	Gender	Race	Popularity	Decade
spaCy	out-of-the-box		0.916	0.623	0.741	0.003	0.027	0.025	0.005
	clinical	diverse	0.990$\pm$0.007	0.950$\pm$0.006	0.969$\pm$0.002	0.012$\pm$0.004	0.024$\pm$0.005	0.005$\pm$0.002	0.006$\pm$0.001
	clinical	popular	0.998$\pm$0.004	0.737$\pm$0.072	0.846$\pm$0.046	0.012$\pm$0.007	0.094$\pm$0.029	0.127$\pm$0.035	0.003$\pm$0.004
	general	diverse	0.915$\pm$0.072	0.830$\pm$0.083	0.864$\pm$0.035	0.036$\pm$0.005	0.071$\pm$0.011	0.049$\pm$0.042	0.008$\pm$0.005
	general	popular	0.873$\pm$0.110	0.492$\pm$0.069	0.629$\pm$0.083	0.010$\pm$0.003	0.059$\pm$0.032	0.326$\pm$0.060	0.007$\pm$0.003
NeuroNER	out-of-the-box		0.955	0.953	0.954	0.005	0.044	0.030	0.001
	clinical	diverse	0.978$\pm$0.014	0.978$\pm$0.009	0.978$\pm$0.005	0.007$\pm$0.001	0.019$\pm$0.006	0.012$\pm$0.008	0.002$\pm$0.001
	clinical	popular	0.989$\pm$0.003	0.865$\pm$0.021	0.923$\pm$0.013	0.008$\pm$0.004	0.065$\pm$0.007	0.118$\pm$0.010	0.001$\pm$0.001
	general	diverse	0.958$\pm$0.022	0.943$\pm$0.029	0.950$\pm$0.010	0.016$\pm$0.007	0.041$\pm$0.010	0.031$\pm$0.014	0.007$\pm$0.006
	general	popular	0.924$\pm$0.022	0.777$\pm$0.018	0.844$\pm$0.019	0.003$\pm$0.001	0.062$\pm$0.005	0.324$\pm$0.021	0.004$\pm$0.003

Table 3. Overall performance (higher is better) and bias along demographic dimensions (lower is better) of two de-identification methods fine-tuned with different setups. We measure the bias with recall equality difference, report the mean scores and standard errors based on five trials with different seeds, and bold the best score in each column for each method. For both methods, using clinical context and diverse names for fine-tuning improves the overall performance and reduces the demographic bias along most dimensions, especially race and popularity.

5.2. Methods Improve when De-identifying Context-Consistent Names

NER systems usually capture the contextual dependencies for tag decoding (Li et al., 2020), and the semantic context often indicates the gender associated with a name. For example, titles (e.g., Mr. and Mrs.) can appear before full names, and appositions (e.g., son and daughter) can describe relationships. We expect methods leveraging such context for name recognition to have higher recall on names where there is local context agreement with the gender as compared to those with disagreement. To assess this, we identify in our note templates where name gender can be easily inferred from the local context to determine if the consistency between the names and the inferred genders impacts de-identification quality.

Figure 5 plots the recall difference between context-consistent and -inconsistent names by the examined methods. Albeit with relatively large confidence intervals, We find that most methods perform better on names aligned with the implied gender. spaCy is the only exception, perhaps shedding light on its lowest overall recall (see Table 2).

Limitations of Gender-Inconsistent Evaluation in Experiment Setup. We acknowledge that replacing gender-inconsistent pronouns in notes prior to evaluation would be an easier test for models. However, we note that not all clinical records will contain gender-confirming pronouns, especially for transgender and non-binary individuals (Lockhart et al., 2023), and argue that de-identification methods should be able to operate properly in these gender-inconsistent situations. We also note that if we limit our analysis to only using male-originating notes with male name sets and female-originating notes with female name sets, our results still hold (see Appendix A.1). We note that in this setting, we do not explicitly assess the gender gap since male- and female-originating notes do not overlap.

5.3. Performance Decays with Template Length and Name Quantity

Other properties of a note template may also affect the de-identifica-tion performance. We consider three characteristics—template leng-th, number of unique names, and number of name mentions in a template—and visualize their relationships with a template’s average recall in Figure 6. Our findings suggest that recall deteriorates with both the length of a note and the number of unique names that it contains.

We identify two of the worst-performing templates in terms of recall: Templates 86 and 67. These templates appear six and four times, respectively, in the five templates with the lowest recall by a method. As shown in Figure 7, unlike other templates (e.g., Template 50), Templates 86 and 67 are notable for having large blocks of medication history that provide little indication for the names that intersperse them. This unique characteristic of clinical records calls for special attention in future de-identification systems. We further investigate the performance of the examined methods on these hard templates in Appendix A.2 and find that their performance follows the overall pattern in Table 2.

6. Q3: Can Bias Be Mitigated?

To answer the third question of how to mitigate the bias in de-identification methods, we propose a simple and method-agnostic solution of fine-tuning the methods with clinical context and diverse names. This setup not only improves the overall recall but also reduces the bias significantly along most demographic dimensions.

6.1. Fine-tuning De-identification Methods

We prepare the fine-tuning de-identification datasets by considering two types of context and two types of names. We treat the longitudinal clinical narratives in the 2014 i2b2 de-identification challenge (Stubbs and Uzuner, 2015) as the clinical context and the Wikipedia articles in the DocRED dataset (Yao et al., 2019) as the general context. We generate 160 diverse names by randomly sampling ten names from each of the 16 name sets in Table 1 and 160 popular names based on the most popular names over the three chosen decades that do not appear in the 16 name sets. For each type of context, we randomly sample 1,000 templates for training and 100 for validation. These templates are then populated with the names of each type (i.e., diverse names and popular names) separately. In this way, we create four fine-tuning setups in total by pairing the two types of context with the two types of names.

To compare the effectiveness of these setups, we fine-tune two de-identification methods—spaCy (Honnibal et al., 2020) and NeuroNER (Dernoncourt et al., 2017, 2016)—with distinct out-of-the-box performance. spaCy is a widely-adopted NLP library that delivers a low de-identification recall and a moderate demographic bias in Table 2. In contrast, NeuroNER is pre-trained on the original 2014 i2b2 de-identification corpus, which yields a competitive recall with high bias along the dimensions of race and popularity. After fine-tuning with their respective default hyperparameters, these methods are evaluated on 1,600 test notes. These test notes are constructed by filling in the 100 templates in Sec 3.4 with the remaining ten names (not selected for the 160 diverse names during fine-tuning) from each of the 16 name sets separately. Here, the test notes are disjoint with the fine-tuning context/names.

6.2. Clinical Context and Diverse Names Improve Performance

Table 3 displays the overall performance and the demographic bias (i.e., the recall equality difference) of the two methods after fine-tuning. We repeat the fine-tuning five times with different seeds and report the mean scores and standard errors. Impressively, despite the distinct out-of-the-box performance of the two fine-tuned methods, the setup composed of clinical context and diverse names largely enhances the overall performance of both methods and diminishes their unfairness, especially along the dimensions of race and popularity.

In particular, although most of the four fine-tuning setups improve spaCy’s overall performance, fine-tuning with clinical context and diverse names sees the largest boost in spaCy’s recall by over 0.3. On the other hand, since NeuroNER is pre-trained on clinical corpora, most of the four fine-tuning setups are ineffective in enhancing NeuroNER’s strong out-of-the-box performance. However, fine-tuning with clinical context and diverse names is the only exception here, which increases the precision, recall, and F1 of NeuroNER by around 0.02 each. Moreover, along the dimensions of race and popularity, where the degree of unfairness is rather high, this setup can significantly reduce the bias of both methods.

We suggest that fine-tuning de-identification methods with clinical context and diverse names should be done as an immediate fix to improve fairness before applying the methods to clinical tasks. The method-agnostic effectiveness and simplicity of this setup highlight the importance of training data diversity to model fairness (Madan et al., 2022).

7. Discussion

Demographic Associations of Names

Names can be associated with certain demographic features (Liu and Ruths, 2013; Gaddis, 2017). For instance, in our U.S. Social Security (ssa, nd) and Census (cen, nd) data sources, there is variation in name popularity between self-reported ethnic groups. In human decision-making, such associations have been shown to correlate with discriminative hiring (Bertrand and Mullainathan, 2004; Hannák et al., 2017) and loan granting (Hanson et al., 2016) practices. Other work has explored the biases learned by large language models when the demographic context is varied directly in input (Liang et al., 2021) or using names as a proxy for demographic (Mansfield et al., 2022; Mehrabi et al., 2020; Mishra et al., 2020). For example, NLP models link the female gender to specific stereotypical occupations (Bolukbasi et al., 2016) and tend to generate violent or negative-toned text when given “Muslim” as a demographic descriptor for input (Abid et al., 2021). We emphasize that the biases inherently learned by NLP models may perpetuate biases and, therefore, require careful audits. We acknowledge that our analysis based on de-identifying names may not necessarily generalize to other PHI types and leave this further investigation to future work.

Bias in Healthcare

Bias in healthcare can occur in both systematic and implicit ways based on demographic factors such as race, ethnicity, gender, sexual orientation, or socio-economic status (FitzGerald and Hurst, 2017; Zestcott et al., 2016; Marcelin et al., 2019; Hall et al., 2015). These biases can then be unintentionally learned by ML models (Ahmad et al., 2020; Gianfrancesco et al., 2018; Parikh et al., 2019). For instance, NLP models trained on race-redacted clinical notes have been shown to capture self-reported race through other proxy information (Adam et al., 2022) and mimic the existing biases in text completions for clinical treatment decisions (Zhang et al., 2020). Our study demonstrates that existing clinical de-identification methods discriminate based on the demographic associations of names. The bias in these methods could further escalate the unfairness in downstream healthcare systems.

Importance of De-identified Data for Reproducibility

ML models rely on large amounts of data for training (Beam and Kohane, 2018), but in the case of health data, there are privacy concerns. By removing PHI, researchers can protect stakeholders’ privacy with de-identified data (Seastedt et al., 2022) and avoid biasing their models through more representative datasets (Chen et al., 2021). To this end, clinical de-identification has attracted long-lasting attention from the research community (Kayaalp, 2017; Meystre et al., 2010) and large amounts of resources from the industrial world (e.g, (pri, nd; hea, nd)). We highlight the importance of equitable de-identification because legal and ethical data sharing should be encouraged (Seastedt et al., 2022) to improve the reproducibility of clinical findings and the credibility of healthcare systems (McDermott et al., 2021; TSIMA, 2023).

Harm of Minority Exclusion

We stress that it is not acceptable to exclude some populations from de-identified data sharing. When demographic groups are absent in data, models trained on that data will perform poorly on the missing groups (Norori et al., 2021). This can result in misdiagnoses, inadequate treatments, and a failure to address health disparities (Ghassemi and Nsoesie, 2022). Hence, it is crucial to ensure that data for model training is diverse and representative of the populations they will serve (Chen et al., 2021). Future work should consider proactive measures to collect and include data from underrepresented populations and address systemic biases during data collection and analysis.

Ramifications of Poorer Privacy for Marginalized Groups

General disparities in de-identification performance can lead to poorer privacy for marginalized groups and engender crimes such as identity theft (Anderson, 2005, 2006). This adds to the existing difficulties with data collection and monitoring faced by marginalized communities (Browne, 2015; Fagan et al., 2016). Even when data sharing is consented, the data can be used outside of the given context, leading to representational harm for groups that are already targeted (Ghassemi and Mohamed, 2022). In future work, we advocate for data collection and de-identification practices that promote trust and do not discourage minorities from seeking medical care and participating in clinical data sharing.

Importance of Audits to Create Change

Audits in healthcare help to identify areas of improvement (Ivers et al., 2012), assess compliance with regulations and standards (Hut-Mossel et al., 2021), and hold organizations accountable for their actions (Raji and Buolamwini, 2019). Past work on ML audits has demonstrated the ability to make meaningful changes and reduce performance gaps in deployed systems with biases. For example, a recent audit on the bias in automated facial analysis algorithms (Buolamwini and Gebru, 2018) stimulated the targeted companies to reduce accuracy disparities between demographic groups (Raji and Buolamwini, 2019), However, companies that provided similar algorithms and were not included in the original audit did not make corresponding changes (Raji and Buolamwini, 2019). We encourage clinical practitioners to build upon our de-identification audit to provide high-quality, equitable de-identification services to all demographic groups.

8. Conclusion

In this paper, we contribute a large-scale empirical analysis of de-identifying names from clinical records and present findings that demonstrate systemic bias in performance. Our results should sound the alarm for clinical and ML stakeholders, as bias in clinical de-identification not only raises legal concerns but also make certain demographic groups more prone to privacy leakage. Hence, we call for an urgent review of existing de-identification methods and actions (e.g., fine-tuning with our recommended setup) to improve the fairness and accountability of healthcare systems.

Despite the comprehensiveness of our study, we acknowledge the limitation of using coarse racial and gender categorizations when constructing our name sets. In addition, while our analysis is readily applicable to many widely-adopted de-identification methods, we did not evaluate its generalization to approaches focusing on other PHI classes. We leave to future work the investigation of bias in de-identifying other PHI classes based on more fluid racial and gender categorizations.

Figure 8. Recall and 95% bootstrapped confidence interval of the demographic groups along the dimensions of race, name popularity, and the decade of popularity by each examined de-identification method under gender-consistent evaluation. These methods behave similarly compared to the original setup in Figure 2.

Appendix A Appendix

In the appendix, we include additional analysis exploring the robustness of our results in gender-consistent note population, in the subset of notes with the poorest overall performance, and using another fairness metric of recall maximum difference.

A.1. Gender-Consistent Note Population

To examine the influence of gender-inconsistent pronouns used in our note template population, we run a robustness check on our results where we only consider male-originating clinical notes populated with male name sets and female-originating notes populated with female name sets. We note that in this setting, we do not conduct a direct comparison of the gender gap since the male- and female-originating notes are disjoint. Otherwise, the experiment follows the procedure in Sec 3.

Figure 8 illustrates the recall of the demographic groups along the dimensions of race, name popularity, and the decade of popularity by each de-identification method under this gender-confirming evaluation setup. The Wilcoxon signed-rank test with p-value $=0.082$ indicates that these methods behave consistently to the original setup in Sec 3, and our observations about the race, popularity, and decade disparities based on Figure 2 still hold.

Method	Overall Performance ($\uparrow$)			Bias along Dimensions ($\downarrow$)
	Precision	Recall	F1	Gender	Race	Popularity	Decade
spaCy	0.874$\pm$0.003	0.504$\pm$0.003	0.640$\pm$0.003	0.004$\pm$0.003	0.022${}^{*}\pm$0.004	0.037${}^{*}\pm$0.005	0.005$\pm$0.004
Stanza	0.615$\pm$0.003	0.791$\pm$0.003	0.692$\pm$0.002	0.001$\pm$0.002	0.007$\pm$0.003	0.028${}^{*}\pm$0.004	0.011${}^{*}\pm$0.003
flair	0.878$\pm$0.002	0.945$\pm$0.001	0.910$\pm$0.001	0.005${}^{*}\pm$0.001	0.014${}^{*}\pm$0.002	0.016${}^{*}\pm$0.002	0.004${}^{*}\pm$0.001
Amazon	0.882$\pm$0.002	0.883$\pm$0.002	0.883$\pm$0.002	0.009${}^{*}\pm$0.002	0.025${}^{*}\pm$0.003	0.047${}^{*}\pm$0.003	0.003${}^{*}\pm$0.002
Microsoft	0.619$\pm$0.003	0.936$\pm$0.002	0.745$\pm$0.002	0.003${}^{*}\pm$0.001	0.033${}^{*}\pm$0.003	0.013${}^{*}\pm$0.002	0.009${}^{*}\pm$0.002
Google	0.558$\pm$0.003	0.856$\pm$0.002	0.676$\pm$0.002	0.011${}^{*}\pm$0.002	0.034${}^{*}\pm$0.003	0.011${}^{*}\pm$0.003	0.008${}^{*}\pm$0.003
NeuroNER	0.929$\pm$0.002	0.899$\pm$0.002	0.914$\pm$0.001	0.005${}^{*}\pm$0.002	0.044${}^{*}\pm$0.003	0.052${}^{*}\pm$0.003	0.005$\pm$0.002
Philter	0.134$\pm$0.001	0.562$\pm$0.003	0.216$\pm$0.002	0.000$\pm$0.002	0.000$\pm$0.003	0.003${}^{*}\pm$0.004	0.000$\pm$0.004
MIST	0.306$\pm$0.002	0.532$\pm$0.003	0.388$\pm$0.002	0.020${}^{*}\pm$0.003	0.040${}^{*}\pm$0.004	0.019${}^{*}\pm$0.005	0.009${}^{*}\pm$0.004

Table 4. Overall performance (higher is better), bias along demographic dimensions (lower is better), and the associated bootstrap standard error of the examined de-identification methods on the hardest 20 templates. We measure the bias with recall equality difference and bold the best two scores in each column. These methods’ overall performance follows the general pattern when evaluated on the full set of 100 templates in Table 2. Some methods exhibit lower bias here, possibly due to equally poor performance across demographic groups in harder templates.

Method	Recall Maximum Difference ($\downarrow$)
	Gender	Race	Popularity	Decade
spaCy	0.002$\pm$0.002	0.025$\pm$0.004	0.042$\pm$0.004	0.010$\pm$0.004
Stanza	0.002$\pm$0.001	0.032$\pm$0.003	0.017$\pm$0.003	0.008$\pm$0.002
flair	0.003$\pm$0.001	0.013$\pm$0.002	0.013$\pm$0.001	0.003$\pm$0.001
Amazon	0.005$\pm$0.001	0.034$\pm$0.002	0.047$\pm$0.002	0.001$\pm$0.001
Microsoft	0.003$\pm$0.001	0.033$\pm$0.002	0.015$\pm$0.001	0.009$\pm$0.001
Google	0.009$\pm$0.001	0.044$\pm$0.003	0.020$\pm$0.003	0.015$\pm$0.003
NeuroNER	0.001$\pm$0.001	0.089$\pm$0.003	0.040$\pm$0.001	0.003$\pm$0.001
Philter	0.000$\pm$0.001	0.000$\pm$0.002	0.004$\pm$0.003	0.000$\pm$0.002
MIST	0.013$\pm$0.002	0.043$\pm$0.004	0.026$\pm$0.004	0.004$\pm$0.003

Table 5. Recall maximum difference (lower is better) and the associated bootstrapped standard error of the examined de-identification methods. We bold the best two scores in each column. The bias in these methods measured by recall maximum difference along each dimension is similar to the pattern measured by recall equality difference in Table 2.

A.2. Evaluation of Difficult Note Templates

Here we identify the set of 20 templates that receive the lowest average recall by the examined de-identification methods and investigate how the performance of these methods changes in Table 4 when evaluated on these harder templates. Although the scores of their overall performance drop compared to Table 2, the best-performing methods based on the original full set of 100 templates still perform well on these hardest 20 templates. However, some of the examined methods, such as Stanza and Google, exhibit lower bias now, potentially due to equally poor performance across demographic groups in harder templates.

A.3. Recall Maximum Difference

Besides recall equality difference, we consider an additional fairness metric—recall maximum difference, which illustrates the largest gap in recall any demographic group would experience while anticipating the reported average performance. For dimension $D$ and its entailed set of demographic groups $\mathcal{G}^{D}=\{\mathcal{G}_{1}^{D},\mathcal{G}_{2}^{D},\dots\}$, recall maximum difference $=\max_{\mathcal{G}_{i}^{D}\in\mathcal{G}^{D}}|\text{Recall}(\mathcal{G}_{i}^{D})-\text{Recall}(\mathcal{G}^{D})|$.

Table 5 displays the recall maximum difference of each examined de-identification method along each dimension. These methods’ behaviors here are similar to their bias measured by recall equality difference in Table 2. Methods that attain the lowest recall equality difference still perform well in terms of recall maximum difference.

Method	Overall Performance ($\uparrow$)			Bias along Dimensions ($\downarrow$)
	Precision	Recall	F1	Gender	Race	Popularity	Decade
spaCy	0.917$\pm$0.001	0.629$\pm$0.001	0.746$\pm$0.001	0.002${}^{*}\pm$0.001	0.013${}^{*}\pm$0.002	0.028${}^{*}\pm$0.002	0.007${}^{*}\pm$0.002
Stanza	0.678$\pm$0.001	0.881$\pm$0.001	0.766$\pm$0.001	0.002${}^{*}\pm$0.001	0.016${}^{*}\pm$0.002	0.011${}^{*}\pm$0.001	0.005${}^{*}\pm$0.001
flair	0.920$\pm$0.001	0.974$\pm$0.000	0.946$\pm$0.000	0.003${}^{*}\pm$0.000	0.006${}^{*}\pm$0.001	0.008${}^{*}\pm$0.001	0.002${}^{*}\pm$0.000
Amazon	0.923$\pm$0.001	0.925$\pm$0.001	0.924$\pm$0.001	0.005${}^{*}\pm$0.001	0.022${}^{*}\pm$0.001	0.032${}^{*}\pm$0.001	0.001$\pm$0.001
Microsoft	0.664$\pm$0.001	0.960$\pm$0.001	0.785$\pm$0.001	0.003${}^{*}\pm$0.001	0.023${}^{*}\pm$0.001	0.010${}^{*}\pm$0.001	0.006${}^{*}\pm$0.001
Google	0.609$\pm$0.001	0.869$\pm$0.001	0.716$\pm$0.001	0.009${}^{*}\pm$0.001	0.025${}^{*}\pm$0.001	0.014${}^{*}\pm$0.002	0.010${}^{*}\pm$0.001
NeuroNER	0.946$\pm$0.001	0.944$\pm$0.001	0.945$\pm$0.000	0.001$\pm$0.001	0.045${}^{*}\pm$0.001	0.026${}^{*}\pm$0.001	0.002$\pm$0.001
Philter	0.227$\pm$0.001	0.794$\pm$0.001	0.353$\pm$0.001	0.000$\pm$0.001	0.000$\pm$0.001	0.003${}^{*}\pm$0.002	0.000$\pm$0.001
MIST	0.474$\pm$0.001	0.751$\pm$0.001	0.581$\pm$0.001	0.013${}^{*}\pm$0.001	0.022${}^{*}\pm$0.002	0.017${}^{*}\pm$0.002	0.003${}^{*}\pm$0.002
GPT-4	0.980$\pm$0.000	0.980$\pm$0.000	0.980$\pm$0.000	0.006${}^{*}\pm$0.000	0.001$\pm$0.000	0.002${}^{*}\pm$0.001	0.007${}^{*}\pm$0.001

Table 6. Overall performance (higher is better), bias along demographic dimensions (lower is better), and the associated bootstrapped standard error of the examined de-identification methods and GPT-4. We measure the bias with recall equality difference and bold GPT-4’s scores when they are among the best two in each column. In particular, GPT-4 achieves the highest precision, recall, and F1 and beats the runner-up strongly. Moreover, the asterisk next to a bias score indicates a statistically significant difference in performance at an adjusted significance level ($5\%$ for gender, $0.833\%$ for race, $1.667\%$ for popularity and decade). GPT-4 delivers the lowest bias among the non-rule-based methods along the dimensions of race and popularity.

Figure 9. Recall and 95% bootstrapped confidence interval on polysemy first names associated with three racial groups by each examined de-identification method and GPT-4. The increase in recall illustrated by the lighter color bar refers to the partially correct de-identification of non-polysemy last names. Unlike the other methods, GPT-4 is robust to polysemy names across the three considered racial groups with almost no performance drop.

Appendix B GPT-4

To investigate the effectiveness of GPT-4 (OpenAI, 2023) in clinical record de-identification, we assess GPT-4-0613 via Azure OpenAI Service (azu, nd) after opting out of human review of the data. More specifically, we prompt GPT-4 with a simple instruction (“Identify the names in the following clinical note. Output names only separated by commas.”), followed by each of the 16,000 evaluation notes constructed in Section 3. We then locate the occurrences of the generated names by using pattern matching between GPT-4’s output and the corresponding input note.

The last row in Table 6 illustrates the overall performance of GPT-4 and its bias along the four demographic dimensions. In particular, GPT-4 is notably effective in de-identifying names in clinical notes, achieving the highest precision, recall, and F1 among the tested baselines and beating the second-best method (i.e., flair) by $0.024$ in F1. Furthermore, GPT-4 also delivers the lowest bias (i.e., recall equality difference) among the non-rule-based de-identification methods along the dimensions of race and name popularity, where most other methods suffer. This result is further supported by the negligible gaps in recall among the racial and popularity groups considered in Figure 10 (b) and (c), respectively. However, according to our hypothesis tests, GPT-4 still acts differently with statistical significance along the dimensions of gender and decade of name popularity, which should be a call for further investigation for this commercial service.

In addition, we examine the effect of polysemy names and context consistency on GPT-4’s performance in Figures 9 and 11, respectively. Unlike most other tested methods, GPT-4 is robust to polysemy names across the three considered racial groups with almost no performance drop, as shown in the rightmost column of Figure 9. On the other hand, similar to spaCy, GPT-4 also performs better on names misaligned with the gender implied by the context, which suggests that GPT-4 probably relies more on the memorized names, given its tremendous pre-training corpus, instead of the local context when inferring the occurrence of names.

These findings indicate that large language models (LLMs) like GPT-4 can serve as an effective tool for de-identifying clinical notes (Agrawal et al., 2022) but require further improvement in terms of mitigating the bias along the demographic dimensions of gender and decade of name popularity. We leave the development of more advanced LLM-based de-identification methods to future work. This research project has benefitted from the Microsoft Accelerate Foundation Models Research (AFMR) grant program through which leading foundation models hosted by Microsoft Azure, along with access to Azure credits, were provided to conduct the research.

Figure 10. Recall and 95% bootstrapped confidence interval of the demographic groups along each dimension by each examined de-identification method and GPT-4. Unlike the other methods, GPT-4 demonstrates negligible disparities in performance along the dimensions of race and popularity but more significant performance gaps along the dimensions of gender and decade.

Figure 11. Difference in recall and 95% bootstrapped confidence interval between names that are consistent and inconsistent with the genders suggested by the local context. A positive recall difference means that performance was best when there was gender consistency, while a negative recall difference means that performance was best when there was gender inconsistency. Notably, GPT-4 performs better on names that are inconsistent with the gender suggested by the context.