Examination of Racial Bias in Alpha Omega Alpha Inductions: A Single-Center 15-Year Retrospective Study.

Importance: With USMLE Step 1 becoming pass/fail, subjective clinical evaluations will hold greater weight in residency applications. However, no longitudinal studies exist that examine the role of race in clinical success during medical training. Objective: Utilizing Alpha Omega Alpha (AOA) as an objective marker of clinical achievement, I investigated the relationship between race and AOA membership at Northwestern University Feinberg School of Medicine over a span of 15 years. Design, Setting, and Participants: To accomplish this retrospective, single center, multi-year cohort study, names of all Feinberg graduates between 2003 and 2018 were collected via the school public website. Images for each student were gathered by social media, assigned an ethnic identity by a blind evaluator, and confirmed by an unblinded evaluator. Finally, each name was verified against the AOA database to determine membership status. Main Outcomes and Measures: AOA membership among medical students of various racial groups at Feinberg. Results: From a 2,466 student body, there were 546 (22.1%) Eastern/Southeastern Asian, 123 (5.0%) African-American, 102 (4.1%) Hispanic/Latino, 399 (16.2%) South Asian, 59 (2.4%) Other, and 1205 (48.9%) Caucasian students, with 32 (1.3%) exclusions. Within this collective group, 428 students were inducted to AOA: 62 (14.5%) Eastern/Southeastern Asian, 4 (0.9%) African-American, 10 (2.3%) Hispanic/Latino, 70 (16.4%) South Asian, 10 (2.3%) Other, and 270 (63.1%) Caucasian students, with 2 (0.5%) exclusions. By class/year, the percentage of Caucasians inducted into AOA were higher than the class percent in 15 out of 16 classes, compared to 1 by Eastern/Southeastern Asians and 7 by South Asians. Odds ratio analysis demonstrated Eastern/Southeastern Asian (OR, 0.44; 99.67% CI, 0.28 - 0.69) and African-American (OR, 0.12; 99.67% CI, 0.03 - 0.53) students were at disadvantage relative to Caucasians for AOA membership. Conclusion: I revealed Eastern/Southeastern Asian and African-American students were statistically less likely to be selected for AOA compared to Caucasian counterparts. Additionally, Eastern/Southeastern Asian students were under-represented almost every year despite being the most represented minority demographic. These results demonstrate subjective bias in AOA membership for both under- and over-represented minorities and suggests the recent Step I paradigm shift may disproportionately affect certain students over others.


INTRODUCTION
Starting January 2022, USMLE Step 1 will become a pass-fail exam, thereby raising the value of subjective metrics in the residency application. As a result, non-cognitive disparities that have been traditionally entrenched within academia may become greater emphasized. For instance, racial inequality exists in many aspects of healthcare, including research funding 1 and the workforce 2 . Yet how these racial barriers manifest and affect medical education is complicated and has been the subject of extensive investigation. Unfortunately, the sheer magnitude of diverse studies on this topic often complicates interpretation.
One underexplored area that may potentially serve as a simplified gestalt of these biases is the relationship between race and membership to Alpha Omega Alpha (AOA) -an exclusive honor society reserved for the best of each class. Members are highly sought after by all residencies and therefore possess innumerable career and income advantages 3,4 , making induction a competitive goal for many medical students. Although multiple factors exist in determining membership eligibility, by far the most important is clinical grades 5,6 . Since non-cognitive bias is most influential within this realm 7,8 , I believe an examination of ethnic trends in AOA membership can result in objective snapshots of racial disparity in medical education. Indeed, a recent study by Boatright et al. demonstrated significant bias against African-and Asian-American students 9 and ultimately led to the closure of one AOA chapter 10 . While powerful, their study was only a one-year cross-sectional analysis, which precludes definitive conclusions about trends over time.
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted April 29, 2020. . Here, I performed the first single-center, multi-year study to examine longitudinal AOA membership trends among major ethnic groups. I find that African-American and is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted April 29, 2020. . https://doi.org/10.1101/2020.04. 22.20075622 doi: medRxiv preprint he/she became a member as resident rather than a medical graduate). This entire process yielded a total number of 2466 total students and 428 AOA inductees (eTable 1).
Ethics. I did not utilize an IRB approval form because all derived data for analysis were openly available within the public domain.
Race Determination. I performed ethnic assignments using a combination of subjective evaluation and genealogy confirmation rather than objective AAMC-based data primarily because the latter does not distinguish between South (S) Asian and E/SE Asian students. I believe the collective grouping of two over-represented ( Figure 2, eTable 1) but culturally distinct minorities is inaccurate and does not reflect differences that may exist between the two groups 15 . For that reason, anonymized AAMC-based data would not be able to provide me with the appropriate information.
An evaluator blinded to the study's purpose was instructed to utilize social media (Facebook, Linkedin, Doximity, and various medical websites) to search and assign each student by ethnicity (E/SE Asian, S Asian, African American, Hispanic/Latino, Caucasian, or Other) via subjective evaluation. Of note, Native Hawaiian and Pacific Islander were included in the "E/SE Asian" category (due to low total number) and all Middle-Eastern nationalities were designated as "Other".
Another evaluator (unblinded to the study's purpose) verified assignment accuracy via five consecutive iterations of subjective evaluation and confirmation with Forebears -a . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted April 29, 2020. . genealogical search engine (http://Forebears.io). Students with any identification problems in this process were automatically excluded. Half-Caucasian students were assigned to their minority ethnicity (i.e. E/SE Asian-Caucasian students were assigned as E/SE Asian) as it is well documented children of white/non-white marriages tend to identify with the minority race 16 . For the same reason, racially heterogeneous non-Caucasian students were excluded. The breakdown of these results by year is seen in eTable 2.

Statistical Analysis. For each class between 2003 to 2018, I measured and graphed
the difference between the percentage of AOA members per ethnic group and the percentage per collective class (i.e. if in one specific year, 20% of an ethnic group were AOA versus 15% within the overall class, the difference for said ethnic group that year would be +5%).
To assess the association between race and AOA membership, I performed a series of odds ratio pair-wise comparisons with Bonferroni correction for multiple testing (15 unique tests). For all analyses, p < 0.0033 (0.05/15) was considered significant and all confidence intervals were set at 99.67%. Graphs were created with Graphpad Prism 7 while odds ratios were calculated using MedCalc.

RESULTS
Using publicly-accessible information from the Feinberg website 11 , I reviewed the ethnicities of 2,466 students over the span of 15 years (and 16 classes) at the . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted April 29, 2020.  . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted April 29, 2020. . https://doi.org/10.1101/2020.04. 22.20075622 doi: medRxiv preprint Their AOA disparity may thus be the manifestation of a combination of these barriers.
On the other hand, much less is known about Asian American medical students' performances in medical education, partially because over-representation may preclude serious reflection 19 . One possible explanation for these results is that E/SE Asian students, by means of cultural habit, may have lower inclination for leadership activities and may lose nominations points because of it. However, Boatright et al. found no association between AOA membership and leadership 9 . Perhaps more insidiously, both ethnic groups may be victims of subtle discrimination: while official criteria for AOA medical student membership require a conglomeration of scholastic achievement, leadership prowess, and consummate professionalism, many AOA chapters place disproportionately high emphasis on clinical grades as selection criteria 5 . For example, Feinberg uses clinical performance as the sole criteria for determining the initial pool of eligible candidates 6 . These subjective evaluations have consistently been shown to be influenced by a multitude of non-cognitive variables 7,8,25 . Future studies correlating different aspects of subjective clinical grades (i.e. clinical performance evaluations and OSCEs) in different ethnic groups will be warranted to verify this theory.
A collective decision by the NBME, USMLE, and FSMB was recently made to alter USMLE Step 1 reporting from a three-digit score to a pass-fail outcome starting January 2022. The primary logic behind this change was to encourage more holistic evaluations from residency programs and to alleviate the student mental health burden associated with this examination. However, since scores have traditionally been markers of academic excellence, this well-intended action -in conjunction with growing numbers of . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted April 29, 2020.  9 . Another weakness of this study was determining student ethnicity via subjective evaluation, which may be error-prone because it is not self-reported information. I employed this approach because I believed AAMC's integration of S and E/SE Asian students as a collective group is inaccurate, considering their individual demographics at Feinberg. For the same reason, I was not able to control for other co-variables, such as Step I/II scores, extracurricular activities, or Gold Humanism membership (my analysis technique precludes anonymization), thus decreasing the accuracy of this study. Lastly, due to experimental design, some inferences may not be accurate and may skew results. For example, categorization of several students as Middle-Eastern (i.e. Other) versus S Asian presented some issue, given the proximity of both regions to one another. In addition, assignment of half-Caucasian students as minorities is another assumption that may be unfounded.
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted April 29, 2020. . https://doi.org/10.1101/2020.04. 22.20075622 doi: medRxiv preprint Given traditional institutional disadvantages with African-Americans 22,27 and the recent evidence of racial discrimination against Asian-Americans at Harvard 28 and other institutional settings 29 , my study raises awareness and sets clear precedence for future investigation. Not only have I reproduced the well-known bias against underrepresented minorities 9 but I also uncover inherent disadvantages for certain overrepresented minorities -trends that would not have been easily discerned using traditional analytical methods. These results demonstrate the need for AAMC and AOA chapters to acknowledge/address current demographic standards and implicit biases respectively moving forward.

AUTHOR CONTRIBUTIONS
R.G. designed the study, gathered the data, and wrote the manuscript. An external evaluator performed the initial assessment of the data while R.G. was responsible for the subsequent iterations. R.G. is responsible for the integrity of the entire study.
Northwestern University's Biostatistics Collaboration Center provided statistical consultation.
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted April 29, 2020. . Figure 1. Work-flow of information acquisition and processing. "Forebears" is a genealogy web database. "Evaluation" refers to the subjective determination of ethnicity by an evaluator.   Significance was p < 0.0033 (Bonferroni correction). Abbreviations: "E/SE Asian", Eastern/Southeastern Asian; "AA", African-American; "H/L", Hispanic/Latino; "S Asian", South Asian.

FIGURE LEGENDS
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted April 29, 2020.  20  30  25  24  25  31  26  28  25  28  27  30  25  23  28  428