Hostname: page-component-cd9895bd7-lnqnp Total loading time: 0 Render date: 2024-12-27T03:01:35.088Z Has data issue: false hasContentIssue false
  • English
  • Français

Mitigating Racial Bias in Machine Learning

Published online by Cambridge University Press:  04 March 2022

Kristin M. Kostick-Quenet ,
I. Glenn Cohen ,
Sara Gerke ,
Bernard Lo ,
James Antaki ,
Faezah Movahedi ,
Hasna Njah ,
Lauren Schoen ,
Jerry E. Estep  and
J.S. Blumenthal-Barby
Get access Rights & Permissions [Opens in a new window]

Abstract

When applied in the health sector, AI-based applications raise not only ethical but legal and safety concerns, where algorithms trained on data from majority populations can generate less accurate or reliable results for minorities and other disadvantaged groups.

Keywords

Algorithmic BiasRacial BiasMachine LearningArtificial IntelligenceEthics
Type
Symposium Articles
Information
Journal of Law, Medicine & Ethics , Volume 50 , Issue 1: Health Law and Anti-Racism: Reckoning and Response , Spring 2022 , pp. 92 - 100
Copyright
© 2022 The Author(s)

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Friedman, B. and Nissenbaum, H., “Bias in Computer Systems,” 1996, available at <https://nissenbaum.tech.cornell.edu/papers/Bias%20in%20Computer%20Systems.pdf> (last visited December 6, 2021).+(last+visited+December+6,+2021).>Google Scholar
Caliskan, A., Bryson, J.J., and Narayanan, A., “Semantics Derived Automatically from Language Corpora Contain Human-Like Biases,” Science 356, no. 6334 (2017): 183–186; A. Hadhazy, “Biased Bots: Artificial-Intelligence Systems Echo Human Prejudices,” Princeton University (April 2017): at 18; T. McSweeny, “Psychographics, Predictive Analytics, Artificial Intelligence, & Bots: Is The FTC Keeping Pace?” Georgetown Law Technology Review 2 (2018): 514, 516, 530; J. Buolamwini and T. Gebru, “Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification,” paper presented at Conference on Fairness, Accountability and Transparency, 2018; I.A. Hamilton, “Why It’s Totally Unsurprising That Amazon’s Recruitment AI Was Biased against Women,” Business Insider, 2018, available at <https://www.businessinsider.com/amazon-ai-biased-against-women-no-surprise-sandra-wachter-2018-10> (last visited December 6, 2021); T. Brennan, W. Dieterich, B. Ehret, “Evaluating the Predictive Validity of the COMPAS Risk and Needs Assessment System,” Criminal Justice and Behavior 36, no. 1 (2009): 21-40; J. Dressel and H. Farid. “The Accuracy, Fairness, and Limits of Predicting Recidivism,” Science Advances 4, no. 1 (2018): eaao5580.CrossRefGoogle ScholarPubMed
Ryan-Mosely, T., “The New Lawsuit That Shows Facial Recogntion Is Officially a Civil Rights Issue,” MIT Technology Review (2021).Google Scholar
Ross, C., “As the FDA Clears a Flood of AI Tools, Missing Data Raise Troubling Questions on Safety and Fairness, “ STAT, February 3, 2021, available at <https://www.statnews.com/2021/02/03/fda-clearances-artificial-intelligence-data/> (last visited December 6, 2021).Google Scholar
Obermeyer, Z., Powers, B., Vogeli, C., and Mullainathan, S., “Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations,” Science 366, no. 6464 (2019): 447–453.CrossRefGoogle Scholar
European Commission, “Laying Down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act) and Amending Certain Union Legislative Acts,” 2021, available at <https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A52021PC0206> (last visited December 6, 2021).+(last+visited+December+6,+2021).>Google Scholar
Ansell, D.A. and McDonald, E.K., “Bias, Black Lives, and Academic Medicine,” New England Journal of Medicine 372, no. 12 (2015): 1087–1089; Price, I., Nicholson, W., “Medical AI and Contextual Bias,” 2019.CrossRefGoogle ScholarPubMed
van Meeteren, J., Maltais, S., Dunlay, S.M., et al., “A Multi-Institutional Outcome Analysis of Patients Undergoing Left Ventricular Assist Device Implantation Stratified by Sex and Race,” Journal of Heart and Lung Transplantation 36, no. 1 (2017): 64–70.CrossRefGoogle Scholar
Molina, E.J., Shah, P., Kiernan, M.S., et al., “The Society of Thoracic Surgeons Intermacs 2020 Annual Report,” Annals of Thoracic Surgery 111, no. 3 (2021): 778–792.CrossRefGoogle Scholar
Goyal, P., Paul, T., Almarzooq, Z.I., et al., “Sex‐and Race‐Related Differences in Characteristics and Outcomes of Hospitalizations for Heart Failure with Preserved Ejection Fraction,” Journal of the American Heart Association 6, no. 4 (2017): e003330.CrossRefGoogle ScholarPubMed
Id. Google Scholar
Ueyama, H., Malik, A., Kuno, T., et al., “Racial Disparities in Hospital Outcomes after Left Ventricular Assist Device Implantation,” Journal of Cardiac Surgery 35, no. 10 (2020): 2633–2639.CrossRefGoogle ScholarPubMed
Lui, C., Fraser, C.D. III, Zhou, X., et al., “Racial Disparities in Patients Bridged to Heart Transplantation with Left Ventricular Assist Devices,” Annals of Thoracic Surgery 108, no. 4 (2019): 1122–1126.CrossRefGoogle ScholarPubMed
Ueyma et al., supra note 12.Google Scholar
Okoh, A.K., Selevanny, M., Singh, S., et al., “Racial Disparities and Outcomes of Left Ventricular Assist Device Implantation as a Bridge to Heart Transplantation,” ESC Heart Failure 7, no. 5 (2020): 2744–2751.CrossRefGoogle ScholarPubMed
Jaiswal, A., Truby, L.K., Chichra, A., et al., “Impact of Obesity on Ventricular Assist Device Outcomes,” Journal of Cardiac Failure 26, no. 4 (2020): 287–297.CrossRefGoogle ScholarPubMed
Okoh et al., supra note 15.Google Scholar
Khan, M.S., Yuzefpolskaya, M., Memon, M.M., et al., “Outcomes Associated with Obesity in Patients Undergoing Left Ventricular Assist Device Implantation: A Systematic Review and Meta-analysis,” Asaio Journal 66, no. 4 (2020): 401–408.CrossRefGoogle ScholarPubMed
Ueyma et al., supra note 12.Google Scholar
van Meeteren et al., supra note 8; Itoh, A., “Impact of Age, Sex, Therapeutic Intent, Race and Severity of Advanced Heart Failure on Short-Term Principal Outcomes in the MOMENTUM 3 Trial,” Journal of Heart and Lung Transplantation 37, no. 1 (2018): 7–14.Google Scholar
Id.; Ueyma et al., supra note 12.Google Scholar
Bowles, E.J.A., Wellman, R., Feigelson, H.S., et al., “Risk of Heart Failure in Breast Cancer Patients after Anthracycline and Trastuzumab Treatment: A Retrospective Cohort Study,” Journal of the National Cancer Institute 104, no. 17 (2012): 1293–1305.CrossRefGoogle ScholarPubMed
Okoh et al., supra note 15.Google Scholar
Sheikh, F.H., Ravichandran, A.K., Goldstein, D.J., et al., “Impact of Race on Clinical Outcomes after Implantation with a Fully Magnetically Levitated Left Ventricular Assist Device: An Analysis From the MOMENTUM 3 Trial,” Circulation: Heart Failure 14, no. 10 (2021): e008360.Google ScholarPubMed
Ueyma et al., supra note 12.Google Scholar
Tsiouris, A., Brewer, R.J., Borgi, J., Nemeh, H., Paone, G., and Morgan, J.A., “Continuous-Flow Left Ventricular Assist Device Implantation as a Bridge to Transplantation or Destination Therapy: Racial Disparities in Outcomes,” Journal of Heart and Lung Transplantation 32, no. 3 (2013): 299–304; X. Wang, A.A. Luke, J.M. Vader, T.M. Maddox, K.E. Joynt Maddox , “Disparities and Impact of Medicaid Expansion on Left Ventricular Assist Device Implantation and Outcomes,” Circulation: Cardiovascular Quality and Outcomes 13, no. 6 (2020): e006284.CrossRefGoogle ScholarPubMed
Okoh et al., supra note 15; Tsiouris et al., supra note 26.Google Scholar
Ueyma et al., supra note 12.Google Scholar
Id.; Breathett, K., Yee, E., Pool, N., et al., “Does Race Influence Decision Making for Advanced Heart Failure Therapies?” Journal of the American Heart Association 8, no. 22 (2019): e013592.CrossRefGoogle ScholarPubMed
Bui, Q.M., Allen, L.A., LeMond, L., Brambatti, M., Adler, E., “Psychosocial Evaluation of Candidates for Heart Transplant and Ventricular Assist Devices: Beyond the Current Consensus,” Circulation: Heart Failure 12, no. 7 (2019): e006058.Google ScholarPubMed
Steinberg, R.S., Nayak, A., Dong, T., Morris, A.A., “Primary Caregiver Relationships for Advanced Heart Failure Therapy Differ Based on Sex and Race and Predict Eligibility,” Journal of Cardiac Failure 26, no. 10 (2020): S10.CrossRefGoogle Scholar
Takshi, S., “Unexpected Inequality: Disparate-Impact from Artificial Intelligence in Healthcare Decisions,” Journal of Law and Health 34, no. 2 (2021): 215.Google ScholarPubMed
Selbst, A.D., “Negligence and AI/MLs Human Users,” Boston University Law Review 100, no. 1315 (2020): 1354–1360.Google Scholar
Price, W.N. II, Sachs, R., and Eisenberg, R.S., “New Innovation Models in Medical AI,” Washington University Law Review (forthcoming 2022); S. Gerke, B. Babic, T. Evgeniou, and I.G. Cohen, “The Need for a System View to Regulate Artificial Intelligence/Machine Learning-Based Software as Medical Device,” npj: Digital Medicine 3, no. 1 (2020): 53.Google Scholar
Ross, supra note 4.Google Scholar
Federal Trade Commission. Aiming for truth, fairness, and equity in your company’s use of AI. 2021.Google Scholar
Mathews, M.E.A.W., “New York Regulator Probes UnitedHealth Algorithm for Racial Bias,” Wall Street Journal, Octover 26, 2019.Google Scholar
White House, Executive Order on Maintaining American Leadership in Artificial Intelligence, 2019.Google Scholar
Cohen, I.G., Evgeniou, T., Gerke, S., Minssen, T., “The European Artificial Intelligence Strategy: Implications and Challenges for Digital Health,” The Lancet Digital Health 2, no. 7 (2020): e376-e379; AI HLEG, Policy and Investment Recommendations for Trustworthy AI, 2019; European Commission, White Paper on Artificial Intelligence—A European Approach to Excellence and Trust, 2020.CrossRefGoogle ScholarPubMed
SPDP Commission, Infocomm Media Development Agency Proposed Model AI Governance Framework, 2020.Google Scholar
Microsoft, Microsoft AI principles, 2021.Google Scholar
Partnership on AI, Tenets, June 15, 2021.Google Scholar
European Commision, supra note 6.Google Scholar
See supra note 7; also Gerke et al., supra note 34.Google Scholar
Gerke et al., supra note 34.Google Scholar
Nayak, A., Hicks, A.J., and Morris, A.A. , “Understanding the Complexity of Heart Failure Risk and Treatment in Black Patients,” Circulation: Heart Failure 13, no. 8 (2020): e007264.Google ScholarPubMed
Sepucha, K.R., Abhyankar, P., Hoffman, A.S., et al., “Standards for Universal Reporting of Patient Decision Aid Evaluation Studies: The Development of SUNDAE Checklist,” BMJ Quality & Safety 27, no. 5 (2018): 380–388.CrossRefGoogle ScholarPubMed
Lee, N.T., Resnick, P., and Barton, G., “Algorithmic Bias Detection and Mitigation: Best Practices and Policies to Reduce Consumer Harms,” Brookings Institute, 2019.Google Scholar
Price et al., supra note 7.Google Scholar
Friedman and Nissenbaum, supra note 1.Google Scholar
Lui, supra note 7.Google Scholar
Gebru, T., Morgenstern, J., Vecchione, B., et al. “Datasheets for Datasets,” 2018, Cornell University, available at <https://arxiv.org/abs/1803.09010> (last visited December 6, 2021).+(last+visited+December+6,+2021).>Google Scholar
Moss, E., Watkins, E.A., Singh, S., Elish, M.C., and Metcalf, J., “Assembling Accountability: Algorithmic Impact Assessment for the Public Interest,” 2021, available at <https://datasociety.net/library/assembling-accountability-algorithmic-impact-assessment-for-the-public-interest/> (last visited December 6, 2021).+(last+visited+December+6,+2021).>Google Scholar
Flanagin, A., Frey, T., Christiansen, S.L., “Committee AMoS. Updated Guidance on the Reporting of Race and Ethnicity in Medical and Science Journals,” JAMA 326, no. 7 (2021): 621–627.CrossRefGoogle Scholar
Cunningham, P. and Delany, S.J., “Algorithmic Bias and Regularisation in Machine Learning,” 2020, Cornell University, available at <https://arxiv.org/abs/2005.09052arXiv preprint arXiv:2005.09052> (last visited December 6, 2021).+(last+visited+December+6,+2021).>Google Scholar