Hostname: page-component-78c5997874-t5tsf Total loading time: 0 Render date: 2024-11-14T13:20:12.225Z Has data issue: false hasContentIssue false
  • English
  • Français

Biases in machine learning models and big data analytics: The international criminal and humanitarian law implications

Published online by Cambridge University Press:  18 March 2021

Nema Milaninia
Get access Rights & Permissions [Opens in a new window]

Abstract

Advances in mobile phone technology and social media have created a world where the volume of information generated and shared is outpacing the ability of humans to review and use that data. Machine learning (ML) models and “big data” analytical tools have the power to ease that burden by making sense of this information and providing insights that might not otherwise exist. In the context of international criminal and human rights law, ML is being used for a variety of purposes, including to uncover mass graves in Mexico, find evidence of homes and schools destroyed in Darfur, detect fake videos and doctored evidence, predict the outcomes of judicial hearings at the European Court of Human Rights, and gather evidence of war crimes in Syria. ML models are also increasingly being incorporated by States into weapon systems in order to better enable targeting systems to distinguish between civilians, allied soldiers and enemy combatants or even inform decision-making for military attacks.

The same technology, however, also comes with significant risks. ML models and big data analytics are highly susceptible to common human biases. As a result of these biases, ML models have the potential to reinforce and even accelerate existing racial, political or gender inequalities, and can also paint a misleading and distorted picture of the facts on the ground. This article discusses how common human biases can impact ML models and big data analytics, and examines what legal implications these biases can have under international criminal law and international humanitarian law.

Keywords

machine learningbig datainternational criminal lawinternational humanitarian lawbiasesInternational Criminal Court
Type
Artificial intelligence, autonomous weapon systems and their governance
Information
International Review of the Red Cross , Volume 102 , Issue 913: Digital technologies and war , April 2020 , pp. 199 - 234
Copyright
Copyright © The Author(s), 2021. Published by Cambridge University Press on behalf of the ICRC.

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

The views expressed herein are those of the author alone and do not necessarily reflect the views of the ICC Office of the Prosecutor or Google. The author would like to thank Nasrina Bargzie, Alexa Koenig, Matthew Cross, Beth Van Schaack, María Sol Beker, Jérôme de Hemptinne, Nikki Ahmadi, Gayane Khechoomian, Yulia Nuzban, and the editors of the International Review of the Red Cross, Bruno Demeyere, Sai Venkatesh and Ash Stanley-Ryan, for their considerably important and insightful input and feedback. The article is written without any first-hand knowledge of any of the investigations described herein.

References

1 In the Ratko Mladić case at the International Criminal Tribunal for the former Yugoslavia (ICTY), for example, 377 witnesses were called and over 10,000 exhibits, including videos, forensic reports, photographs, audio recordings and handwritten documents, were admitted at trial. ICTY, “Case Information Sheet: Ratko Mladić”, 2020, available at: https://bit.ly/39CgOaa (all internet references were accessed in January 2021).

2 For a definition of “big data”, see Council of Europe, Guidelines on the Protection of Individuals with Regard to the Processing of Personal Data in a World of Big Data, 23 January 2017, n. 3, available at: https://bit.ly/34zMcVn (“The term ‘Big Data’ usually identifies extremely large data sets that may be analysed computationally to extract inferences about data patterns, trends, and correlations”).

3 Unstructured data can be human-generated or machine-generated. Some examples of unstructured human-generated data include text files, emails, social media data and mobile data. Examples of unstructured machine-generated data include satellite imagery, scientific data, digital surveillance and sensor data. See UN Secretary-General, Data Strategy for Action by Everyone, Everywhere (2020–2022), 2020, p. 81, available at: https://bit.ly/3iqCdY2.

4 This is in contrast to traditional structured data, like bank transactions, which are typically highly organized and formatted in a way that makes them easily searchable in relational databases. Ibid., p. 81.

5 Ibid., p. 80; International Committee of the Red Cross (ICRC), Artificial Intelligence and Machine Learning in Armed Conflict: A Human-Centred Approach, Geneva, 6 June 2019, pp. 1, 10, available at: https://bit.ly/3qtAODc.

6 Nikola Todorovic and Abhi Chaudhuri, “Using AI to Help Organizations Detect and Report Child Sexual Abuse Material Online”, The Keyword, 3 September 2018, available at: https://bit.ly/2HJx9Qi.

7 Mimi Onuoha, “Machine Learning Is Being Used to Uncover the Mass Graves of Mexico's Missing”, Quartz, 19 April 2017, available at: https://bit.ly/31PxFDo.

8 Jay D. Aronson, Shicheng Xu and Alex Hauptmann, Video Analytics for Conflict Monitoring and Human Rights Documentation: Technical Report, Carnegie Mellon University, July 2015, available at: https://bit.ly/2LXJhiH.

9 Annette Vestby and Jonas Vestby, “Machine Learning and the Police: Asking the Right Questions”, Policing: A Journal of Policy and Practice, 14 June 2019, p. 5, available at: https://bit.ly/3nVyLp8.

10 Karen Hao, “Human Rights Activists Want to Use AI to Help Prove War Crimes in Court”, MIT Technology Review, 25 June 2020, available at: https://bit.ly/3e9M1mX.

11 Congressional Research Service, Artificial Intelligence and National Security, 10 November 2020, p. 10, available at: https://bit.ly/2XNcEH5.

12 Theresa Hitchens, “Air Force Expands 5G as It Transforms to Multi-Domain Ops: Donovan”, Breaking Defense, 4 September 2019, available at: https://breakingdefense.com/2019/09/air-force-expands-5g-as-it-transforms-to-multi-domain-ops-donovan/.

13 Michael N. Schmitt, “Autonomous Weapons Systems and International Humanitarian Law: A Reply to the Critics”, Harvard National Security Journal: Features Online, 5 February 2013, p. 28, available at: https://bit.ly/3ip5pyh.

14 Facebook, Facebook's Civil Rights Audit – Final Report, 8 July 2020, p. 76, available at: https://bit.ly/3nVlCwk.

15 ICRC, above note 5, p. 10.

16 Tom Simonite, “Machines Taught by Photos Learn a Sexist View of Women”, Wired, 21 August 2017, available at: https://bit.ly/3qvxaIm.

17 ICC, Prosecutor v. Mahmoud Mustafa Busayf Al-Werfalli, Case No. ICC-01/11-01/17, Warrant of Arrest (Pre-Trial Chamber I), 15 August 2017.

18 See, for example, Emma Irving, “And So It Begins… Social Media Evidence in an ICC Arrest Warrant”, Opinio Juris, 17 August 2017, available at: https://bit.ly/3kvEtNI.

19 Rome Statute of the International Criminal Court, UN Doc. A/CONF.183/9, 17 July 1998 (entered into force 1 July 2002) (Rome Statute), Art. 58(1).

20 Ibid., Art. 66(3).

21 All that is required is that the interpretation of the evidence advanced by the Prosecution is a reasonable one. ICC, Prosecutor v. Omar Hassan Ahmad Al Bashir, Case No. ICC-02/05-01/09, Decision on the Prosecution's Application for a Warrant of Arrest against Omar Hassan Ahmad Al Bashir (Pre-Trial Chamber I), 4 March 2009, paras 32–34.

22 UN Institute for Disarmament Research, Algorithmic Bias and the Weaponization of Increasingly Autonomous Technologies, 2018, p. 3, available at: https://bit.ly/3nPmiTX.

23 The following list contains just a small selection of biases that are often uncovered in ML data sets. It is not intended to be exhaustive. Wikipedia's catalogue of cognitive biases enumerates over 100 different types of human bias that can affect our judgement and, in turn, ML models; see Wikipedia, “List of Cognitive Biases”, available at: https://en.wikipedia.org/wiki/List_of_cognitive_biases. See also Forensic Science Regulator, Cognitive Bias Effects Relevant to Forensic Science Investigations, 4 April 2018, available at: https://bit.ly/3bNOQe9.

24 Jeffrey Dastin, “Amazon Scraps Secret AI Recruiting Tool that Showed Bias against Women”, Reuters, 10 October 2018, available at: https://reut.rs/2HItB0B.

25 Katyal, Sonia K., “Private Accountability in the Age of Artificial Intelligence”, UCLA Law Review, Vol. 66, No. 1, 2019, p. 79Google Scholar; Bloch, Kate E., “Harnessing Virtual Reality to Prevent Prosecutorial Misconduct”, Georgetown Journal of Legal Ethics, Vol. 32, No. 1, 2019, p. 5Google Scholar.

26 Alafair S. Burke, “Improving Prosecutorial Decision Making: Some Lessons of Cognitive Science”, William & Mary Law Review, Vol. 47, No. 5, 2006, pp. 1603–1604.

27 Peter Margulies, “The Other Side of Autonomous Weapons: Using Artificial Intelligence to Enhance IHL Compliance”, in Ronald T. P. Alcala and Eric Talbot Jensen (eds), The Impact of Emerging Technologies on the Law of Armed Conflict, Oxford University Press, Oxford, 2019, pp. 147, 158–159.

28 A. S. Burke, above note 26, pp. 1594, 1596–1599; Alafair S. Burke, “Commentary: Brady's Brainteaser: The Accidental Prosecutor and Cognitive Bias”, Case Western Reserve Law Review, Vol. 57, No. 3, 2007, p. 578.

29 A. S. Burke, above note 26, pp. 1594, 1599–1601.

30 Ibid., pp. 1594, 1601–1602.

31 Ibid., p. 1614.

32 ICC, Situation in the People's Republic of Bangladesh/Republic of the Union of Myanmar, Case No. ICC-01/19, Decision Pursuant to Article 15 of the Rome Statute on the Authorisation of an Investigation into the Situation in the People's Republic of Bangladesh/Republic of the Union of Myanmar (Pre-Trial Chamber III), 14 November 2019.

33 See, for example, Human Rights Council, Report of the Independent International Fact-finding Mission on Myanmar, UN Doc. A/HRC/39/64, 27 August 2018; Médecins Sans Frontières, “No One Was Left”: Death and Violence against the Rohingya in Rakhine State, Myanmar, 9 March 2018, available at: https://bit.ly/3edvEFV.

34 ICC, Situation in the People's Republic of Bangladesh/Republic of the Union of Myanmar, Case No. ICC-01/19, Request for Authorisation of an Investigation Pursuant to article 15 (Pre-Trial Chamber III), 4 July 2019.

35 Human Rights Council, Annex to the Report of the Special Rapporteur on Extrajudicial, Summary or Arbitrary Executions: Investigation into the Unlawful Death of Mr. Jamal Khashoggi, UN Doc. A/HRC/41/CRP.1, 19 June 2019, para. 37.

36 ICC, OTP, Policy Paper on Preliminary Examinations, November 2013, paras 79, 80, 104, available at: https://bit.ly/3nXQ2y6. See also ICC, Proposed Programme Budget for 2021 of the International Criminal Court, ICC-ASP/19/10, 10 September 2020, para. 128, available at: https://bit.ly/2LxHkJZ.

37 US Central Command, “Summary of the Airstrike on the MSF Trauma Center in Kunduz, Afghanistan on October 3, 2015”, 29 April 2016, p. 389; Matthew Rosenburg, “Pentagon Details Chain of Errors in Strike on Afghan Hospital”, New York Times, 29 April 2016, available at: https://nyti.ms/3irFBBJ; P. Margulies, above note 27, pp. 149–150.

38 Ben Tarnoff, “Weaponised AI is Coming. Are Algorithmic Forever Wars Our Future?”, The Guardian, 11 October 2018, available at: https://bit.ly/3qz3hqT.

39 Patrick Ball, “The Bigness of Big Data”, in Philip Alston and Sarah Knuckey (eds), The Transformation of Human Rights Fact-Finding, Oxford University Press, Oxford, 2015, pp. 425, 436–437.

40 Joann Stonier, “Fighting AI Bias – Digital Rights Are Human Rights”, Forbes, 19 March 2020, available at: https://bit.ly/35FGKzH.

41 Hanna Tolonen, Miika Honkala, Jaakko Reinikainen, Tommi Härkänen and Pia Mäkelä, “Adjusting for Non-Response in the Finnish Drinking Habits Survey”, Scandinavian Journal of Public Health, Vol. 47, No. 4, 2019, p. 470.

42 Andrea F. de Winter, Albertine J. Oldehinkel, René Veenstra, J. Agnes Brunnekreef, Frank C. Verhulst and Johan Ormel, “Evaluation of Non-Response Bias in Mental Health Determinants and Outcomes in a Large Sample of Pre-Adolescents”, European Journal of Epidemiology, Vol. 20, No. 2, 2005.

43 Sam Whitt, “Institutions and Ethnic Trust: Evidence from Bosnia”, Europe-Asia Studies, Vol. 62, No. 2, 2010.

44 Andrew D. Selbst, “Disparate Impact in Big Data Policing”, Georgia Law Review, Vol. 52, No. 1, 2017, pp. 109, 134–135.

45 Megan Price and Patrick Ball, “Big Data, Selection Bias, and the Statistical Patterns of Mortality in Conflict”, SAIS Review, Vol. 36, No. 1, 2014, p. 11.

46 Jay D. Aronson, “Mobile Phones, Social Media, and Big Data in Human Rights Fact-Finding: Possibilities, Challenges, and Limitations”, in P. Alston and S. Knuckey (eds), above note 39, pp. 441, 447.

47 Facebook, above note 14, p. 80.

48 UN Assistance Mission in Afghanistan, Afghanistan: Protection of Civilians in Armed Conflict, 2019, 2020, pp. 5–6, available at: https://bit.ly/3e8ObmQ (noting that “Anti-Government Elements continued to cause the majority (62 per cent) of civilian casualties in 2019”).

49 ICTY, Prosecutor v. Duško Tadić, Case No. IT-94-1-A, Decision on the Defence Motion for Interlocutory Appeal on Jurisdiction (Appeals Chamber), 2 October 1995, para. 70.

50 Rome Statute, above note 19, Art. 7.

51 Eirini Ntoutsi et al., “Bias in Data-Driven Artificial Intelligence Systems – An Introductory Survey”, Data Mining and Knowledge Discovery, 2019, p. 4, available at: https://bit.ly/3sCECmT. See also Jonathan Gordon and Benjamin Van Durme, “Reporting Bias and Knowledge Acquisition”, Proceedings of the 2013 Workshop on Automated Knowledge Base Construction, 2013, p. 25, available at: https://bit.ly/2LXoD2a (analyzing generally how reporting bias functions in artificial intelligence).

52 M. Price and P. Ball, above note 45, n. 4.

53 S. K. Katyal, above note 25, p. 72.

54 ICC, Situation in the Islamic Republic of Afghanistan, Case No. ICC-02/17, Request for Authorisation of an Investigation Pursuant to Article 15 (Pre-Trial Chamber II), 20 November 2017.

55 Randy Rieland, “Artificial Intelligence Is Now Used to Predict Crime. But Is It Biased?”, Smithsonian Magazine, 5 March 2018, available at: https://bit.ly/2HHg2Pf.

56 Hongyu Chen, Zhiqiang Zheng and Yasin Ceran, “De-Biasing the Reporting Bias in Social Media Analytics”, Production and Operations Management, Vol. 25, No. 5, 2015, p. 849.

57 Chrysanthos Dellarocas and Charles A. Wood, “The Sound of Silence in Online Feedback: Estimating Trading Risks in the Presence of Reporting Bias”, Management Science, Vol. 54, No. 3, 2008, p. 460.

58 Berkeley Protocol on Digital Open Source Investigations, HR/PUB/20/2, 2020 (Berkeley Protocol), pp. 11, 46, 55.

59 Mick P. Couper, “Is the Sky Falling? New Technology, Changing Media, and the Future of Surveys”, Survey Research Methods, Vol. 7, No. 3, 2013, pp. 145, 147.

60 Ricardo Baeza-Yates, “Bias on the Web”, Communications of the ACM, Vol. 61, No. 6, 2018, p. 54.

61 Stefan Wojcik and Adam Hughes, “Sizing Up Twitter Users”, Pew Research Center, 24 April 2019, available at: https://pewrsr.ch/38TfNeD.

62 Holly Young, “The Digital Language Divide”, The Guardian, available at: https://bit.ly/2Kn116q.; Web Technology Surveys, “Usage Statistics of Persian for Websites”, available at: https://bit.ly/2YN4DCk; Web Technology Surveys, “Usage Statistics of Pushto, Pashto for Websites”, available at: https://bit.ly/3oNtVuz.

63 Jill A. Dever, Ann Rafferty and Richard Valliant, “Internet Surveys: Can Statistical Adjustments Eliminate Coverage Bias?”, Survey Research Methods, Vol. 2, No. 2, 2008, p. 47.

64 Laura Silver and Courtney Johnson, “Majorities in Sub-Saharan Africa Own Mobile Phones, but Smartphone Adoption Is Modest”, Pew Research Center, 9 October 2018, available at: https://pewrsr.ch/3nVR6mj.

65 Jacob Poushter, Caldwell Bishop and Hanyu Chwe, “Smartphone Ownership on the Rise in Emerging Economies”, Pew Research Center, 19 June 2018, available at: https://pewrsr.ch/2Ncgkjr.

66 Pew Research Center, “Mobile Fact Sheet”, 12 June 2019, available at: https://pewrsr.ch/2LMq9El.

67 Nils B. Weidmann, “A Closer Look at Reporting Bias in Conflict Event Data”, American Journal of Political Science, Vol. 60, No. 1, 2015, p. 211.

68 Ibid., p. 217.

69 Timothy M. Jones, Peter Van Aelst and Rens Vliegenthart, “Foreign Nation Visibility in U.S. News Coverage: A Longitudinal Analysis (1950–2006)”, Communication Research, Vol. 40, No. 3, 2013, p. 417.

70 N. B. Weidmann, above note 67, p. 216 and Appendix D.

71 US Federal Trade Commission, Big Data: A Tool for Inclusion or Exclusion?, 2016, p. 9, available at: https://bit.ly/31Or102. See also Martin Frické, “Big Data and Its Epistemology”, Journal of the Association for Information Science and Technology, Vol. 66, No. 4, 2015, p. 659.

72 M. Price and P. Ball, above note 45, pp. 10–11.

73 See, for example, ICTY, Prosecutor v. Nikola Šainović et al., Case No. IT-05-87-A, Judgment (Appeals Chamber), 23 January 2014, paras 614–634 (upholding the Trial Chamber's finding that a “discernible pattern” of forcible transfer evidenced the existence of a common plan to displace the Kosovo Albanian population).

74 See, for example, ibid., paras 988, 1784; Prosecutor v. Jean-Pierre Bemba Gombo et al., Case No. ICC-01/05-01/13, Judgment Pursuant to Article 74 of the Statute (Trial Chamber VII), 19 October 2016, paras 702, 707 (noting the pattern of the accused's conduct for the purposes of assessing their intent to commit the crime).

75 Berkeley Protocol, above note 58, p. 57.

76 UN Secretary-General, Conflict-Related Sexual Violence: Report of the Secretary-General, UN Doc. S/2019/280, 29 March 2019 (UNSG Report on Sexual Violence), para. 11. See also World Health Organization, Global and Regional Estimates of Violence against Women: Prevalence and Health Effects of Intimate Partner Violence and Non-Partner Sexual Violence, 20 October 2013, available at: https://bit.ly/3oXrFlp; Iness Ba and Rajinder S. Bophal, “Physical, Mental and Social Consequences in Civilians Who Have Experienced War-Related Sexual Violence: A Systematic Review (1981–2014)”, Public Health, Vol. 142, 10 September 2016; Gerald Schneider, Lilli Banholzer and Laura Albarracin, “Ordered Rape: A Principal–Agent Analysis of Wartime Sexual Violence in the DR Congo”, Violence Against Women, Vol. 21, No. 11, 2015; Tia Palermo, Jennifer Bleck and Amber Peterman, “Tip of the Iceberg: Reporting and Gender-Based Violence in Developing Countries”, American Journal of Epidemiology, Vol. 179, No. 5, 2014.

77 UNSG Report on Sexual Violence, above note 76, paras 31–34.

78 Ibid., paras 35–39. See also UN Panel of Experts on the Central African Republic, Final Report of the Panel of Experts on the Central African Republic Extended Pursuant to Security Council Resolution 2399 (2018), UN Doc. S/2018/1119, 14 December 2018, paras 164–167; Phuong N. Pham, Mychelle Balthazard and Patrick Vinck, “Assessment of Efforts to Hold Perpetrators of Conflict-related Sexual Violence Accountable in Central African Republic”, Journal of International Criminal Justice, Vol. 18, No. 2, 2020, pp. 394–395.

79 UNSG Report on Sexual Violence, above note 76, paras 54–59.

80 ICC, OTP, Policy Paper on Sexual and Gender-Based Crimes, June 2014, available at: https://bit.ly/3in5nHk (OTP Policy Paper on SGBC), para. 50.

81 S. K. Katyal, above note 25, pp. 80–81, citing Michael J. Bernstein, Steven G. Young and Kurt Hugenberg, “The Cross-Category Effect: Mere Social Categorization Is Sufficient to Elicit an Own-Group Bias in Face Recognition”, Psychological Science, Vol. 18, No. 8, 2007.

82 S. K. Katyal, above note 25, p. 81, citing S. Alex Haslam, Penny J. Oakes and John C. Turner, “Social Identity, Self-Categorization, and the Perceived Homogeneity of Ingroups and Outgroups: The Interaction Between Social Motivation and Cognition”, in Richard M. Sorrentino and Edward T. Higgins (eds), Handbook of Motivation and Cognition: The Interpersonal Context, Vol. 3, Guilford Press, New York, 1996.

83 Donald M. Taylor and Janet R. Doria, “Self-Serving and Group-Serving Bias in Attribution”, Journal of Social Psychology, Vol. 113, No. 2, 1981.

84 Kevin K. Fleming, Carole L. Bandy and Matthew O. Kimble, “Decisions to Shoot in a Weapon Identification Task: The Influence of Cultural Stereotypes and Perceived Threat on False Positive Errors”, Social Neuroscience, Vol. 5, No. 2, 2010.

85 Ibid., pp. 206, 219. See also B. Keith Payne and Joshua Correll, “Race, Weapons, and the Perception of Threat”, in Bertram Gawronski (ed.), Advances in Experimental Social Psychology, Vol. 62, Elsevier, Amsterdam, 2020, Chap. 1.

86 Moses Shayo and Asaf Zussman, “Judicial Ingroup Bias in the Shadow of Terrorism”, Quarterly Journal of Economics, Vol. 126, No. 3, 2011, p. 1447.

87 Ibid., p. 1483.

88 Jeffrey J. Rachlinski, Sheri Lynn Johnson, Andrew J. Wistrich and Chris Guthrie, “Does Unconscious Bias Affect Trial Judges?”, Notre Dame Law Review, Vol. 84, No. 3, 2009, pp. 1225–1226.

89 Ibid., p. 1223.

90 Ibid. But see p. 1223 (showing that when race is explicitly manipulated, judges show the capacity to treat defendants comparably).

91 Jeff Guo, “Researchers Have Discovered a New and Surprising Racial Bias in the Criminal Justice System”, Washington Post, 24 February 2016, available at: https://wapo.st/37Nz0hR; Briggs Depew, Ozkan Eren and Naci Mocan, “Judges, Juveniles and In-Group Bias”, Journal of Law and Economics, Vol. 60, No. 2, 2017.

92 Independent Expert Review of the International Criminal Court and the Rome Statute System: Final Report, 30 September 2020 (IER Report), para. 632, available at: https://bit.ly/2XSkA9Z.

93 Marc Perrin de Brichambaut, “ICC Statute Article 68”, Peking University Law School, Beijing, 17 May 2017, p. 9, available at: https://bit.ly/35SIYg4.

94 Masha Medvedeva, Michel Vols and Martijn Wieling, “Using Machine Learning to Predict Decisions of the European Court of Human Rights”, Artificial Intelligence and Law, Vol. 28, 2020; Conor O'Sullivan and Joeran Beel, “Predicting the Outcome of Judicial Decisions Made by the European Court of Human Rights”, 27th AIAI Irish Conference on Artificial Intelligence and Cognitive Science, 2019, available at: https://bit.ly/3nXEBGO.

95 Linda J. Skitka, Kathleen Mosier, Mark Burdick and Bonnie Rosenblatt, “Automation Bias and Errors: Are Crews Better Than Individuals?”, International Journal of Aviation Psychology, Vol. 10, No. 1, 2000, p. 86; Ric Simmons, “Big Data, Machine Judges, and the Legitimacy of the Criminal Justice System”, UC Davis Law Review, Vol. 52, No. 2, 2018, pp. 1109–1110; Mary L. Cummings, “Automation and Accountability in Decision Support System Interface Design”, Journal of Technology Studies, Vol. 32, No. 1, 2006, p. 25.

96 Danielle K. Citron, “Technological Due Process”, Washington University Law Review, Vol. 85, No. 6, 2008, p. 1272.

97 ICRC, Ethics and Autonomous Weapon Systems: An Ethical Basis for Human Control?, Geneva, 3 April 2018, p. 14, available at: https://bit.ly/3ioj3C5.

98 Chantal Grut, “The Challenge of Autonomous Lethal Robotics to International Humanitarian Law”, Journal of Conflict and Security Law, Vol. 18, No. 1, 2013, p. 19. See also Shin-Shin Hua, “Machine Learning Weapons and International Humanitarian Law: Rethinking Meaningful Human Control”, Georgetown Journal of International Law, Vol. 51, No. 1, 2019, p. 141.

99 Julia Angwin, Jeff Larson, Surya Mattu and Lauren Kirchner, “Machine Bias: There's Software Used across the Country to Predict Future Criminals. And It's Biased against Blacks”, ProPublica, 23 May 2016, available at: https://bit.ly/39GHiHK.

100 Wisconsin Supreme Court, State v. Loomis, 881 N.W.2d 749, 13 July 2016, pp. 770–771.

101 Aleš Završnik, “Criminal Justice, Artificial Intelligence Systems, and Human Rights”, ERA Forum, Vol. 20, No. 4, 2020; Katherine Freeman, “Algorithmic Injustice: How the Wisconsin Supreme Court Failed to Protect Due Process Rights in State v. Loomis”, North Carolina Journal of Law & Technology, Vol. 18, No. 5, 2016, pp. 97–98.

102 ICRC, Autonomous Weapon Systems: Implications of Increasing Autonomy in the Critical Functions of Weapons. Expert Meeting, Geneva, March 2016, p. 10, available at: https://bit.ly/35VHscW.

103 DoD, Summary of the 2018 Department of Defense Artificial Intelligence Strategy: Harnessing AI to Advance Our Security and Prosperity, 2018, p. 7, available at: https://bit.ly/2LKOSZN.

104 AP I includes provisions imposing limits on the use of weapons, means and methods of warfare and protecting civilians from the effects of hostilities. See Protocol Additional (I) to the Geneva Conventions of 12 August 1949, and relating to the Protection of Victims of International Armed Conflicts, 1125 UNTS 3, 8 June 1977 (entered into force 7 December 1978), in particular Part III, Section I, and Part IV, Section I, Chaps I–IV.

105 ICRC, “A Guide to the Legal Review of New Weapons, Means and Methods of Warfare: Measures to Implement Article 36 of Additional Protocol I of 1977”, International Review of the Red Cross, Vol. 88, No. 864, 2006 (ICRC New Weapons Guide), pp. 932–933.

106 Yves Sandoz, Christophe Swinarski and Bruno Zimmermann (eds), Commentary on the Additional Protocols, ICRC, Geneva, 1987 (ICRC Commentary on APs), para. 1402.

107 Ibid., para. 1957. See also Michael N. Schmitt, Tallinn Manual on the International Law Applicable to Cyber Warfare, Cambridge University Press, Cambridge, 2013, Rule 41(b).

108 ICRC Commentary on APs, above note 106, para. 1957.

109 ICRC New Weapons Guide, above note 105, pp. 937–938.

110 ICRC Commentary on APs, above note 106, para. 1476.

111 ICRC New Weapons Guide, above note 105, pp. 937–938.

112 Jean-Marie Henckaerts and Louise Doswald-Beck (eds), Customary International Humanitarian Law, Vol. 1: Rules, Cambridge University Press, Cambridge, 2005 (ICRC Customary Law Study), Rule 70, p. 237, available at: https://ihl-databases.icrc.org/customary-ihl/eng/docs/v1.

113 Ibid., Rule 71, p. 244; see also Rule 11, p. 37. And see International Court of Justice, Legality of the Threat or Use of Nuclear Weapons, Advisory Opinion, 8 July 1996, ICJ Reports 1996, paras 78, 95.

114 Bonnie Docherty, “Mind the Gap: The Lack of Accountability for Killer Robots”, Human Rights Watch, 9 April 2015, p. 7, available at: https://tinyurl.com/16fvbit4.

115 ICRC New Weapons Guide, above note 105, p. 943.

116 AP I, Art. 51(4)(b); ICTY, Prosecutor v. Dragomir Milošević, Case No. IT-98-29/1-A, Judgment (Appeals Chamber), 12 November 2009, para. 53; ICTY, Prosecutor v. Stanislav Galić, Case No. IT-98-29-A, Judgment (Appeals Chamber), 30 November 2006, para. 190; ICTY, Prosecutor v. Tihomir Blaškić, Case No. IT-95-14-A, Judgment (Appeals Chamber), 29 July 2004, para. 109.

117 AP I, Arts 51(5)(b), 57(2)(iii); ICTY, Galić, above note 116, para. 190.

118 AP I, Art. 57(4); see also Art. 57(2).

119 Jean-François Quéguiner, “Precaution Under the Law Governing the Conduct of Hostilities”, International Review of the Red Cross, Vol. 88, No. 864, 2006, pp. 793, 797–808.

120 Michael N. Schmitt, “Targeting and International Humanitarian Law in Afghanistan”, International Law Studies, Vol. 85, No. 1, 2009, p. 311. See also ICRC, Autonomous Weapon Systems: Technical, Military, Legal and Humanitarian Aspects, Geneva, March 2014, p. 83, available at: https://bit.ly/3c7h1F1; Maura Riley, “Killer Instinct: Lethal Autonomous Weapons in the Modern Battle Landscape”, Texas Law Review, Vol. 95, 2017, pp. 33–34, available at: https://bit.ly/3iFAsGp.

121 M. N. Schmitt, above note 120.

122 ICRC, Autonomy, Artificial Intelligence and Robotics: Technical Aspects of Human Control, Geneva, August 2019, p. 3, available at: https://bit.ly/3a8787w.

123 ICRC New Weapons Guide, above note 105, p. 949.

124 DoD, Autonomy in Weapon Systems: Directive 3000.09, 21 November 2012, available at: https://bit.ly/2XVRDtW; DoD, Law of War Manual, Washington, DC, December 2016 (DoD Law of War Manual), § 6.5.9.4, available at: https://bit.ly/3sFrrBJ.

125 ICRC New Weapons Guide, above note 105, pp. 931, 934; James D. Fry, “Contextualized Legal Reviews for the Methods and Means of Warfare: Cave Combat and International Humanitarian Law”, Columbia Journal of Transnational Law, Vol. 44, No. 2, 2006, pp. 453, 473–479.

126 ICRC Customary Law Study, above note 112, Rule 151.

127 ICRC, above note 5, p. 7. See also DoD Law of War Manual, above note 124, § 6.5.9.3

128 ICRC, above note 5. See also Eric Talbot Jensen, “The (Erroneous) Requirement for Human Judgment (and Error) in the Law of Armed Conflict”, International Law Studies, Vol. 96, No. 1, 2020, pp. 37–42 (summarizing the views of several States on why human control is necessary).

129 Report of the 2019 Session of the Group of Governmental Experts on Emerging Technologies in the Area of Lethal Autonomous Weapons Systems, UN Doc. CCW/GGE.1/2019/3, 25 September 2019, Annex IV, para. (b).

130 E. T. Jensen, above note 128, pp. 42–44.

131 ICRC Commentary on APs, above note 106, para. 1466 (emphasis added).

132 ICRC, above note 102, p. 13. See also S.-S. Hua, above note 98, pp. 128–129.

133 S.-S. Hua, above note 98, pp. 128–129.

134 Rome Statute, above note 19, Art. 30.

135 See, for example, ICTY, Prosecutor v. Milan Milutinović et al., Case No. IT-05-87-T, Judgment (Trial Chamber), 26 February 2009, para. 933.

136 ICC, Prosecutor v. Jean-Pierre Bemba Gombo, Case No. ICC-01/05-01/08, Judgment on the appeal of Mr Jean-Pierre Bemba Gombo against Trial Chamber III's “Judgment Pursuant to Article 74 of the Statute” (Appeals Chamber), 8 June 2018.

137 ICC, Prosecutor v. Jean-Pierre Bemba Gombo, Case No. ICC-01/05-01/08, Separate Opinion of Judge Christine Van den Wyngaert and Judge Howard Morrison (Appeals Chamber), 8 June 2018, para. 44.

138 Rome Statute, above note 19, Art. 54(1).

139 Ibid.

140 Claus Kress, “The Procedural Law of the International Criminal Court in Outline: Anatomy of a Unique Compromise”, Journal of International Criminal Justice, Vol. 1, No. 3, 2003, p. 608.

141 ICC, Situation in the Islamic Republic of Afghanistan, Case No. ICC-02/17, Judgment on the Appeal against the Decision on the Authorisation of an Investigation into the Situation in the Islamic Republic of Afghanistan (Appeals Chamber), 5 March 2020, para. 60. See also ICC, Prosecutor v. Thomas Lubanga Dyilo, Case No. ICC-01/04-01/06, Judgment on the Prosecutor's Appeal against the Decision of Pre-Trial Chamber I Entitled “Decision Establishing General Principles Governing Applications to Restrict Disclosure Pursuant to Rule 81(2) and (4) of the Rules of Procedure and Evidence” (Appeals Chamber), 12 October 2006, para. 52.

142 OTP Policy Paper on SGBC, above note 80, para. 48.

143 ICC, OTP, Code of Conduct for the Office of the Prosecutor, 5 September 2013, para. 49(b), available at: https://bit.ly/3itiSoU.

144 See, for example, ICTY, Prosecutor v. Radovan Karadžić, Case No. IT-95-5/18-T, Decision on Accused's Ninety-Fourth Disclosure Violation Motion (Trial Chamber), 13 October 2014; ICC, Prosecutor v. Alfred Yekatom and Patrice-Edouard Ngaïssona, Case No. ICC-01/14-01/18, Decision on the Yekatom Defence Request Concerning Disclosure Violation (Trial Chamber V), 18 January 2021.

145 Andrew G. Ferguson, “Big Data Prosecution and Brady”, UCLA Law Review, Vol. 67, No. 1, 2020, p. 184.

146 Richard A. Wilson and Matthew Gillett, The Hartford Guidelines on Speech Crimes in International Criminal Law, 2018, para. 265, available at: https://bit.ly/2M0To6e.

147 ICC, Proposed Programme Budget for 2018 of the International Criminal Court, ICC-ASP/16/10, 11 September 2017, para. 330, available at: https://bit.ly/3itlDXi.

148 Nema Milaninia, “Using Mobile Phone Data to Investigate Mass Atrocities and the Human Rights Considerations”, UCLA Journal of International Law and Foreign Affairs, Vol. 24, No. 2, 2020, pp. 283–286.

149 Elizabeth E. Joh, “The New Surveillance Discretion: Automated Suspicion, Big Data, and Policing”, Harvard Law and Policy Review, Vol. 10, No. 1, 2016, p. 25; Jennifer A. Johnson, John David Reitzel, Bryan F. Norwood, David M. McCoy, D. Brian Cummings and Renee R. Tate, “Social Network Analysis: A Systematic Approach for Investigating”, FBI Law Enforcement Bulletin, 5 March 2013, available at: https://bit.ly/35SFlH6.

150 IER Report, above note 92, para. 479.

151 ICC, OTP, Regulations of the Office of the Prosecutor, ICC-BD/05-01-09, 23 April 2009, Regulation 8; ICC, Proposed Programme Budget for 2020 of the International Criminal Court, ICC-ASP/18/10, 25 July 2019, para. 278, available at: https://bit.ly/39HayOs.

152 N. Milaninia, above note 148, p. 297.

153 Amy E. Street et al., “Developing a Risk Model to Target High-risk Preventive Interventions for Sexual Assault Victimization among Female U.S. Army Soldiers”, Clinical Psychological Science, Vol. 4, No. 6, 2016.

154 US Supreme Court, Herring v. United States, 555 U.S. 135, Justice Ginsburg Dissenting, 14 January 2009, p. 155.

155 ICC, Prosecutor v. Jean-Pierre Bemba Gombo et al., Case No. ICC-01/05-01/13, Judgment on the Appeals of Mr Jean-Pierre Bemba Gombo, Mr Aimé Kilolo Musamba, Mr Jean-Jacques Mangenda Kabongo, Mr Fidèle Babala Wandu and Mr Narcisse Arido against the Decision of Trial Chamber VII Entitled “Judgment Pursuant to Article 74 of the Statute” (Appeals Chamber), 8 March 2018, paras 576–601; ICC, Prosecutor v. Jean-Pierre Bemba Gombo, Case No. ICC-01/05-01/08, Judgment on the Appeals of Mr Jean-Pierre Bemba Gombo and the Prosecutor against the Decision of Trial Chamber III Entitled “Decision on the Admission into Evidence of Materials Contained in the Prosecution's List of Evidence” (Appeals Chamber), 3 May 2011, para. 37.

156 ICC, Prosecutor v. Alfred Yekatom and Patrice-Edouard Ngaïssona, Case No. ICC-01/14-01/18, Initial Directions on the Conduct of the Proceedings (Trial Chamber V), 26 August 2020, paras 52–53; ICC, Prosecutor v. Al Hassan Ag Abdoul Aziz Ag Mohamed Ag Mahmoud, Case No. ICC-01/12-01/18, Annex A to the Decision on the Conduct of Proceedings (Trial Chamber X), 6 May 2020, paras 30–31; ICC, Prosecutor v. Dominic Ongwen, Case No. ICC-02/04-01/15, Initial Directions on the Conduct of the Proceedings (Trial Chamber IX), 13 July 2016, paras 24–25; ICC, Prosecutor v. Laurent Gbagbo and Charles Blé Goudé, Case No. ICC-02/11-01/15, Decision on the Submission and Admission of Evidence (Trial Chamber I), 29 January 2016; ICC, Prosecutor v. Jean-Pierre Bemba Gombo et al., Case No. ICC-01/05-01/13, Decision on Prosecution Requests for Admission of Documentary Evidence (ICC-01/05-01/13-1013-Red, ICC-01/05-01/13-1113-Red, ICC-01/05-01/13-1170-Conf) (Trial Chamber VII), 24 September 2015, paras 10–13.

157 ICC, Prosecutor v. Thomas Lubanga Dyilo, Case No. ICC-01/04-01/06, Decision on the Defence Request for Unrestricted Access to the Entire File of the Situation in the Democratic Republic of the Congo (Pre-Trial Chamber I), 17 May 2006, pp. 2–3 (rejecting the Defence's request for access to the entire file of the DRC situation, noting the Prosecution's submission that the request constituted a “fishing expedition” and did not identify the legitimate forensic purpose for the request). See also ICTY, Prosecutor v. Dragomir Milošević, Case No. IT-98-29/l-A, Decision on Motion Seeking Disclosure of Rule 68 Material (Trial Chamber I), 7 September 2012, para. 5.

158 ICC, Prosecutor v. Jean-Pierre Bemba Gombo, Case No. ICC-01/05-01/08, Decision on Defence Requests for Disclosure (Trial Chamber III), 2 July 2014, para. 29.

159 ICC, Prosecutor v. Jean-Pierre Bemba Gombo et al., Case No. ICC-01/05-01/13, Decision on Mangenda Defence Request for Cooperation (Trial Chamber VII), 14 August 2015, para. 11; ICC, Prosecutor v. Saif Al-Islam Gaddafi and Abdullah Al-Senussi, Case No. ICC-01/11-01/1, Corrigendum to Decision on the “Defence Request for an Order of Disclosure”, (Pre-Trial Chamber I), 1 August 2013, para. 40.

160 ICC, Prosecutor v. Jean-Pierre Bemba Gombo et al., Case No. ICC-01/05-01/13, Decision on Defence Requests for Prosecution Requests for Assistance, Domestic Records and Audio Recordings of Interviews (Trial Chamber VII), 10 September 2015, para. 13.

161 Ibid.

162 ICC, Bemba Gombo et al., above note 159, para. 10.

163 ICC, Prosecutor v. William Samoei Ruto, Henry Kiprono Kosgey and Joshua Arap Sang, Case No. ICC-01/09-01/11, Decision on the Defence Requests in Relation to the Victims’ Applications for Participation in the Present Case (Pre-Trial Chamber II), 8 July 2011, para. 9; ICC, Prosecutor v. Callixte Mbarushimana, Case No. ICC-01/04-01/10, Decision on Issues relating to Disclosure (Pre-Trial Chamber I), 30 March 2011, para. 15.

164 ICC, Prosecutor v. Thomas Lubanga Dyilo, Case No. ICC-01/04-01/06, Redacted Decision on the Prosecution's Disclosure Obligations Arising Out of an Issue Concerning Witness DRC-OTP-WWWW-0031 (Trial Chamber I), 20 January 2011, para. 16.

165 IER Report, above note 92, para. 481.

166 See, for example, ICC, Prosecutor v. Alfred Yekatom and Patrice-Edouard Ngaïssona, Case No. ICC-01/14-01/18, Prosecution's Request to Vary the Decision on Disclosure and Related Matters (ICC-01/14-01/18-64-Red) (Pre-Trial Chamber II), 20 March 2019, para. 7.

167 ICC, Prosecutor v. Abdallah Banda Abakaer Nourain and Saleh Mohammed Jerbo Jamus, Case No. ICC-02/05-03/09 OA 4, Judgment on the Appeal of Mr Abdallah Banda Abakaer Nourain and Mr Saleh Mohammed Jerbo Jamus against the Decision of Trial Chamber IV of 23 January 2013 Entitled “Decision on the Defence's Request for Disclosure of Documents in the Possession of the Office of the Prosecutor” (Appeals Chamber), 28 August 2013, para. 42.

168 ICC, Prosecutor v. Jean-Pierre Bemba Gombo et al., Case No. ICC-01/05-01/13, Defence Request for Leave to Reply to the Prosecution's Response to Bemba's “Consolidated Request for Disclosure and Judicial Assistance”, ICC-01/05-01/13-2236-Conf, 6 October 2017, ICC-01/05-01/13-2236-Conf-Corr, 10 October 2017 (Appeals Chamber), 12 October 2017, para. 10.

169 IER Report, above note 92, para. 480 (“It was submitted that during the confirmation stage the Prosecutor does not commence redaction and disclosure until the Chamber first adopts a redaction protocol”).

170 ICC, Prosecutor v. Alfred Yekatom and Patrice-Edouard Ngaïssona, Case No. ICC-01/14-01/18, Public Redacted Version of “Decision on Disclosure and Related Matters” (Pre-Trial Chamber II), 23 January 2019. See also ICC, Prosecutor v. Alfred Yekatom and Patrice-Edouard Ngaïssona, Case No. ICC-01/14-01/18, Prosecution's Communication of the Disclosure of Evidence (Pre-Trial Chamber II), 31 July 2019.

171 ICC, Prosecutor v. Ali Muhammad Ali Abd-Al-Rahman (“Ali Kushayb”), Case No. ICC-02/05-01/20, Second Order on Disclosure and Related Matters (Pre-Trial Chamber II), 2 October 2020, para. 24; ICC, Prosecutor v. Ali Muhammad Ali Abd-Al-Rahman (“Ali Kushayb”), Case No. ICC-02/05-01/20, Prosecution's Third Progress Report on the Evidence Review, Translation and Disclosure Process (Pre-Trial Chamber II), 9 October 2020, para. 25 (noting “that this order will substantially increase the time required for the primary and secondary review of items for disclosure, especially in relation to lengthy documents, such as interview transcripts”).

172 IER Report, above note 92, paras 577–584.

173 Ibid., para. 479.

174 US Supreme Court, Williams v. Pennsylvania, 136 S. Ct. 1899, 9 June 2016, p. 1905.