Hostname: page-component-586b7cd67f-gb8f7 Total loading time: 0 Render date: 2024-11-28T03:21:15.521Z Has data issue: false hasContentIssue false
  • English
  • Français

Ethical and Legal Implications of the Methodological Crisis in Neuroimaging

Published online by Cambridge University Press:  22 September 2017

PHILIPP KELLMEYER
Get access Rights & Permissions [Opens in a new window]

Abstract:

Currently, many scientific fields such as psychology or biomedicine face a methodological crisis concerning the reproducibility, replicability, and validity of their research. In neuroimaging, similar methodological concerns have taken hold of the field, and researchers are working frantically toward finding solutions for the methodological problems specific to neuroimaging. This article examines some ethical and legal implications of this methodological crisis in neuroimaging. With respect to ethical challenges, the article discusses the impact of flawed methods in neuroimaging research in cognitive and clinical neuroscience, particularly with respect to faulty brain-based models of human cognition, behavior, and personality. Specifically examined is whether such faulty models, when they are applied to neurological or psychiatric diseases, could put patients at risk, and whether this places special obligations on researchers using neuroimaging. In the legal domain, the actual use of neuroimaging as evidence in United States courtrooms is surveyed, followed by an examination of ways that the methodological problems may create challenges for the criminal justice system. Finally, the article reviews and promotes some promising ideas and initiatives from within the neuroimaging community for addressing the methodological problems.

Keywords

neuroimagingreproducibilityreplicationneuroethicsneurolaw
Type
Articles
Information
Cambridge Quarterly of Healthcare Ethics , Volume 26 , Special Issue 4: Clinical Neuroethics , October 2017 , pp. 530 - 554
Copyright
Copyright © Cambridge University Press 2017 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Notes

1. Eklund A, Nichols TE, Knutsson H. Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates. Proceedings of the National Academy of Sciences 2016;113:7900–905.

2. Wager TD, Lindquist MA, Nichols TE, Kober H, Van Snellenberg JX. Evaluating the consistency and specificity of neuroimaging data using meta-analysis. NeuroImage 2009;45:S210–21.

3. Pauli R, Bowring A, Reynolds R, Chen G, Nichols TE, Maumet C. Exploring fMRI Results Space: 31 Variants of an fMRI Analysis in AFNI, FSL, and SPM. Frontiers in Neuroinformatics 2016;10:24.

4. Kazemi, K, Noorizadeh, N. Quantitative comparison of SPM, FSL, and Brainsuite for brain MR image segmentation. Journal of Biomedical Physics and Engineering 2014;4(1):13–26.Google Scholar

5. Lieberman, MD, Cunningham, WA. Type I and Type II error concerns in fMRI research: Re-balancing the scale. Social Cognitive and Affective Neuroscience 2009;4(4):324–8.Google Scholar

6. Ball, T, Breckel, TPK, Mutschler, I, Aertsen, A, Schulze-Bonhage, A, Hennig, J, et al. Variability of fMRI-response patterns at different spatial observation scales. Human Brain Mapping 2012;33:1155–1171.Google Scholar

7. Munafò MR, Nosek BA, Bishop DVM, Button KS, Chambers CD, du Sert NP, et al. A manifesto for reproducible science. Nature Human Behaviour 2017;1:21.

8. Forstmeier W, Wagenmakers E-J, Parker TH. Detecting and avoiding likely false-positive findings – a practical guide. Biological Reviews 2016 [Epub ahead of print].

9. Kerr, NL. HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review 1998;2:196–217.Google Scholar

10. Gershman, SJ, Daw, ND. reinforcement learning and episodic memory in humans and animals: An integrative framework. Annual Review of Psychology 2017;68:101–28.CrossRefGoogle ScholarPubMed

11. Phelps, EA, LeDoux, JE. Contributions of the amygdala to emotion processing: From animal models to human behavior. Neuron 2005;48:175–87.Google Scholar

12. EEG and FMRI Papers by the Numbers| Neuroscience | Human Brain Diversity Project. Sapien Labs | Neuroscience | Human Brain Diversity Project 2016; available at http://sapienlabs.co/500000-human-neuroscience-papers/ (last accessed 12 Mar 2017).

13. Morse S. Brain Overclaim Syndrome and Criminal Responsibility: A Diagnostic Note. Rochester, NY: Social Science Research Network; 2006.

14. Collaboration OS. Estimating the reproducibility of psychological science. Science 2015;349:aac4716.

15. Szucs, D, Ioannidis, JPA. Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature. PLOS Biology 2017;15:e2000797.Google Scholar

16. Begley, CG, Ioannidis, JPA. Reproducibility in science. Circulation Research 2015;116:116–26.Google Scholar

17. Stroebe, W, Strack, F. The alleged crisis and the illusion of exact replication. Perspectives on Psychological Science 2014;9:59–71.Google Scholar

18. Peng, R. The reproducibility crisis in science: A statistical counterattack. Significance 2015;12:30–2.Google Scholar

19. Ashburner, J, Friston, KJ. Voxel-based morphometry—The methods. NeuroImage 2000;11:805–21.CrossRefGoogle ScholarPubMed

20. Carp, J. The secret lives of experiments: methods reporting in the fMRI literature. NeuroImage 2012;63:289–300.Google Scholar

21. Raichle, ME. The brain’s default mode network. Annual Review of Neuroscience 2015;38:433–47.Google Scholar

22. Chalmers, DJ. Facing up to the problem of consciousness. Journal of Consciousness Studies 1995;2:200–19.Google Scholar

23. Button KS, Ioannidis JPA, Mokrysz C, Nosek BA, Flint J, Robinson ESJ, et al. Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience 2013;14:365–76.

24. Ioannidis JPA. Excess significance bias in the literature on brain volume abnormalities. Archives of General Psychiatry 2011;68:773–80.

25. Shuter, B, Yeh, IB, Graham, S, Au, C, Wang, S-C. Reproducibility of brain tissue volumes in longitudinal studies: Effects of changes in signal-to-noise ratio and scanner software. NeuroImage 2008;41:371–9.Google Scholar

26. Rajagopalan V, Pioro EP. Disparate voxel based morphometry (VBM) results between SPM and FSL softwares in ALS patients with frontotemporal dementia: Which VBM results to consider? BMC Neurology 2015;15:32.

27. Rajagopalan V, Yue GH, Pioro EP. Do preprocessing algorithms and statistical models influence voxel-based morphometry (VBM) results in amyotrophic lateral sclerosis patients? A systematic comparison of popular VBM analytical methods. Journal of Magnetic Resonance Imaging 2014;40:662–7.

28. Kümmerer, D, Hartwigsen, G, Kellmeyer, P, Glauche, V, Mader, I, Klöppel, S, et al. Damage to ventral and dorsal language pathways in acute aphasia. Brain 2013;136:619–29.CrossRefGoogle ScholarPubMed

29. Mah, Y-H, Husain, M, Rees, G, Nachev, P. Human brain lesion-deficit inference remapped. Brain 2014;137:2522–31.CrossRefGoogle ScholarPubMed

30. Worsley KJ, Marrett S, Neelin P, Evans AC. Searching scale space for activation in PET images. Human Brain Mapping 1996;4:74–90.

31. David SP, Ware JJ, Chu IM, Loftus PD, Fusar-Poli P, Radua J, et al. Potential Reporting Bias in fMRI Studies of the Brain. Plos One 2013;8:e70104.

32. Lindquist MA, Meng Loh J, Atlas LY, Wager TD. Modeling the hemodynamic response function in fMRI: Efficiency, bias and mis-modeling. NeuroImage 2009;45:S187–98.

33. See note 4, Kazemi, Noorizadeh 2014.

34. See note 6, Ball et al. 2012.

35. See note 5, Lieberman, Cunningham 2014

36. Kriegeskorte, N, Simmons, WK, Bellgowan, PSF, Baker, CI. Circular analysis in systems neuroscience: the dangers of double dipping. Nature Neuroscience 2009;12:535–40.Google Scholar

37. Reddan MC, Lindquist MA, Wager TD. Effect size estimation in neuroimaging. JAMA Psychiatry 2017;74:207–8.

38. See note 37, Reddan et al. 2017; Vul E, Harris C, Winkielman P, Pashler H. Puzzlingly high correlations in fMRI studies of emotion, personality, and social cognition. Perspectives on Psychological Science 2009;4:274–90.

39. See note 1, Eklund et al. 2016.

40. Reynolds E. Bug in fMRI software calls 15 years of research into question. WIRED UK, July 6, 2016; available at http://www.wired.co.uk/article/fmri-bug-brain-scans-results (last accessed 10 Mar 2017).

41. Computer says: oops. The Economist, July 16, 2016; available at http://www.economist.com/news/science-and-technology/21702166-two-studies-one-neuroscience-and-one-palaeoclimatology-cast-doubt (last accessed 12 March 2017).

42. See note 1, Eklund et al. 2016, at 7900.

43. Pernet C, Nichols T. Has a software bug really called decades of brain imaging research into question? The Guardian, 2016; available at https://www.theguardian.com/science/head-quarters/2016/sep/30/has-a-software-bug-really-called-decades-of-brain-imaging-research-into-question (last accessed 1 Mar 2017).

44. Mumford JA, Pernet C, Yeo BTT, Nickerson D, Muhlert N, Stikov N, et al. Keep calm and scan on. Organization for Human Brain Mapping (OHBM), July 21, 2016; available at http://www.ohbmbrainmappingblog.com/blog/keep-calm-and-scan-on (last accessed 13 Mar 2017).

45. Zuo, X-N, He, Y, Betzel, RF, Colcombe, S, Sporns, O, Milham, MP. Human connectomics across the life span. Trends in Cognitive Sciences 2017;21:32–45.Google Scholar

46. Glasser, MF, Smith, SM, Marcus, DS, Andersson, JLR, Auerbach, EJ, Behrens, TEJ, et al. The human connectome project’s neuroimaging approach. Nature Neuroscience 2016;19:1175–87.Google Scholar

47. Grayson, DS, Bliss-Moreau, E, Machado, CJ, Bennett, J, Shen, K, Grant, KA, et al. The rhesus monkey connectome predicts disrupted functional networks resulting from pharmacogenetic inactivation of the amygdala. Neuron 2016;91:453–66.CrossRefGoogle ScholarPubMed

48. Guevara, M, Román, C, Houenou, J, Duclap, D, Poupon, C, Mangin, JF, et al. Reproducibility of superficial white matter tracts using diffusion-weighted imaging tractography. NeuroImage 2017;147:703–25.Google Scholar

49. O’Donnell LJ, Pasternak O. Does diffusion MRI tell us anything about the white matter? An overview of methods and pitfalls. Schizophrenia Research 2015;161:133–41.

50. Jones DK, Knösche TR, Turner R. White matter integrity, fiber count, and other fallacies: The do’s and don’ts of diffusion MRI. NeuroImage 2013;73:239–54.

51. Jones DK, Symms MR, Cercignani M, Howard RJ. The effect of filter size on VBM analyses of DT-MRI data. NeuroImage 2005;26:546–54.

52. See note 15, Szucs, Ioannidis 2017.

53. Johnson, J. A dark history: Memories of lobotomy in the new era of psychosurgery. Medicine Studies 2009;1:367–78.CrossRefGoogle Scholar

54. Long T. Nov. 12, 1935: You Should (Not) Have a Lobotomy. WIRED, December 11, 2010; available at https://www.wired.com/2010/11/1112first-lobotomy/ (last accessed 14 Mar 2017).

55. Gross D, Schäfer G. Egas Moniz (1874–1955) and the “invention” of modern psychosurgery: A historical and ethical reanalysis under special consideration of Portuguese original sources. Neurosurgical Focus 2011;30:E8.

56. Kucharski, A. History of frontal lobotomy in the United States, 1935-1955. Neurosurgery 1984;14:765–72.CrossRefGoogle ScholarPubMed

57. Silberman S. NeuroTribes: The Legacy of Autism and the Future of Neurodiversity. New York: Penguin Publishing Group; 2015.

58. Harris, JC. The origin and natural history of autism spectrum disorders. Nature Neuroscience 2016;19:1390–1.Google Scholar

59. Marblestone AH, Wayne G, Kording KP. Toward an integration of deep learning and neuroscience. Frontiers in Computational Neuroscience 2016;94.

60. Greenspan, H, van Ginneken, B, Summers, RM. Guest editorial deep learning in medical imaging: overview and future promise of an exciting new technique. IEEE Transactions on Medical Imaging 2016;35:1153–9.Google Scholar

61. Smyser CD, Dosenbach NUF, Smyser TA, Snyder AZ, Rogers CE, Inder TE, et al. Prediction of brain maturity in infants using machine-learning algorithms. NeuroImage 2016;136:1–9.

62. Moradi, E, Pepe, A, Gaser, C, Huttunen, H, Tohka, J. Machine learning framework for early MRI-based Alzheimer’s conversion prediction in MCI subjects. NeuroImage 2015;104:398–412.Google Scholar

63. Salvatore C, Cerasa A, Castiglioni I, Gallivanone F, Augimeri A, Lopez M, et al. Machine learning on brain MRI data for differential diagnosis of Parkinson’s disease and progressive supranuclear palsy. Journal of Neuroscience Methods 2014;222:230–7.

64. Greene J, Cohen J. For the law, neuroscience changes nothing and everything. Philosophical Transactions of the Royal Society B: Biological Sciences 2004;359:1775–85.

65. Morse SJ. Inevitable mens rea. Harvard Journal of Law & Public Policy 2003;27:51.

66. Denno DW. The myth of the double-edged sword: An empirical study of neuroscience evidence in criminal cases. Boston College Law Review 2015;56:493.

67. Farahany, NA. Neuroscience and behavioral genetics in US criminal law: An empirical analysis. Journal of Law and the Biosciences 2016;2:485–509.Google Scholar

68. Gaudet LM, Marchant GE. Under the radar: Neuroimaging evidence in the criminal courtroom. Drake Law Review 2016;64:577.

69. Weisberg DS, Keil FC, Goodstein J, Rawson E, Gray JR. The seductive allure of neuroscience explanations. Journal of Cognitive Neuroscience 2007;20:470–7.

70. McCabe, DP, Castel, AD. Seeing is believing: The effect of brain images on judgments of scientific reasoning. Cognition 2008;107:343–52.CrossRefGoogle ScholarPubMed

71. Gurley JR, Marcus DK. The effects of neuroimaging and brain injury on insanity defenses. Behavioral Sciences & the Law 2008;26:85–97.

72. See note 69, Weisberg et al. 2007.

73. Schweitzer NJ, Saks MJ, Murphy ER, Roskies AL, Sinnott-Armstrong W, Gaudet LM. Neuroimages as evidence in a mens rea defense: No impact. Psychology, Public Policy, and Law 2011;17:357–93.

74. Gaudet LM, Kerkmans JP, Anderson NE, Kiehl KA. Can neuroscience help predict future antisocial behavior. Fordham Law Review 2016;85:503.

75. Greene JD, Paxton JM. Patterns of neural activity associated with honest and dishonest moral decisions. Proceedings of the National Academy of Sciences of the United States of America 2009;106:12506–11.

76. Sip KE, Roepstorff A, McGregor W, Frith CD. Detecting deception: the scope and limits. Trends in Cognitive Sciences 2008;12:48–53.

77. Lee TMC, Liu H-L, Tan L-H, Chan CCH, Mahankali S, Feng C-M, et al. Lie detection by functional magnetic resonance imaging. Human Brain Mapping 2002;15:157–64.

78. Spence SA. Playing Devil’s advocate†: The case against fMRI lie detection. Legal and Criminological Psychology 2008;13:11–25.

79. Ganis G, Rosenfeld JP, Meixner J, Kievit RA, Schendan HE. Lying in the scanner: Covert countermeasures disrupt deception detection by functional magnetic resonance imaging. NeuroImage 2011;55:312–319.

80. Ryan MM. Daubert Standard. LII / Legal Information Institute 2009. Ithaca, NY: Cornell Law School, https://www.law.cornell.edu/wex/daubert_standard (last accessed March 10th 2017).

81. See note 13, Morse 2006; note 65, Morse 2003.

82. See note 64, Greene, Cohen 2004; Shariff, AF, Greene, JD, Karremans, JC, Luguri, JB, Clark, CJ, Schooler, JW, et al. Free will and punishment: A mechanistic view of human nature reduces retribution. Psychological Science 2014;25:1563–70.CrossRefGoogle ScholarPubMed

83. Singer T, Seymour B, O’Doherty JP, Stephan KE, Dolan RJ, Frith CD. Empathic neural responses are modulated by the perceived fairness of others. Nature 2006;439:466–9.

84. Fehr E, Singer T. The Neuroeconomics of Mind Reading and Empathy. Rochester, NY: Social Science Research Network; 2005.

85. See note 44, Mumford et al. 2016; Poldrack RA, Baker CI, Durnez J, Gorgolewski KJ, Matthews PM, Munafò MR, et al. Scanning the horizon: towards transparent and reproducible neuroimaging research. Nature Reviews Neuroscience 2017;18:115–26; Nichols TE, Das S, Eickhoff SB, Evans AC, Glatard T, Hanke M, et al. Best practices in data analysis and sharing in neuroimaging using MRI. Nature Neuroscience 2017;20:299–303.

86. Liem F, Mérillat S, Bezzola L, Hirsiger S, Philipp M, Madhyastha T, et al. Reliability and statistical power analysis of cortical and subcortical FreeSurfer metrics in a large sample of healthy elderly. NeuroImage 2015;108:95–109.

87. See note 20, Carp 2012.

88. Poldrack RA, Gorgolewski KJ. OpenfMRI: Open sharing of task fMRI data. NeuroImage 2017;144, Part B:259–61.

89. Al-Azizy, D, Millard, D, Symeonidis, I, O’Hara, K, Shadbolt, N. A literature survey and classifications on data deanonymisation. In Lambrinoudakis, C, Gabillon, A, eds. Risks and Security of Internet and Systems. Cham, Switzerland: Springer International Publishing; 2015:36–51.Google Scholar

90. Woo, C-W, Chang, LJ, Lindquist, MA, Wager, TD. Building better biomarkers: Brain models in translational neuroimaging. Nature Neuroscience 2017;20:365–77.Google Scholar