Calibration of Forensic Risk Assessment Instruments

Abstract

Risk assessment instruments have been the subject of validation studies which have primarily investigated the properties known from psychometrics (objectivity, reliability and validity). Validation of forensic risk assessment instruments is confronted with a whole row of methodical challenges, which differ considerably from the topics addressed in psychological test development. Risk assessments instruments state the probability of a particular offense to occur. To disregard the probabilistic nature of risk calculations leads to methodically faulty assumptions on the predictive validity of an instrument. Furthermore, test developers will use questionable statistical methods to assess the validity of the instruments such as ROC analyses. This analysis strategy is considered to be state of the art even though this method does not take into account the probabilistic nature of prognoses. Its results can thus be interpreted only to a limited degree. ROC analyses for example disregard certain aspects of an instrument’s calibration, which might lead in an instrument’s validation to high ROC values while demonstrating only low validity.