Requirement 9: Tools must support data retention and reproducibility to enable meaningful contestation and challenges Requirement 9: Tools must support data retention and reproducibility to enable meaningful contestation and challenges
In order for defendants to contest decisions made by risk assessment tools, they must have access to information about how the tools’ predictions are made. For a discussion of the due process concerns that arise when information is withheld in the context of automated decision-making, see Danielle Keats Citron, Technological Due Process, 85 Wash. U. L. Rev. 1249 (2007), https://ssrn.com/abstract=1012360. See also, Paul Schwartz, Data Processing and Government Administration: The Failure of the American Legal Response to the Computer, 43 Hastings L. J. 1321 (1992). As discussed above, there are many potential technical concerns related to the use of these tools, in particular with regard to bias. Given the adversarial nature of the U.S. criminal justice system, which depends on defendants and their attorneys to advance any arguments in their favor, denying defendants the ability to access information about how these decisions are made hampers their ability to contest these decisions.
Individual-level information used in the assessments should be recorded in an audit trail that is made available to defendants, counsel, and judges. Such audit trails must be maintained in an immutable form for future review, so auditors can achieve a reproducible calculation for that individual’s level of risk.Additionally, the ability to reconstitute decisions evidences procedural regularity in critical decision processes and allows individuals to trust the integrity of automated systems even when they remain partially non-disclosed. See Joshua A. Kroll et al., Accountable algorithms, 165 U. Pa. L. Rev. 633 (2016). Defendants should also have an opportunity to contest any inaccuracies in the input information or inferences in the resulting risk classification and to present additional mitigating information.The ability to contest scores is not only important for defendant’s rights to adversarially challenge adverse information, but also for the ability of judges and other professionals to engage with the validity of the risk assessment outputs and develop trust in the technology. See Daniel Kluttz et al., Contestability and Professionals: From Explanations to Engagement with Algorithmic Systems (January 2019), https://dx.doi.org/10.2139/ssrn.3311894 This is especially important given the potential for risk assessment tools to be manipulated. For example, risk assessments often rely on questionnaires administered to arrestees, which presents the opportunity for abuse by administrators, as illustrated by instances of “criteria tinkering.” “Criteria tinkering” occurs when court clerks manipulate input values to obtain the score they think is correct for a particular defendant. See Hannah-Moffat, Kelly, Paula Maurutto, and Sarah Turnbull, Negotiated risk: Actuarial illusions and discretion in probation, 24.3 Canada J. of L. & Society/La Revue Canadienne Droit et Société 391 (2009). See also Angele Christin, Comparing Web Journalism and Criminal Justice, 4.2 Big Data & Society 1. Adversarial analysis is likely the best way to protect against such manipulations. Through these processes, defendants can demonstrate how applicable and robust risk assessments are or are not with respect to their particular circumstances.