SoundThinking’s Black-Box Gunshot Detection Method: Untested and Unvetted Tech Flourishes in the Criminal Justice System

Abstract

SoundThinking has successfully marketed their ShotSpotter forensic gunshot detection method to police departments and prosecutors as a reliable method for detecting and locating gunfire incidents in urban environments and generating admissible evidence for use in criminal prosecutions. The ShotSpotter method involves networks of microphones deployed in urban settings, which are tasked with detecting the impulsive sounds of gunfire and transmitting the sound recordings to SoundThinking’s black-box algorithm and human examiners for forensic analysis. If the impulsive noises are suspected to have originated from gunfire, SoundThinking alerts local police departments so that officers can quickly respond and investigate. SoundThinking promotes ShotSpotter as a high-tech improvement on traditional 911 calls. The reliability of ShotSpotter is hindered by the technically-challenging environments where ShotSpotter systems are deployed (neighborhoods with dense buildings and other infrastructure) and the routine occurrence of impulsive noises (from vehicle traffic, construction equipment, and many other sources) that are known to trigger ShotSpotter false alerts. To assess the ability of methods like SoundThinking’s gunshot detection method to reliably complete their forensic tasks and to quantify important rates of error, method developers like SoundThinking are supposed to engage in a multi-step development process involving validation testing, algorithm verification, and error rate analysis. Yet SoundThinking has largely ignored this development process, instead promoting accuracy and performance claims that have no legitimate scientific bases. And neither the scientific community nor the judicial system have engaged in the forms of oversight that should preclude the use of such untested forensic evidence in the criminal justice system. The result has been a proliferation of ShotSpotter systems in 150 U.S. cities and the use of ShotSpotter evidence in over 200 criminal trials without proof that the ShotSpotter method works and in the face of growing evidence that the method is plagued by a high incidence of false alerts. In light of the facts that ShotSpotter systems are deployed primarily in communities of color and the harms associated with ShotSpotter false alerts are borne primarily by people of color, both the scientific and legal communities need to engage in rigorous oversight of SoundThinking’s forensic method. Until such oversight establishes the reliability of ShotSpotter evidence and its true rates of error, ShotSpotter evidence should play no role in the criminal justice system.

Details

Publisher:
Stanford University Stanford, California
Citation(s):
  • Brendan Max, SoundThinking’s Black-Box Gunshot Detection Method: Untested and Unvetted Tech Flourishes in the Criminal Justice System, 26 Stan. Tech. L. Rev. 193 (2023).
Related Organization(s):