Quality Review of Mass Adjudication: A Randomized Natural Experiment at the Board of Veterans Appeals, 2003-16

Details

Author(s):
Publish Date:
July, 2019
Publication Title:
Journal of Law, Economics, & Organization
Format:
Journal Article Volume 35 Issue 2 Page(s) 239-288
Citation(s):
  • Daniel E. Ho et al., Quality Review of Mass Adjudication: A Randomized Natural Experiment at the Board of Veterans Appeals, 2003-16, 35 Journal of Law, Economics, & Organization 239 (2019).

Abstract

We study a unique natural experiment, during which 5–10% of draft opinions by judges of the Board of Veterans Appeals (BVA) were randomly selected for “quality review (QR)” by a team of full-time staff attorneys. The express goals of this performance program were to measure accuracy and reduce remands on appeal. In cases of legal error, the QR team wrote memoranda to judges for correction of draft opinions. We use rich internal administrative data on nearly 600,000 cases from 2003 to 2016 to conduct the first rigorous evaluation of this program. With precise estimates, we show that QR had no appreciable effects on appeals or remands. Based on internal records, we demonstrate that this inefficacy is likely by design, as meeting the performance measure of “accuracy” conflicted with error correction. These findings inform longstanding questions of law, organization, and bureaucracy, including performance management, standards of review, and institutional design of adjudication. (JEL K23, K40, H11)