Kleros—a decentralized blockchain-based arbitration solution that relies on smart contracts and crowdsourced jurors—was conceived to bridge the trust gap that separates e-commerce participants, as in the paradigmatic example of “Alice and Bob.” Previous studies have mainly focused on describing blockchain-based arbitration solutions such as Kleros, and at least one has warned about the potential risks to essential procedural safeguards presented by anonymous and economically-incentivized jurors. Using Kleros as a socio-legal case study of a token-based business providing cutting-edge decentralized blockchain arbitration services, we paint the “big picture” of how Kleros works, and analyze its efficacy to improve on traditional online dispute resolution. We also report the results of our exploratory, observational user survey and a series of interface trials. We make a number of contributions to the literature. First, our case study introduces the concept of the “decentralized sheriff,” which we use to refer to the crowd-based compliance mechanism implemented by Kleros to certify crypto tokens. We argue that Kleros’ “decentralized sheriff” contributes to the public good by filling a regulatory hole with respect to the crypto market. Second, building on the work of scholars who studied Kleros, we analyze the major obstacles to widespread adoption of Kleros. Third, we explore whether social interactions of trust and claiming present unique characteristics under Kleros compared to models described in Law and Society literature. Throughout our analysis, we formulate questions for future research and investigation.