While the refugee protection system is one of international law’s most recognizable features, it routinely places massive numbers of refugees in camps in the developing world, where they face chronic threats to their physical security from crime and disorder, physical coercion, and military attacks. Yet key actors responsible for refugee protection, including host states, advanced industrialized countries, and the United Nations High Commissioner for Refugees (UNHCR), have failed to prioritize refugee security. This article asks: (1) Why? (2) What have been the consequences? (3) And what do these answers reveal about how organizations carry out legal mandates in complicated political environments? Conventional wisdom holds that security only recently became a major problem in the refugee protection system, that UNHCR’s role in enhancing refugees’ physical security is limited by the agency’s legal mandate and practical constraints, and that problems of violence and physical security are largely episodic concerns affecting small numbers in discrete refugee populations. Drawing on historical documents, interviews, data on budgets and performance measures, and legal doctrine, I show this conventional wisdom to be wrong. Only some of the problems associated with the current system can be explained by international geopolitics or by legal compromises reflected in refugee law. Instead, that system’s brutal realities also reflect the intersecting effects of bureaucratic dynamics, political pressures, and legal interpretations shaping the discretionary choices of UNHCR and its nongovernmental organization partners. I develop the argument by tracing the remarkable history of UNHCR as it transformed itself from a refugee advocacy organization with a limited mandate into a modern relief agency. This evolution helps explain the persistence of security problems, and sheds light on the challenges of implementing ambitious legal mandates under uncertainty, particularly when the organizations doing so operate in complex political environments.