Williams v. Pennsylvania, the judicial recusal case the Supreme Court decided last Thursday, was as much about roles as it was about rules. Yes, the Court announced a rule: judges have to recuse themselves from any criminal cases in which they acted as prosecutors. But the case serves as a reminder that in our system of criminal justice, the people who serve as judges, prosecutors, and defense attorneys—their skills, their characters, and their professional self-conceptions—can matter as much as or more than the rules under which they operate. It reminds us, too, how much can ride on how we define these roles: how impartial we say a judge needs to be, what kind of officials we ask prosecutors to be, and what obligations and aspirations we think defense attorneys should assume.

In 1986 the District Attorney of Philadelphia, Ronald Castille, authorized his office to seek the death penalty for Terrance Williams, who was then on trial for murder. The jury convicted Williams and sentenced him to die. Years later, shortly before Williams was scheduled to be executed, a Pennsylvania trial judge threw out the sentence, concluding that Castille’s office had violated its obligations under Brady v. Maryland to disclose evidence favorable to the defense. The prosecutors appealed that ruling to the Pennsylvania Supreme Court, which was by this time presided over by Chief Justice Ronald Castille. Castille declined to recuse himself, and when the Pennsylvania Supreme Court reinstated the death sentence he wrote a separate concurring opinion lambasting the lower court’s decision. He has since retired.

Last Thursday the United States Supreme Court ruled 5-3 that Castille’s failure to recuse himself violated the Due Process Clause of the Fourteenth Amendment, because his “significant, personal involvement in a critical decision in Williams’s case gave rise to an unacceptable risk of actual bias.” The Justices ordered the Pennsylvania Supreme Court to reconsider the case without Castille’s participation.

The unfairness is pretty obvious when someone serves as both prosecutor and judge in the same case. In fact, the European Court of Human Rights ruled as long ago as 1982 that a judicial tribunal in a criminal case can’t be deemed impartial if it includes someone who previously helped supervise the prosecution. And that’s in Europe, where prosecutions are supervised by career civil servants who are appointed, not elected, and where prosecutors are often described as “quasi-judicial” figures. In the United States, where prosecutors are far more adversarial, the conflict of interest is even more blatant. We’re not talking about Donald Drumpf ludicrously claiming that he can’t be judged fairly by anyone whose ethnicity he has defamed. Williams v. Pennsylvania was about—to borrow Justice Kennedy’s phrasing in his majority opinion last Thursday— “the same person serv[ing] as both accuser and adjudicator.”

It was the blending of roles that made the unfairness seem so blatant. Not even the dissenters last Thursday defended Castille’s refusal to recuse himself. Chief Justice Roberts, joined by Justice Alito, demurred on whether Castille’s participation in the case had been “appropriate”; Justice Thomas allowed that it might have been “unwise.” But the dissenters didn’t believe the Constitution had been violated. Roberts and Alito thought due process would have been violated only if Castille, when supervising the prosecution of Williams, had addressed the very same issue he later decided as a judge. That hadn’t happened, because the Pennsylvania Supreme Court didn’t review the prosecutors’ decision to seek the death penalty. Thomas insisted that recusal wasn’t required because the case before the Pennsylvania Supreme Court was a collateral, post-conviction challenge, not a direct appeal of the original judgment against Williams. That made it a different case, Thomas thought.

Behind this wrangling over rules, and interlaced with the long procedural history of Williams v. Pennsylvania, lay disagreements and uncertainties about roles: not just the role of a judge, but the role of a prosecutor, the role of defense counsel, and the role of the Supreme Court itself. Start with the last of these. Roberts, Alito and Thomas favored narrower rules about recusal because they wanted a smaller role for the Supreme Court in policing the fairness of state court adjudications, even in capital cases. Roberts and Alito stressed that the Due Process Clause required only “those practices that are fundamental to principles of liberty of justice [and] inhere ‘in the very idea of free government.’” For Thomas, the “touchstone of due process” was whatever English common law required before the colonization of the America. All three Justices were plainly worried about the Supreme Court gaining too much authority over state judicial proceedings.

Then there was the role of the prosecutor. The majority opinion in Williams emphasized the adversary nature of the prosecutor’s role—the way in which the prosecutor’s mindset differs fundamentally from the mindset we expect of a judge. That was sensible enough. Prosecutors are advocates. Often, though, the Supreme Court has insisted that prosecutors are more than advocates, that their function is to seek justice, not victory. That is part of why, for example, the Court has trusted prosecutors to determine in the first instance whether evidence is exculpatory and therefore needs to be turned over to the defense under Brady v. Maryland.

The precise reach of Brady—the extent to which prosecutors can and should be expected to serve not just as partisans but as eyes and ears for the defense—was part of Williams v. Pennsylvania, as well. The Pennsylvania Supreme Court rejected Williams’s Brady claims because the suppressed evidence involved things Williams himself already knew. The Supreme Court has never actually held, squarely, that prosecutors can ignore Brady with regard to information the defendant already has, and it’s far from clear that a rule like that would make sense. Defendants don’t always share what they know with their lawyers, and if the prosecutor’s job is truly to see that justice is done, there’s a good argument that they should share all exculpatory information with defense counsel—even information that defense counsel could conceivably learn from their clients. (That argument may have been particularly strong in Williams’s case, because the evidence in question pertained to his alleged sexual abuse by the murder victim.)

The role of defense counsel didn’t figure much if at all in the Supreme Court’s decision in Williams v. Pennsylvania, but it was an important part of the backstory. Williams was represented by the Capital Habeas Unit of the Federal Community Defender Organization for the Eastern District of Pennsylvania. During his last several years on the Pennsylvania Supreme Court, Castille was on something of a crusade against that unit. He was incensed by its involvement in state post-conviction proceedings, and even more so by its zealotry on behalf of its clients. In a 2014 opinion he deplored the Federal Community Defender’s “obstructionist,” anti-death penalty “agenda” and he complained that it devoted the kind of manpower to fighting death sentences that “one would expect in major litigation involving large law firms.” Castille reiterated his complaints in Williams; his grievances against the Federal Community Defender—which he complained was behaving as an “advocacy group”—were a major theme of his separate concurring opinion in that case.

Meanwhile, spurred on by Castille’s earlier opinion, prosecutors in Pennsylvania were seeking to disqualify the Capital Habeas Unit from representing defendants in state post-conviction proceedings. That effort was ultimately rejected by a unanimous panel of the Third Circuit. Judge McKee, a member of that panel and a former prosecutor, wrote separately to note the oddity of the concern raised by the state (and, originally, by Castille): not that “the Federal Community Defender is providing inadequate representation” but rather that it “is providing too much defense to the accused.” McKee couldn’t see why a criminal defendant condemned to die shouldn’t receive representation every bit as energetic and thorough as the commercial clients of major law firms.

The proper role of defense attorneys—how they should see themselves, where their allegiances should lie, what they should take as their goals and their obligations—is a defining feature of our criminal justice system, and it remains in many ways a matter of debate. Defense attorneys are in this respect no different from Supreme Court Justices, lower court judges, prosecutors or, for that matter, the police. All of these officials play critical roles in determining how fairly, how effectively, and how humanely the system operates. And getting their roles right is at least as important as getting the rules right.

David Alan Sklansky is the Stanley Morrison Professor of Law, faculty co-director of the Stanford Criminal Justice Center, and a former federal prosecutor. He teaches and writes about criminal law, criminal procedure, and evidence.