When law professors were asked to choose between answers written by their peers and those generated by AI, the machines won—decisively.

In a blind study led by Professor Julian Nyarko, co-chair of the Stanford Law AI Initiative, 16 professors evaluated nearly 3,000 responses to the kinds of contract law questions students might ask after class or during office hours. They preferred the AI-generated answer 75 percent of the time.

Even more striking, professors flagged just 3.5 percent of the AI responses as potentially misleading or harmful, compared with 12 percent of the answers written by other professors.

“We were frankly surprised by the magnitude of the results,” says Nyarko, who leads Stanford Law School’s Legal Innovation through Frontier Technology Lab, or liftlab.

The study, “Law Professors Prefer AI Over Peer Answers,” suggests that AI tutors may be capable of offering useful support even in a field built on ambiguity and competing arguments. The next challenge, Nyarko says, is how to use AI tools without weakening the deeper learning law school is meant to foster. SL