In March, I argued in this space that Nippon Life v. OpenAI, filed in Illinois, should be viewed as a product liability case. The reasoning was that OpenAI released a system that crossed the uncrossable threshold, the line between legal information and a tailored legal conclusion, and nothing in its architecture stopped it. In this post, I examine the role sycophancy plays in carrying a conversation across that line. Sycophancy does not define legal advice, and a system can give legal advice without a trace of it. But sycophancy is a defect and an (if not the) ingredient that morphs an explanation into an endorsement, and from there the distance to a legal filing deluge is short. Holding the AI tight to the legal-information side of that line requires controls that are keyed to the uncrossable threshold and a test for preference sensitivity that clearly signals when a system has started building the user’s case around the answer the user wants.

Graciela Dela Torre settled a long-term disability dispute with Nippon in January 2024, and the case was dismissed with prejudice. A year later she asked her lawyer to reopen the settlement, and he refused, explaining that she had signed a release. She uploaded his response to ChatGPT and asked whether she was being gaslighted. The system said yes. She fired her lawyers, asked ChatGPT how to vacate the agreement, and filed a motion to reopen under Rule 60(b) that the complaint says ChatGPT drafted. According to the complaint, dozens of filings followed across two lawsuits: the previously settled Nippon case she was trying to reopen, and a new action she filed against Davies Life Health and Allsup, LLC. After the court denied her reopening motion, she amended the latter action to add Nippon as a defendant.

In January 2012, in my “Computational Law Applications and the Unauthorized Practice of Law,” I wrote that the uncrossable threshold is traversed when an AI system moves from comparative information to a tailored legal conclusion about a user’s specific situation. This is what Nippon alleges happened. The Illinois Attorney Act prohibits both the unlicensed practice of law and the unauthorized provision or solicitation of legal services, and case law supplies the governing definition. The practice of law encompasses giving advice to clients and taking action for them in matters connected with the law, along with preparing pleadings and other papers incident to actions and special proceedings. The Illinois Supreme Court articulated that formulation in People ex rel. Illinois State Bar Association v. Peoples Stock Yards State Bank, 344 Ill. 462 (1931). It has also repeatedly reaffirmed that legal practice extends beyond courtroom representation to advice and other services requiring legal knowledge or skill.

Personalization makes the threshold hard to locate, so I break it into four stages. In stage one, a system tailors an explanation to the legally relevant facts the user supplied. No problem there; that is merely providing legal information and requires no license. In stage two, the AI applies the law to the user’s circumstances and says how a rule bears on the case. That is where the threshold sits. In stage three, the AI changes its assessment because of the conclusion the user wants. And in stage four, the AI recommends action or produces a legal instrument. Sycophancy lives at stage three, and it is what makes the second stage lead to the fourth. Agreement with the user is not, on its own, sycophancy, because the user may be right. Tailoring the output is not sycophancy either, because the facts matter. Sycophancy is the user signaling the conclusion she wants and the system bending its assessment to match a preference that should have no bearing on the outcome.

Dela Torre’s gaslighting exchange shows the mechanism. Asked whether her lawyer’s letter was manipulation, ChatGPT rendered a conclusion about a specific legal relationship, which moved it to stage two. ChatGPT then rendered the conclusion Dela Torre asked for, which is stage three. Nippon’s complaint illustrates the risk that a sycophantic system may function as an apparent advocate. It quotes Dela Torre describing ChatGPT as a tool designed for pro se litigants trying to navigate the legal system without counsel, and it alleges that ChatGPT, having validated her reading of the letter, assisted her efforts to challenge the settlement. From there, the path to the Rule 60(b) motion was short, which is stage four. Drafting the pleading would have been unauthorized practice under the Illinois definition with or without the flattery, but sycophancy transforms the conversation from an explanation to a filing.

Nippon says ChatGPT is intentionally programmed to keep the user interacting so OpenAI can collect training data, and that the legal assistance Dela Torre received was shaped by that objective. On this account, ChatGPT agrees because agreement keeps the user talking. The UN’s Independent International Scientific Panel on AI, in its July 2026 preliminary report (at 24–25, §§ 2.7–2.8), describes the general mechanism the same way: “Because humans prefer responses that agree with them, AI chatbots have developed sycophancy, the art of offering exaggerated flattery, to prolong interactions and create emotional attachment.” The panel adds that systems “rewarded for validation rather than accuracy or care remain largely ungoverned.”

The staging places sycophancy at stage three, but staging alone cannot show whether a given system carries it. That takes a test for preference sensitivity. Give the system the same facts and the same neutral task twice, assess this letter, and vary only what the user seems to want. One user hints that she hopes the letter is wrong. The other hints at nothing. Compare the conclusion, the confidence, the authorities cited, the treatment of adverse law, and the recommended action. Then change a fact that should matter, say the release was never signed, and see how the answer moves. Run the same protocol across multiple trials and across whole conversations, because a system can pass with one question and still drift toward sycophancy as the exchange lengthens.

The test has three attributes. First, it targets only the user’s preferred legal conclusion. Second, it scores system independence. If the system disagrees as a default, that is as defective as reflexive agreement. Third, it scores responsiveness, and a refusal to engage counts as an answer. Combined, the stages and the preference-sensitivity test show that sycophancy can be located and proven.

But proving the defect is not preventing it. A counterfactual test run before release tells the developer whether the model drifts. Prevention is what the developer does next. Hold back a model that fails the test. Tune against the drift. Watch for it in production. Escalate a live conversation that shows it.

The remedy is where the design question becomes unavoidable. Nippon asks the court to enjoin OpenAI from practicing law in Illinois. A system can obey that order only through its architecture, so the injunction comes down to an order to redesign, and a court would have to specify which outputs must be refused, in which contexts, and how compliance is measured. That is the uncrossable threshold written as a decree. An output-based injunction with no measurable compliance criteria cannot be administered, and a court is poorly placed to write the criteria from the bench. Someone has to write them.

In March, I called for deterministic guardrails, hard-coded refusals of tailored legal conclusions that no user instruction could override, and I think that approach still makes sense. The question is when the refusal should fire. The first time a response speaks to the user’s own case is premature. Fired at stage two, the refusal blocks the harmless personalized explanation, which for people who cannot afford counsel is the only readily available legal help. And it aims at the wrong moment. Sycophancy does not cross the threshold in a single sentence. It carries the conversation across one agreeable turn at a time.

A layered guardrail design keeps general information available and introduces friction as the conversation becomes specific and legally consequential. Two triggers supply the necessary friction. A boundary trigger fires when the system is about to produce an individualized recommendation or a document meant to be filed or signed. The system then asks for missing jurisdictional facts, separates what the user asserts from what the record shows, surfaces the counterarguments, calibrates its confidence, and slows down. None of those steps renders a conclusion about the user’s case, and none meets the Illinois definition. Questions, sorting, and caution are not advice or action taken for a client in a matter connected with the law, and nothing in the sequence prepares a pleading or other paper incident to an action. A heightened-risk trigger fires on the introduction of facts such as deadlines, criminal exposure, an adverse final judgment, or an existing lawyer-client relationship. The system then stops short of continuing with the task and directs the user to licensed counsel. Dela Torre’s conversation would have tripped both triggers.

The FTC recently supplied a regulatory analogy. Its July 2026 proposed policy statement on the suppression of accuracy in AI systems says that steering a model’s output toward objectives other than the ones the user asked for, without disclosure, can deceive the consumer under Section 5. Issued under Executive Order 14365, which directed the Commission to address how state laws requiring alterations to otherwise accurate model outputs may conflict with Section 5, the statement is principally concerned with state AI laws, particularly liability rules that, in the Commission’s view, pressure developers into altering outputs to avoid liability. But its analysis reaches any undisclosed steering toward objectives the user did not request or reasonably expect, whatever the motive. The Commission’s statement nowhere names engagement optimization or sycophancy, so applying it to a reward function is my inference, and a Section 5 claim built on that inference would still require proof of a misleading representation or omission, measured by a reasonable consumer’s expectations, and materiality. But the fit is close enough, because an engagement objective is an objective other than the one Dela Torre asked for.

Now back to the Dela Torre conversation. Legally relevant facts and objectives should move the answer. The user’s desired legal conclusion, never. ChatGPT could have limited itself to explaining Rule 60(b) to Dela Torre, telling her what the rule requires and how courts treat signed releases and dismissals with prejudice, directing her to licensed counsel, and leaving her better informed than she was. It should not have let her preferred narrative dictate its assessment of her lawyer’s letter, and it should not have gone on to generate the filings. It did both, by design.