Compound AI: Where Logic meets LLMs

This project investigates how deterministic AI technologies like computational law, probabilistic AI technologies like deep learning, and other technologies can be combined to solve legal reasoning tasks better than using one of these technologies alone. Combining these systems allows us to benefit from their different strengths while compensating for their weaknesses.

Artificial intelligence (AI) programs generally fall into two primary categories: deterministic and probabilistic, each with distinct methodologies and applications.

Deterministic AI systems operate strictly on predefined rules (e.g., logic programs) to extrapolate new information from known data. A key advantage is their transparency in decision-making. They provide clear, predictable outputs and expose their reasoning processes, making it easy to modify outcomes by adjusting rules. Additionally, they require less computational power, translating to faster processing and reduced costs.

Probabilistic AI systems also use rules (kind of), but these are derived differently. Instead of being explicitly programmed, these rules are inferred through the analysis of large datasets to identify patterns. This process introduces uncertainty since statistical probabilities do not guarantee accuracy. These rules, complex mathematical models, do not inherently carry meaning and are not readily interpretable by humans. Consequently, probabilistic systems lack explainability, and their outcomes are less predictable. This unpredictability can be useful, resembling creativity, but can also be problematic in domains requiring consistency, such as legal judgments.

Exclusively using deterministic systems is impractical because codifying every possible scenario without introducing conflicts or excessive complexity is infeasible. Deterministic systems struggle with interpreting non-formal information such as natural language, images, and audio, which do not easily lend themselves to rule-based analysis.

This is where probabilistic AI is advantageous. Though not perfectly accurate, probabilistic systems handle ambiguity and complexity better, producing sufficiently accurate interpretations where deterministic systems fall short.

Considering both strengths and weaknesses, a hybrid category emerges as a promising solution. Combining deterministic and probabilistic approaches, such hybrid systems leverage the control, predictability, and efficiency of deterministic reasoning while benefiting from the adaptability and broader scope of probabilistic reasoning. This approach harnesses the best of both worlds, effectively addressing real-world complexities.

Project lead: Paul Welter


The descriptions of current and past projects of CodeX non-residential fellows are provided to illustrate the kind of work our non-residential fellows are carrying out. These projects are listed here for informational purposes only and are not endorsed by CodeX, Stanford Law School, or Stanford University.