Irene Liu: Welcome to AI Sidebar. I’m Irene Liu, your host and executive director of the AI Initiative at Stanford Law School. Today, I’m excited to welcome Jason Boehmig, who leads OpenAI’s legal vertical as their GM. Jason brings a particularly interesting perspective to this moment in legal tech. Before joining OpenAI, he founded Ironclad, where he helped pioneer the concept of legal engineering and spent years building technology for lawyers.

So today, Jason and I will talk about the different cycles of legal tech Jason has seen, how legal engineering is evolving in the AI era, what it means to build AGI for law at OpenAI, and ultimately what the role of lawyers looks like as these systems become more capable. So, with that, let’s jump into my conversation with Jason Boehmig.

Jason, thank you so much for joining us on the AI Sidebar.

Jason Boehmig: It’s great to be here. Thanks for having me.

Irene Liu: So, Jason, you spent years building at Ironclad, and now you’re leading OpenAI’s legal vertical. So, can you tell us what drew you to this role, and what feels fundamentally different about building for the legal industry today versus when you built Ironclad back in 2014?

Jason Boehmig: Yeah. So, it’s been really fun being here. I’m kind of a builder at heart, and when I started Ironclad back in, in 2014, honestly, some of my favorite Ironclad memories were those early days of just getting to build for the first time and really approach the space with fresh eyes and with no priors.

And so, that’s really what kind of drew me to OpenAI, was this chance to do the zero to one thing, but at a scaled company. And that juxtaposition has been really interesting for me of, like, very much zero to one, very much early stage startup vibes almost, but obviously the company has enormous scale and enormous reach, and enormous resources. So, I just felt like it was such a special opportunity that I kind of couldn’t pass it up. And I, I’d also known some of the leadership here for a while, so it was fun to potentially get to work with them.

I think the industry is in such a different place now, and legal tech is a thing, which is really exciting. The state of the industry feels really different this time around in an awesome way. It’s a vibrant ecosystem, and a lot of what I’m thinking about in my role here is, how do we make that ecosystem really successful? Because a lot of it is built on the OpenAI platform. And now that we’re doing this vertical in a first-class way, like, how can we be an even better partner for these companies that have been able to do some really incredible things by building on, on the OpenAI platform?

Irene Liu: Yeah. I mean, just even looking back at 2014, for example, you actually helped create some of the legal tech ecosystem with Ironclad. So, when you think about 2014, was there a lot of interest there? And I know you’ve also mentioned to me how you feel like you’ve lived through multiple legal tech cycles.

Jason Boehmig: Yeah.

Irene Liu: So, what were some of those tech cycles, and what was it like in 2014? And obviously it’s much vibrant today, but what happened in between that created that vibrant ecosystem?

Jason Boehmig: Yeah. Well, one of the things that really convinced me to quit my job as a law firm associate and start Ironclad was this idea that software was becoming more collaborative, and that that could change the way that you thought about traditional market sizing.

Because when you looked at the; in 2014, people thought that legal tech was a bad market. One, there’s this perception that lawyers are, you know, change resistant. But two, if you looked at, like, the number of seats in an organization, legal just wasn’t that big of a seat count. In fact, it was, you know, really rarely were people adding to the legal seat head count. So, you’d have, like, two lawyers, and you’d have hundreds of GTM people. So, if you were selling seats, you’d be like, “Why would you wanna sell to the two-person team when you could sell to the hundreds-person team?”

Part of what convinced me that it was a good idea was this idea that software is becoming more collaborative, and you could look at not just the size of the team you were selling to, but the size of the team that you were selling to plus one layer of collaboration.

And then all of a sudden, legal became, like, super interesting because those two people were collaborating with all of the GTM users. So, the surface area of the legal team for collaboration was actually huge, and maybe one of the biggest teams for collaboration. And we’ve definitely seen that play out, I think, obviously with Ironclad, with companies like Figma. Design is a, I always thought was a really similar thematically industry in that there’s not a ton of designers, but they sort of collaborate with everybody. And if you can build something that helps them do that, it’s amazing.

I think we really proved that out in legal, and we also kind of, like, proved out the, the fact that you could build a good traditional SaaS business by selling to corporate legal teams as, as opposed to just going after, like, what people think of when they think of legal tech, which is law firms.

What’s really fun about getting to come back and do the zero to one thing now is that a lot of things I was interested in at the time but just, like, didn’t make sense to commercially work on are now kind of here. And in some ways, I feel like I’m, like, getting to relive almost what I wanted to do when I started Ironclad, which was think about how AI is going to change the nature of work that attorneys do.

And we actually started as, as Ironclad.ai, and our first customers were law firms. But pretty quickly realized, one, the AI is not there to a point where people can really rely on it to be as precise as they need it to be in the legal profession. And two, the law firms weren’t really ready for that type of change yet. That led us to in-house, and that led us to workflow automation.

But now those two things are really true. Law firms are really ready for change. And when I talk to law firm leadership, they’re there. And they understand that this is a time of great change and great opportunity for those that embrace it. And then two, obviously the technology is incredible, and it’s already very useful for legal work. I think it can be made even more useful, and we’re working on that. So those things being true is, like, really exciting to me, and part of what drew me back into how can we build zero to one here.

Irene Liu: Yeah.

And what’s really exciting what you were saying is that lawyers are really embracing AI. They are embracing the change. And like you said earlier, lawyers were resistant to change before, and now we’re entering this new phase where they are embracing change and embracing AI in a way that they weren’t doing in the past.

Jason Boehmig: Yeah.

Irene Liu: And so now that the ground is ready, how are you building from zero to one at OpenAI?

Jason Boehmig: Yeah. So, it’s cool to get to do this in a different landscape where there are a bunch of thriving legal technology companies, and I think that’s really exciting. As someone who, like, when I started on Cloud, I was like, “Man, I hope someday there’s a sector and there’s a bunch of technology companies doing the really, truly wide variety of things that you can do in the legal sector.” That’s a reality now.

And I, I think the most foundational thing that I wanna make sure we execute on at OpenAI is that we are the platform for anyone building in legal technology. Obviously, a big part of that is the legal tech ecosystem. Another part of that is law firms are now builders. And there was some hints of this happening 10, 12 years ago with people hiring innovation teams and starting to experiment with automation. It’s now almost universally the case at top firms that they have a true first class innovation team with engineers and AI people, and they’re really building.

And if you talk to them about how they view themselves, a lot of them will say that they view themselves as an AI company, and not necessarily as even a law firm as their number one thing that they’re doing. I think that’s fascinating. And that’s really how I want to show up to the people building, is to arm them with the tools that they can then take and assemble these amazing things. Whether that’s a technology company, whether that’s a law firm, whether that’s an in-house team. More raw materials than finished product maybe.

Irene Liu: Yeah. With the raw materials, one of the things that you recently launched is Astra for Law. So do you view that as one of the raw materials for them, for-

Jason Boehmig: Yeah

Irene Liu: … this, this team of lawyers? And what does Astra for Law actually provide for these teams that are innovating within law firms as well as in-house teams?

Jason Boehmig: Yeah. So, there’s sort of been a choice that people building have had to make at the model layer, which is do you wanna stay on the frontier? And staying on the frontier is great because the frontier moves really fast, and so you get the benefits of the state-of-the-art immediately if you stay on the frontier. If you stay on the frontier, you’ve historically had to do more work to get the output of the frontier to be relevant for the domain that you’re working in, in this case law. But you can sort of like go off of the frontier and get a short-term boost by, for instance, doing something like post-training, a legal specific model.

And I think there’s certainly value in doing that. But- I think what we found as an industry is that post-training around a particular domain has a very short-lived, lifespan because you go off of the frontier, and there, if the frontier moves, the baseline capabilities might quickly eclipse the short-term benefits from post-training.

So, we looked at a bunch of different options and ultimately came to this conclusion that we were gonna do something a little bit new for the industry and OpenAI, which was to offer a model configuration. And the attempt here is really to hit both of those benefits, to allow folks to stay on the frontier and get all of the benefits of that, but also have the outputs be a little bit more relevant to the domain, so you have to do less work to get them to be relevant. And there’s really three techniques we use to do that.

So the first is instructions. The latest models are extremely good at following instructions. Now, if you tell the model what to do in a very precise way, it generally does it, and I think that that is only going to become more and more of a capability as models continue to progress. So, we’ve delivered it a very opinionated set of instructions and context around the way that lawyers work, the way they like to write, the way they like to receive information, even things about background context of rules of professional responsibility and how lawyers are educated. Giving those as instructions to the model really helps make the outputs a lot more relevant to the domain.

The second is settings. So, settings is, you could think of as the admin panel, and we have a super-secret menu of admin settings that we can do at OpenAI, which some of which aren’t accessible to even API customers. To give one example that I kind of think is funny, it’s basically a setting that tells the model how verbose to be in its responses and how diligently to work. And so, we’ve cranked that up to, like, 11. The highest setting available to an API customer is lower than that because obviously lawyers tend to prefer verbose responses, and they like a lot of diligence and thoughtfulness in the responses that they get. So, we kind of have the settings tuned to the outputs that lawyers prefer and work for the profession.

Irene Liu: The verbose part is really interesting because I think not many in-house people will prefer verbose responses because they have to present to the businesspeople, and they may not want verbose responses. But maybe you’re right, that they might want verbose responses right away for them to synthesize, and then they will present that-

Jason Boehmig: Yeah

Irene Liu: to the business side.

Jason Boehmig: Yeah. And so, I think, where that tends to show up is in a longer memo. So, you know, if, if you’re doing a, a filing or a court pleading, you might expect it to need to recite all of the facts, for instance, and not just, like, summarize the facts. So, what that setting helps it do, especially in line with the instructions, is actually deliver all of the elements as opposed to summarizing, and not necessarily, like, adding additional-

Irene Liu: So, it’s not flowery words, but more details.

Jason Boehmig: Exactly. So, like, the diligence part of that is that exactly, so it won’t leave things out setting.

Irene Liu: Interesting. And what’s the third?

Jason Boehmig: Yeah. So, the models have access to tools. The one most people are probably familiar with is the web search index. So, if you’ve used ChatGPT and it’s searched the web for you, then that model has access to a tool that’s the web search index.

In this case, we created a new special index, a legal search index, and we gave it access to this; Astra for Law has access to the legal search index, and legal search index has relevant information that applies to legal search, the most obvious of which is case law. So, we have 99.9% of US case law, and we work with folks like the Free Law Project to bring that amazing work into the context of the legal search index. So now, it has an ability to first party primary sources that are relevant to law that you might not even wanna have base Astra because it might skew the results. So, we’re able to bundle these tools that are domain specific into this configuration.

So the combined output of those three ends up being really powerful because now you’re on the frontier, but you also have a lot of domain specific context and sources that can be pulled in there. And the great thing is, like, this is only gonna get better and better and better over time. So, it has access to one headline tool now. I’d expect it to have access to more tools in the future. We can continue to prime the right settings and instructions as well. So, this configuration is something I’m really excited about for the industry.

And what’s cool about that too is you can use it in ChatGPT if you select the model selector and put it over at Astra for Law if you have access, or you can use it via the API as well. So, builders can build directly with Astra for Law and build it directly into their applications as well, just as they would a different model. Although this is a model configuration as opposed to a model.

Irene Liu: And since it’s a relatively new product, who has access to Astra for Law?

Jason Boehmig: Yeah. So, we wanted to start with the legal tech ecosystem and the Am Law 200. We plan to roll it out more broadly over time, but that’s the initial target market. And the commonality between those is going back to the concept of the builders. These are folks with engineering teams and deployment teams that can help us rapidly iterate and make the model configuration better and better over time but also have the sophistication to deal with taking this at the API level and building it into a product experience.

Irene Liu: And a lot of law firms are obviously concerned about accuracy, especially since they’re hired by companies for their expertise and for accurate information. And so how are they trusting Astra from an accuracy standpoint? You mentioned 99.9% of case law is available to the Astra for Law product. How does that enhance the accuracy, and what’s the response from the law firms around the accuracy aspect of the model?

Jason Boehmig: Yeah. So, accuracy’s really important in law, and mistakes are really costly in law in that it’s not a move fast and break things industry. It’s a move as fast as you can without breaking things industry.

So, probably the best tool we have for measuring that is benchmarks. Did a benchmark on Astra for Law relative to regular Astra, ’cause that’s the main thing that we care about is like, is this worth using rather than base Astra? And I believe we saw a 13 percentage points improvement on the legal research task, which is really where we’re focused at the beginning. So, the real meaningful gain, this is just the start of what I expect to be even larger improvements in the future.

But to answer your question more philosophically on how law firms should think about this and how I think they are thinking about this, I very much believe in human in the loop here, and the world in which I think we are headed is there absolutely should be a human checking this and directing the tool. This is not designed to be an autonomous thing that’s just out there producing a filing, and you find out about it after it’s pinged to the court about it. This is designed to assist the human in the practice of law.

And something that I am thinking a lot about is how can we make the human’s job of checking the machine easier? So, I think you can expect to see some things from us in the future that will really help increase confidence in what the machine is doing and allow for easier double-checking of what the machine does. I think that that is like a really important design criteria in the design of legal systems is how do you build that trust with the end user?

Irene Liu: And you mentioned earlier that Astra for Law, one of the key components of it is that it follows instructions really well. But we’ve been hearing about a lot of agents not following instructions really well. So, in light of all of that, is there anything that makes Astra unique in a way that it will follow it more and that there’s more trust factor that can be established around Astra for Law versus the other models that have been in the news?

Jason Boehmig: Yeah. Well, I can’t really comment on the larger industry-wide issue in a unique way, other than to say that I think safety is extremely important, and we think a lot about safety in what we’re doing here as well. It’s particularly important because lawyers work with clients on really the most important matters of their lives, and there’s real liberty and property things at stake when you’re dealing with legal issues. So, we really work closely with our safety team, and we think about safety a lot here on my team, the legal industry team. Although I do think that you’ve correctly flagged the larger industry thing there.

Irene Liu: One of the things that you mentioned that’s so interesting is that you are focused on building at the model level versus the application layer, but there is a robust ecosystem that is operating at that layer. And so, how did you decide to choose the model layer versus some of the approaches by other model companies that are approaching their models for the legal profession?

Jason Boehmig: Yeah. I think about this as like how can we as humans get to the end result as fast as possible? And the end result I think we’re trying to get to is a world in which we can return the lawyer to being more of a counselor and advisor and less of a paper pusher.

I think there’s incredible work being done at the application layer, and I don’t think there’s as much unique value for us to add there. And the thing that only we can work on is pushing the frontier for legal. There’s like two, three companies in the world that can credibly work on pushing the frontier for legal ahead. So, it is a tremendous value for us if we get that right, and something that we uniquely have the right to win.

That’s not to say we’re not gonna have application layer things that people can use. We have ChatGPT. ChatGPT is amazing. Really what we’re focused on doing there, though, is like giving people the tools in ChatGPT to build whatever they want. And so that’s how I think of ChatGPT, is less of a like out of the box, it’s configured for your exact use case, and more about like ChatGPT can be a chameleon for whatever you particularly wanna do.

And that’s some of the things that we demoed at our launch event were Ropes & Gray doing this really cool diligence thing in ChatGPT. We, Cooley has GO Public this amazing S1 creation engine that they’ve been able to build in ChatGPT. And Sullivan & Cromwell built an amazing agreement analyzer. So, these are things that before would’ve been custom applications that they made, and they didn’t even need to hire developers to make them. They were able to build those experiences directly in ChatGPT.

So I think about our application layer as how can you make ChatGPT Do anything you can dream up. And that is a really fun place to get to show up for these firms. Now they have a place to go where they can build whatever they can think of.

Irene Liu: So, at the launch event, you had a number of law firm partners, and you also mentioned that you partner with 200 Am Law law firms as well, and their innovation partners at those firms. And so, we’re really seeing an emergence of a lot of innovation leaders within law firms, as well as in-house and legal teams.

And one of the terminologies that we hear about these innovation leaders is this talent group called legal engineers. And when I think about legal engineering, Jason, you and I crossed paths actually a decade ago when you were CEO of Ironclad, and we were talking about your product, Ironclad, and how legal engineers can help implement.

So, one of the first times I’ve ever crossed paths on that terminology of legal engineer was with you at Ironclad. And so, do you think you coined the legal engineer term? Was that term ever around before? Can you tell us a little bit about that history of that term?

Jason Boehmig: Yeah. That’s a really fun one, and you’re a good memory for remembering that back from 2017.

Yeah, so we hired our first legal engineer in 2015, actually. And it was really my super naive idea that we were only gonna have two roles at Ironclad. We were gonna have software engineers and legal engineers. And obviously I don’t think that that would be possible to do that, but that was our initial genesis of it.

And so actually, our second employee was a legal engineer that we hired who was a former paralegal. And at one point, probably around 2017, if you typed in legal engineer into LinkedIn, like you got the 12 people that worked at Ironclad and had legal engineer as their job title.

And what we meant by that was grounded in this observation that there is a lot of technical knowledge that is not software technical knowledge that you sort of need to have in order to engage with the legal audience. This is, I think, the most sophisticated form of knowledge work. It’s highly specialized and it uses technical terms and jargon, and if you don’t have that context, you’re not gonna be successful no matter how technical you are at actually implementing a solution that works for that field. So, we really kind of created it out of necessity of there needs to be some gap that is technical, but not necessarily like code technical, legal code.

It was very successful for us because I think we were able to meet the customer where they are, and we were able to come into conversations with that technical knowledge around the profession, and it helped produce better deployments for us. So yeah, now it’s like super the rage in the industry. I do believe that we created the term. We had 98, if not 100% of the legal engineers in the world working at Ironclad at one point. Yeah, it’s been cool to see that evolve.

Irene Liu: Yeah, and so at OpenAI, do you have legal engineers, or if not, what do you call them there?

Jason Boehmig: Yeah, so it’s really become a full circle moment.

We call them forward deployed engineers here, which was part of the inspiration for our legal engineer job title at Ironclad is my co-founder had come from Palantir, and we had a number of early employees that came from Palantir as well, and I think Palantir really popularized, if not created, the forward deploy engineer archetype.

What that’s meant here at OpenAI is leaning a little bit more on the technical side than the legal technical side because there’s so much extensibility in our platform and the ability to really kind of like show up as a software engineer to help our customers get the most out of the customization and like raw power that exists both at the API and at the, the ChatGPT level has been more important.

At Ironclad, it was more the software is fairly configurable, and you need less technical expertise, but you need more legal technical domain experience. Here it, it’s kind of like swung back to being more on the software engineer technical side. And I also kind of believe that despite us having done it with the legal engineer job title, it should be a pretty high bar for creating a new job title, and forward deployed engineer is one that is well-received and embraced here at OpenAI, so didn’t necessarily wanna invest in the new job title time around.

Irene Liu: Definitely. But when you think about that legal engineering title of back then and today, and even like you said today when you’re thinking about it as a forward deployed engineer, do you think the talent that was required back then is different from the talent that it requires today given how far the technology has moved?

Jason Boehmig: Yeah, I think that’s probably true. One of the funny things too about the legal engineer thing was the biggest team management challenge there was people feeling like they didn’t have a career advancement thing because the job title only existed at one company. So, the constant refrain on that team for a few years was like, “What is my career gonna look like, because there’s no other legal engineering jobs?” and I would always say, “There’s gonna be a lot more legal engineering jobs, and you’ll be one of the first. It’ll be great.”

But I think where I’m going with that, is this idea that the work that the job function is doing needs to map to what the company and the customer need, and we really did find a lot of that fit with the legal engineers. The customers really needed someone who could translate, understand the legal technical jargon and translate it into an actual like software architecture.

Here, I think it’s more about taking a big idea and figuring out the like technical implementation and surface area where that should live. Should it live in ChatGPT? Should it live in the API? I do think that the forward deployed engineer does map better to the problem and the gap that that person ends up filling for getting the customer to the end result.

Irene Liu: Yeah, and one of the things that we’re hearing is that there is a demand for legal engineers, and yet it’s really hard to find the talent. So, one of the things-

Jason Boehmig: Yeah

Irene Liu: … that Stanford Law is doing is we’re building a legal engineering academy, and that’s launching in summer 2027. But before then, how did you find your forward deployed engineers for Astra for Law? Like, what pipeline did you use to find someone that can be fluent in both, particularly the technology side?

Jason Boehmig: Yeah. So, I think it, it’s a little bit easier actually when the job is more, focused on the technical aspects because you then need to just really test that, you know, can someone walk into a room of senior law firm partners or tech executives at a legal tech company and be able to hold that conversation and kind of like drive it towards a productive outcome.

That’s an easier skill set than the legal engineer piece was at Iron Cloud, where you did need some level of technical expertise, but you also needed someone who knew enough about the profession. There is extreme levels of specialization in law, and finding someone who could be able to be enough of a generalist to be able to talk to a litigation person and then a corporate contracts person and find the commonalities between that was a lot harder.

So I feel like it was kind of a motley crew of amazing people that had that overlap. That we had people from Yale Law School, the people that would’ve been like paralegals at Dropbox, to a former paralegal that I worked with at my old law firm. And they had enough of a baseline that they could translate it into the way that that worked.

Irene Liu: Well, you’re building again, and one of the interesting things that I saw on your LinkedIn is that your LinkedIn title says that you’re building AGI for law. So, what does building AGI for law actually mean to you?

Jason Boehmig: For me, AGI means that the machine can do tasks that we would normally think of having to be done by people in a sort of like indistinguishable way.

And it forces me to think about the world that we’re building, to look through that lens of like, okay, if there’s individual things that a lawyer can do can be done by a machine, what does the lawyer’s job look like? And what is the world in which we’re building towards look like? And, like, what do we need to be true in order to have that be the version of the world that we wanna live in?

AGI for law for me does not mean that we have like a bunch of agents and you don’t work with a human lawyer and you just work with like a team of agents that’s representing you and, like, filing lawsuits for you. That’s not the world that I think is the good world to be living in. What it does mean though is that, like, your lawyer is delivering much more effective service, and that lawyers are able to deliver service into domains and like segments that previously were not addressable.

And in particular, like something I’m super passionate about is making sure that this technology benefits all of humanity. And, you know, OpenAI’s mission is to ensure that AGI benefits all of humanity. And for me, like that was a huge reason in taking the job, is that I wanted to make sure that the technology we’re building here at OpenAI and, and the industry generally truly does benefit all of humanity.

And I really think that one of the most high impact things we can do as a company to make sure that we achieve that mission is to deliver high quality legal services through, in my opinion, scaled attorneys, not replacing them. It is going to have a tremendous impact on all of humanity, not just like the elite. Even though from a product perspective and a commercial perspective, we may start with that. So yes, that is very important to me.

Irene Liu: So, you’re still viewing a very human lawyer-centered world in this AGI for law?

Jason Boehmig: Yeah. Primarily that’s because I think that the range of tasks that lawyers do today that can eventually be done by the machine is quite large. But I think that the fundamental nature of the lawyer can’t be done by a machine.

And I think anyone who has had a great lawyer assist them through a consequential legal matter intuitively knows that and knows that it’s not just the briefing that they prepare. It’s not just the filing that they make. It’s not just the contract. It’s the partnership and care that goes alongside of you. And so, I think the lawyer’s job will be more of the client guidance and advice than it will be the sort of like raw materials that need to happen in order for that more irreplaceable part to exist right now.

Irene Liu: So, what are you most excited about building next in the legal vertical?

Jason Boehmig: Yeah, it was really fun. Thanks for, for hosting me.

Irene Liu: A huge thank you to Jason Boehmig for joining us and for sharing his perspective on where legal tech has been, where it is today, and where it may be heading next. I especially appreciated our conversation about the evolution of legal engineering, the importance of keeping lawyers in the loop as AI becomes more capable, and what it actually might mean to build AGI for law.

And thank you for tuning in to The AI Sidebar. If you enjoyed today’s conversation, please follow the podcast and share this episode with someone who is thinking about the future of law and AI. Until next time, stay curious and keep learning.