[00:00:00] Irene Liu: Welcome to AI Sidebar. I’m Irene Liu, your host and executive director of the AI Initiative at Stanford Law School. Today, I’m so thrilled to welcome Mary O’Carroll, CEO of LegalEng Consulting Group and a legal industry leader who has been at the forefront of legal operations, legal technology, and legal engineering for more than two decades.

[00:00:22] Mary has a rare perspective across in-house legal departments, law firms, legal tech companies, and consulting. So today, we’re going to talk about the evolution from legal operations to legal engineering, the growing talent gap inside legal teams, and why the next critical role may be what we’re calling the legal solutions engineer, the person who can understand the legal work, redesign workflows, and make technology and AI actually work in practice. So, with that in mind, let’s jump right into my conversation with Mary O’Carroll.

[00:00:59] Mary, thank you so much for joining the AI, Sidebar.

[00:01:02] Mary O’Carroll: Thanks so much for having me.

[00:01:04] Irene Liu: So, Mary, I can’t think of anyone who has had a greater influence on legal operations than you. You’ve had such an incredible career at Google, and then you’ve had legal ops leadership roles at Clock, Ironclad, Goodwyn, and now CEO of LegalEng Consulting Group.

[00:01:21] So in all of these roles, you’ve seen iterations of legal ops, and now you’re seeing another iteration in this age of AI. And so now that you’re looking at it, and you’ve seen so many variations of this profession, how should the legal operations role and the team evolve in this age of AI?

[00:01:39] Mary O’Carroll: You know, it’s so interesting.

[00:01:40] It’s been over 20 years that I’ve been in this space that we now call legal operations, and it now has a name. But the role itself, it’s, it’s still brand new. It’s still nascent, you know, even, even after two decades. But because it’s been evolving this whole time, I think at the end of the day though, like, what it stands for has stayed the same.

[00:02:00] So, when we try to think about early on, like, what’s the elevator pitch? How do we describe this in a really easy way to understand? And I think the way that I’ve always used, I use the term ‘rightsourcing’. So, our job as legal operations professionals on in-house legal departments is to ‘rightsource’ the work, meaning looking across your organization, seeing all the work that’s coming in, and figuring out, okay, I’ve got all these resources at hand.

[00:02:24] I have in-house counsel, I have outside counsel, I have technology, I have alternative service providers, I have non-attorneys. Where should the work go in order for us to optimize the balance of speed, quality, and the cost of legal services delivery? That’s it. It’s pretty simple, like looking at your resources and constantly rebalancing, and it’s not a one-time thing, right? You’re, you’re constantly looking at how the work is changing and how the resources are changing and how your department is changing.

[00:02:50] And what AI has done is just added another tool to the resource list. And so, now we’re thinking about, same work that’s coming in, but how do we ‘rightsource’ it beyond just inside counsel, outside counsel, and the tech that we currently have? But what can AI now do, and how should we be using it to do things better, faster?

[00:03:12] Irene Liu: And a lot of companies, especially in-house teams, as they’re thinking about ‘rightsourcing’, they’re often thinking about whether they should just buy an AI tool that’ll solve the issue that they have at hand. Is that the right way to think about it, or do you think the bottleneck is really with talent or the buying element?

[00:03:31] Mary O’Carroll: Yeah. I think the technology has all of a sudden become the easy part.

[00:03:34] Irene Liu: Yeah.

[00:03:35] Mary O’Carroll: You know, choosing a tool, you can’t really go wrong. There’s a lot of great tools out there. You know, we used to think build versus buy, and I think everyone realizes there’s so many tools you should definitely go the buy route now.

[00:03:46] But buying the tool and even implementing the tool is not the destination. That’s not the end game. That’s really actually just the beginning. And I think what I’ve seen, if you look across all these different organizations, you know, I see them in like three different buckets of where people are on their adoption.

[00:04:03] There, there are those who are sort of, I call them paralyzed. You know, they’re looking across, totally overwhelmed by all the options and unable to make a choice, and so they have committees and they’re thinking, and they’re kind of looking around at big companies and law firms and saying, “Maybe one of them will figure out, and once everything sort of like settles down, I’ll, I’ll dip my toe in.” but we all know that nothing’s gonna settle down and this pace of change is just gonna be what we are used to living in.

[00:04:30] I think the biggest bucket is the next one where we see most companies are, I call them accelerators. They are really excited about the potential of what AI can do for their organizations in terms of speeding things up, and they are quick to buy a tool and get it out there and, and roll it out to all their people, which is great, and you will find, that the attorneys who are using it are getting good value. They’re seeing how it makes their work faster. But I think that’s the key term is like people are looking at the work that is currently being done and saying, “How can this tool make it go faster?”

[00:05:04] Where we really want people to be is in this third bucket, which, you know, we call sort of reimagining the work. And that is where you have the technology and the AI, and you’re not just making it faster, but taking this big step back and saying, “Okay, what are the inputs, and what is the desired outcome or output from the end of this entire process? And let’s actually take everything that happens from point A to point B and throw it out the window- and say, how can we totally rethink, reimagine, how do we deliver this outcome or this output given what we can do today with all these tools?”

[00:05:41] And the folks that are really thinking sort of outside that box and not just making things faster, but different, that is where you’re gonna have the biggest value and unlock from leveraging AI.

[00:05:51] So I guess it’s a long way of saying, like, buying the tool doesn’t really get you there. You really need, the people to be thinking creatively. And so, yes, I think the talent is the bottleneck. We have the tools, and you can kind of give everyone a chatbot.

[00:06:06] Irene Liu: Yeah.

[00:06:06] Mary O’Carroll: But where are the talented people who are gonna sit down with your organization and really think through how all the work is currently done and really reimagine that for tomorrow.

[00:06:18] Irene Liu: And that part about reimagining work, it probably is not new. I mean, when you were at Google thinking about this, I’m sure while legal operations at that point was probably more of a nascent role, it probably required you also then to reimagine work and workflows, to reimagine how you can make the legal team even more efficient and productive-

[00:06:41] Mary O’Carroll: Yeah

[00:06:41] Irene Liu: …and valuable.

[00:06:42] Mary O’Carroll: 100%. I mean, in the very, very early days of the profession, there was a lot of blocking and tackling. There was a lot of stuff that you just had to put in place to keep the trains running on time.

[00:06:52] Irene Liu: Yeah.

[00:06:52] Mary O’Carroll: Never mind trying to actually improve the process. As departments got more mature with their foundational, you know, sort of blocking and tackling, you then got to say, “Hey, there’s this piece of work or something that’s taking up a ton of time for my organization. It’s not a really great value add, and I’d love to free up people’s time to work on something else.”

[00:07:12] So when I was at Google, we created a team. We didn’t have a great name for them. We, you know, were looking for, like, business process optimization type folks. We ended up hiring a lot of people who had that background from consulting firms, and we kind of threw them into the mix and said, “Sit side by side with our attorneys. Understand how all the work is done today. You know, is it standardized? Do 10 people doing the same work do it 10 different ways? You know and do that entire process map of the current state, and then exactly as we’re saying, rethink, re-engineer, reimagine what that could look like with different resources, moving things around, disaggregating the work or using technology. And maybe some of that work doesn’t need to be done at all.”

[00:07:57] And, and it’s not you know, we got a lot of requests as our team got better from our lawyers, and they would say, “Hey, I wanna sit down with your team. I need this solution.” and what we would always say is, “Let’s not start with the solution. Like, maybe that’s the right answer, but why are you asking?” “Why do you need this solution? Why do you need this report or tool? And what is the problem you’re trying to solve?” And as you start to, like, unpack what, what’s the issue at hand, what’s the root cause, there might just be a different way to think about it altogether.

[00:08:27] So that was a team we had, yeah. And we just didn’t know until, you know, now we’re starting to call these folks legal engineers, and there’s a lot of people who are saying, “Oh, I’m so glad there’s a name for this. I’ve been doing this job for a long time,” and it’s very, a lot of parallels to the early days of legal operations where-

[00:08:44] Irene Liu: Yeah

[00:08:44] Mary O’Carroll: …we defined legal ops and all these people came out of the woodwork saying, “I’ve been doing this for a decade. I didn’t know.”

[00:08:49] Irene Liu: And so, was it called, was your team then called legal operations or those consultants? What was the name of that team?

[00:08:55] Mary O’Carroll: Yeah. So, we, legal operations to me was like the umbrella for everything that was, you know, sort of the business of law side of our organization, not the practice of law.

[00:09:06] So, we had people within legal operations that were focused on outside counsel and financial management.

[00:09:12] Irene Liu: Sure.

[00:09:12] Mary O’Carroll: We had people working on systems and tools. We had people working on knowledge management. And this, and then we had, like, business intelligence. And then this team, we called them, like I said, kind of like business process improvement.

[00:09:27] There was a team at Google that was known outside of legal for internal consulting, and they were called just Biz Ops, very simply. And so, we called them Legal Biz Ops because at least within Google, people knew what that meant. But yeah, we didn’t really have we, a good name for that group.

[00:09:42] Irene Liu: Yeah. And it’s so interesting because you just mentioned the term legal operations, and then also legal engineering, and this new term legal engineering. But it’s actually not new because it started at Ironclad, the company where you were at. And it’s interesting because I remember that role, emerging because I was a general counsel at another tech company, and we’re looking at using Ironclad.

[00:10:03] And we, I remember encountering a legal engineer, and it actually was a paralegal with some contract experience from another-

[00:10:10] Mary O’Carroll: Yes

[00:10:10] Irene Liu: … tech company.

[00:10:11] Mary O’Carroll: Yes.

[00:10:11] Irene Liu: And so, it wasn’t really highly technical. It wasn’t a technical legal engineering role. And so, can you tell me about what that legal engineer role was when you, when you started at Ironclad, and how it’s being used now, and how it’s evolved?

[00:10:25] Mary O’Carroll: Yeah. You’re absolutely right, and I had the same sort of reaction. So, Ironclad is a contracts management or lifecycle management, company with AI. And when they first started, they created this, team of people called them legal engineers. And I think like you, I sort of expected that, like, that was the technical team. They’re the engineers. They’re the software engineers. And I thought they were the people actually building the technology.

[00:10:53] But no, it actually wasn’t that at all. It was the people who were gonna sit down with the end users or legal team, and that could be attorneys or contracts managers or procurement people, and kind of understand the work, the challenges they had, and then leveraging the Ironclad platform to think, “How can we apply this to your work and make your work better?”

[00:11:16] And so that’s what it was. And when I kind of, you know, it clicked, and I understood what the term meant, I started realizing, okay, it is engineer as in process re-engineering- or designing the systems and re-engineering them, not software engineering. That was a bit of an ah-ha moment for me. It sounds like you had a similar, reaction, and I think a lot of people out there probably think that a legal engineer, as we use that term, is someone who’s highly technical and can be coding all day, but I don’t think that’s what it is.

[00:11:45] Irene Liu: Although now it’s evolved to be more of a technical, where people are expecting a legal engineer to have technical prowess.

[00:11:53] Mary O’Carroll: Yes, and, and I mean, everything is sort of moving towards technical ability.

[00:11:57] Irene Liu: Yeah.

[00:11:57] Mary O’Carroll: You know, even lawyers, are being asked to, you know, learn how to be comfortable with AI, whatever that means, and that doesn’t mean you need to know how to code, but you certainly need to have a much higher level of comfort with, technology and systems and tools and AI and prompting in a way that I don’t think was necessarily demanded of everyone, you know, just a few years ago.

[00:12:20] Irene Liu: Yeah. So, do you imagine that legal operations professionals will need to evolve into legal engineers, or do in-house teams and law firms need to hire separately for legal engineers? And if so, where can they even find these-

[00:12:35] Mary O’Carroll: Yeah

[00:12:35] Irene Liu: … talents?

[00:12:36] Mary O’Carroll: It’s a great question. I do think that it, you know, if you think about where does this sit in my organization, it is probably a subset of the umbrella term of legal operations, and maybe that’s not the right term for us anymore.

[00:12:48] We really, like a lot of us, need to be thinking about the future of how do we, what do we call ourselves, but there absolutely is, I think, a sub, subsegment. So, you know, you still need your outside counsel people. You still need your knowledge management. But there is a place for this now, and I think it’s becoming more and more important.

[00:13:05] So, you know, when we started, having this role at Google I, I think it was like a lux- we called it a luxury team. Like, I don’t think you looked around at other legal departments and had this, you know, group of people, but now it’s become much more important. A, because the technology is changing so fast, it’s not like you can just make a decision to buy a tool and roll it out and and move on. Someone has to stay on top of the evolving technology. What else is available? What are the new features? What are the capabilities? And then keep it refreshed and up to date, and new integrations, and, you know-

[00:13:39] Irene Liu: Yeah

[00:13:39] Mary O’Carroll: … all the things that, are ongoing. And it, now that you can kind of get one workflow up and running pretty quickly, you know, you can turn around and find yourself overwhelmed with a lot of things that you’ve deployed. And so how are we going to maintain those over time? So not, it’s not just kind of designing them and building them. I think there’s sort of a maintenance component. The other term that gets thrown around a lot these days is the forward deployed engineer.

[00:14:06] I think it’s, from the technology or AI companies, they’re seeing a lot of value in putting, you know, one of their FDEs with their clients to help them say, “Hey, this tool is so powerful. It can do a lot of things. What can it do for you? Let’s get under the hood with your organization.”

[00:14:25] So, it’s a lot of that same idea, but we’re seeing this sort of, you know, if we wanna call them legal engineer, this role is popping up on the in-house side, on the law firm side, and on the vendor side because everybody needs it. And as you said, where do you find them? Well, where, wherever you can, you know?

[00:14:45] Irene Liu: Yeah.

[00:14:45] Mary O’Carroll: If you can hire them, great. If you can use your vendor as a resource, great. If you have an outsourced company like the one that we that, that we run, and you can bring those people in, that’s great too. But everybody is looking for them, and there isn’t, you know, just massive pool of people who are sitting around. So, I think it’s a, a big gap, going back to what you said. In order for us as an industry to be successful, the biggest bottleneck to me right now is this big gap in the market in terms of talent.

[00:15:14] Irene Liu: Yeah. And another area that I was thinking where you could get the talent is maybe solutions engineers within companies, because oftentimes these solutions engineers are the ones that are interfacing with the customer, trying to diagnose what their issues are-

[00:15:27] Mary O’Carroll: Yes

[00:15:27] Irene Liu: …and trying to help them solution how that product can work best for them. And so, it’s often a customer client-facing position, so it may be a legal solutions engineer that oftentimes these companies might need.

[00:15:40] Mary O’Carroll: Yeah.

[00:15:41] Irene Liu: But probably deeper than that, because not only do you need a solution of someone that solutions it, but like you said, maintenance is such a key-

[00:15:48] Mary O’Carroll: Yeah

[00:15:48] Irene Liu: …element of all of these products. It’s not just buy, solution it-

[00:15:52] Mary O’Carroll: Yeah

[00:15:53] Irene Liu: …but you need to maintain it in order to get the, the best value.

[00:15:56] Mary O’Carroll: Yes. And maintain makes it feel like, you know, you’re just keeping the trains running, but there’s like an, you could constantly upgrade and optimizing it and and making it better.

[00:16:04] But I think, yeah, you and I talked about this term kind of as we were prepping for this, and I like that legal solutions engineer, which maybe if you’re new to this world clicks a little bit better of thinking like this is someone who’s re-engineering a solution rather than like a software engineer. So, yeah. Maybe it’ll work.

[00:16:24] Irene Liu: I think it’s like what you were saying, it’s reimagining work, right? So solutioning, reimagining-

[00:16:28] Mary O’Carroll: Yeah

[00:16:28] Irene Liu: … in many ways. And a lot of these legal departments, even law firms, whether it’s an in-house or law firms, like you said, they need to break down their processes, figure out what their, their end goal is, and with each of them, they have to reimagine how they, how can they inject technology and AI to really vastly improve and add value to their services.

[00:16:48] And so it will require a lot of different types of solutions engineering.

[00:16:53] Mary O’Carroll: Yes.

[00:16:53] Irene Liu: So, it’ll be interesting to see if we’ll see more legal solutions engineers in the future.

[00:17:00] Mary O’Carroll: Yeah. When, I mean, you’re at a law school. I keep wondering, as we, you know, many of us with growing children are like, “What are our kids gonna do when they grow up? What are these roles that are gonna emerge?” We always say, you know, the, the roles that we have today aren’t gonna be the same jobs that the future generations are gonna have.

[00:17:16] And you know, I think we have to look to law schools and business schools and engineering schools to start to rethink what are these gaps in the market? What are the skillsets and are we kind of preparing people for the future in these areas where, you know, I don’t think we are right now, and that’s not a fault of the schools. We’ve never needed this before- but it’s emerging quickly and becoming, I think a big hole in the market that, needs to be filled quickly.

[00:17:43] Irene Liu: Yeah. And I think, at Stanford Law might be very distinct given the fact that we’re right, located right in Silicon Valley-

[00:17:50] Mary O’Carroll: Yes

[00:17:50] Irene Liu: … and we have a very, curious population of students with technology, and they love to tinker with it. And we have 1Ls and 2Ls that are building companies.

[00:17:59] Mary O’Carroll: Oh, I’m sure.

[00:18:00] Irene Liu: We have students that are also going to legal tech companies for internships instead of the traditional law firms. They’re also going to AI native law firms as, for full-time roles as well, too. And so, we have this interesting emerging group as well that are very proficient in technology and are interested and curious as well, too. And so, it’ll be interesting to see how they land and how they fit into this, reimagining of work and the types of work that they take on afterwards as well, too.

[00:18:28] Mary O’Carroll: Yeah. ‘

[00:18:29] Irene Liu: Cause it’s no longer just the law firm associate role as their, as one of the main backstops.

[00:18:34] Mary O’Carroll: Yeah.

[00:18:35] Irene Liu: So.

[00:18:35] Mary O’Carroll: Well, and as companies become more AI native, and law firms, as you mentioned, are now AI native, there’s gonna be, you know, a huge population of folks coming out of school that are going right into those companies. And you probably don’t have to tell them to reimagine, ’cause that’s literally what they’re doing day one.

[00:18:51] Irene Liu: Yeah.

[00:18:51] Mary O’Carroll: They, they start from scratch and say like, “How do we do this better as an overall organization?”

[00:18:56] Irene Liu: Yeah.

[00:18:56] Mary O’Carroll: So, they’re at a bit of an advantage. I think the challenge that we’re gonna see or we are already seeing is the large organizations, and how do we change, you know, when you’ve got companies with, you know, thousands of people and everything is functioning very, very well for the current state, and how do we get everyone to change their mindset and their way of working when you could very easily argue it’s not broken. It’s actually working very well, these companies, you know, and firms are very successful. But if you were to You know, start from scratch tomorrow, probably you would not build it the same way. And so how quickly and how, you know, how hard is it to get there? And very interesting to see, you know, billion-dollar revenue companies now that are a handful of people.

[00:19:40] Irene Liu: Yeah, I mean, it’s that human transformation element that I think is the hardest piece for any of this, AI implementation or integrating it into companies as well too. Like you said, there is, especially at law firms, when you think about law firms, there’s a way of doing things. And you are COO at a law firm too, so I’m sure you have seen it in person, but there are ones that embrace AI more so than others.

[00:20:04] And so to be able to transform the entire, partnership and associates in the law firm, or even companies, like you said, very large companies, to really embrace this technology, in lockstep together is a hard, difficult transition.

[00:20:21] Mary O’Carroll: Yeah, I mean, especially if you’re busy all day. Yeah. You’re already doing the work and barely have time to come up for air, and the-

[00:20:27] Irene Liu: That’s right

[00:20:27] Mary O’Carroll: …volume doesn’t stop, and so you have your heads down. And, you know, when I was just very recently able to take a little bit of time off between jobs, I could take, like, two weeks and just deep dive and research and learn and watch videos and hack some stuff together and teach myself, but that was, like, multiple full days of having nothing else to do, and so that was a big luxury for me.

[00:20:49] And as I look around at people who are currently in companies, it’s not that people aren’t interested, it’s not that they’re, you know, scared of the technology. They actually all want to learn this, but it’s, its time, and, like, who has that luxury of time when you’re, you know, in its deep?

[00:21:05] Irene Liu: It’s so true, and there’s so much, pressures for efficiency as well, too, and to go, more AI native for every type of company that there is, but you don’t have the time,

[00:21:17] Mary O’Carroll: You don’t have the time

[00:21:18] Irene Liu: …to even understand and reimagine.

[00:21:21] Mary O’Carroll: Yes. And I mean, I’ve been talking to so many folks who are in leadership roles in legal operations right now, and, you know, they’re working very closely with their GCs or CLOs. And you were a former GC. I mean, you can sort of imagine how much pressure there is now. It’s actually, I’m hearing them talk, I’m, like, stressed out for them. There’s so much urgency and pressure to, like, get moving on all this stuff faster.

[00:21:46] And what we all have to realize is you don’t get to push pause on, you know, the old way of doing things. Like, people are still totally overwhelmed by the amount of work that is coming in regularly, and to say, “We all have to stop, rethink, retool, roll stuff out, rethink how you need to do your job,” that’s like adding a whole other job to these people’s jobs, which is again why I think sometimes you need outside people or someone from the vendor side to come in and help because, um, there’s just, like, the expectations for how much it could be better is so high, and the sense of urgency is so high also.

[00:22:20] So, people want results and impact fast. But it’s not that easy. It’s not fast.

[00:22:26] Irene Liu: Yeah.

[00:22:26] Mary O’Carroll: And there’s gonna be moments that, you know, your, productivity actually goes down quite a bit as people are trying to figure it out, and you do have to kind of press pause on some of the work or bring someone else who can focus on this.

[00:22:38] Irene Liu: Yeah, definitely. And when you think about the pressure to do more with AI, it’s with the same number of people.

[00:22:45] Mary O’Carroll: Yeah, yeah. Or, or they wanna be less, right?

[00:22:48] Irene Liu: Or less, yeah.

[00:22:48] Mary O’Carroll: How quickly can it be less?

[00:22:50] Irene Liu: Yes.

[00:22:50] Mary O’Carroll: And that’s… It’s not that simple.

[00:22:52] Irene Liu: Yeah, because you have to carve out time to actually map out those processes, reimagine, and figure out what can be leveraged with AI to bring that productivity value.

[00:23:04] Mary O’Carroll: Yes.

[00:23:04] Irene Liu: So, if someone’s listening and they’re in-house, they’re, in legal operations, or they’re in-house as a senior lawyer or even a junior lawyer, what are the things that as a baseline that they should tackle that you think is totally doable?

[00:23:22] Mary O’Carroll: I, well, it really depends on your organization. So, there’s gonna be low-hanging fruit, and then there’s gonna be stuff that is a bit harder but is worth doing. So, I like to get people thinking about things in a, in a bunch of different ways ’cause- A, just telling people, “You need to change this”, is, is hard. And you don’t necessarily understand the art of the possible when you’re not, you know, in deep with these tools.

[00:23:45] So, I encourage people to go around asking, like everyone from your top lawyer down to the most junior person on the team, “What is the work that energizes you, that you get to do all day, and that you wish you could spend more time doing?” and the converse is, you know, what is the stuff that is draining of your energy or you don’t feel like adds a lot of impact, but you have to get it done anyway? And sometimes, it helps to start with that non-threatening work that people want off their plate.

[00:24:13] Maybe that’s something we can think about automating or getting rid of or ‘rightsourcing’ in a different way, and that starts to just kind of build the trust and the understanding that you’re, you know, this isn’t coming after me and my job. And that is like a real, a real fear, and I think it’s-

[00:24:27] Irene Liu: Yeah

[00:24:27] Mary O’Carroll: … I think it’s totally valid. But when people start to see, no, this is about freeing you up because there’s tons more work, like trust me, you know, we optimize hours and hours of your day away, there will be tons more work to keep you busy, but it’s the stuff that you wanna be working on.

[00:24:41] Another way to think about it is, okay, let’s look at your whole organization. Where is most of the time spent? You know, what is the bottleneck in the org, where is, where is there too much risk? Where is there too much time spent? Where is there too much money spent? Where is there too much people, being asked to do something?

[00:25:00] And you might start asking those questions and start to surface areas where there’s low value work or things that just are taking too much time or energy, and is there a way to make that better?

[00:25:11] So, it’s like starting to ask questions other than, do you have any use cases for me today? Is getting them to think a little bit and just listening. I think that’s one of the biggest jobs of a legal ops professional or a legal engineer, is just being really good at listening, asking questions, and kind of drawing out things to figure out what are the root causes, you know, where are the problems, where are the bottlenecks, and then identifying like, are there ways that we can make that better?

[00:25:35] Irene Liu: And solutioning it. Like a solution engine

[00:25:37] Mary O’Carroll: And solutioning it, yeah. And there’s, you know, there’s different, you, you come up with a list of things-

[00:25:41] Irene Liu: Yeah

[00:25:41] Mary O’Carroll: … and then you can start thinking about, I like to look at a matrix of level of effort versus-

[00:25:47] Irene Liu: Yeah

[00:25:47] Mary O’Carroll: … level of impact. And if there are things that are big impact, low level of effort, I mean, those are the obvious places to start.

[00:25:55] You can also think about, level of effort versus risk, or, you know, how much is a human involved-

[00:26:01] Irene Liu: Yeah

[00:26:01] Mary O’Carroll: … needs to be involved in this kind of work, and that might not, you know, if you need a lot of humans, if there’s a lot of risk in automating it, maybe not the best place to start. But, like, those are the kind of things that you should think about in terms of after you get your big list of ideas is prioritizing places to start.

[00:26:17] Irene Liu: Definitely. And one of the things that a lot of general counsels are looking at that’s, like, the biggest low-hanging fruit is NDAs, right?

[00:26:23] Mary O’Carroll: Yes.

[00:26:23] Irene Liu: So, I think that’s a good starting point. and then just starting from there and then reimagining, because there will be a lot of different types of grunt work like that, that has such low risk for the most part, that drains-

[00:26:35] Mary O’Carroll: Yes

[00:26:35] Irene Liu: …the efforts of commercial lawyers and their time, so.

[00:26:38] Mary O’Carroll: So that’s absolutely right, and but it’s not as necessarily easy as we think, depending on your organization. I mean, there are folks that are, like, so ready to spend money and buy AI redlining tools and have it all sort of, you know, have the machines do the negotiations and the redlining.

[00:26:54] But there’s a whole people alignment element first. I mean, does your organization have a position that, you know, standardized positions that it wants to take or language that it, it’s comfortable with? And if you haven’t aligned internally, again, like if you have 10 lawyers negotiating-

[00:27:09] Irene Liu: Yeah

[00:27:09] Mary O’Carroll: … NDAs and they all do it differently, and now you wanna use AI, you’re not gonna be happy with the results.

[00:27:15] Irene Liu: Definitely.

[00:27:15] Mary O’Carroll: If these 10 lawyers can get in a room and say, “Here is our standard, you know, language, position, templates, how we like to see things done, here’s, you know, what our risk tolerance is,” then you certainly, you can start to solution it and build something out that can be very impactful. But we oftentimes forget that there is some lift that is the non-technical part that is-

[00:27:37] Irene Liu: Yeah

[00:27:37] Mary O’Carroll: …actually the hardest. Like getting… I mean, honestly, getting 10 people to agree on language is very, very difficult.

[00:27:43] Irene Liu: Yeah. And, and the the playbook is basically-

[00:27:46] Mary O’Carroll: Exactly

[00:27:46] Irene Liu: …one of the things that you need to build, and that is a hard one to align across the company-

[00:27:51] Mary O’Carroll: Yes

[00:27:51] Irene Liu: … on the risk level, not only on NDAs, but let’s say contracts as well or whatever other process, just aligning and finding the data repository to-

[00:28:00] Mary O’Carroll: Yes

[00:28:00] Irene Liu be able to train the models. That is also additional time that you need to carve out-

[00:28:06] Mary O’Carroll: 100% …

[00:28:06] Irene Liu: as well too. Yes. And that takes a lot of effort in itself too.

[00:28:09] Mary O’Carroll: Yeah. That’s right.

[00:28:10] Irene Liu: Yeah. So, if we’re looking at also moving outside of in-house to law firms, law firms have been investing in AI in very different ways.

[00:28:18] We just heard recently that Kirkland invested, you know, $500 million into their AI solution as well too, and so that’s quite a mark. And a lot of other law firms are trying to also reimagine how they’re going to leverage AI. And so, when you think about law firms, obviously their incentives and their business model doesn’t always align with, the efficiency and the cost that clients wanna see.

[00:28:44] But, what are the things that they should be thinking about as they’re investing in AI? Should they be investing more in the talent side as well too, or is it the technology, or is it both?

[00:28:58] Mary O’Carroll: I think it’s exactly what we spoke about earlier. It’s the same thing where the technology buying is the easy part.

[00:29:04] Irene Liu: Yeah.

[00:29:05] Mary O’Carroll: I think kudos to Kirkland for, you know, putting a big stake in the ground, coming up with a strategy that’s a little bit different than everyone else’s. But they’ve gotten a lot of marketing and a lot of talk-

[00:29:15] Irene Liu: Yes

[00:29:15] Mary O’Carroll: … and they haven’t done anything yet. You know? It’s like, so we will see.

[00:29:17] Irene Liu: Yeah.

[00:29:18] Mary O’Carroll: ‘Cause saying that you’re gonna spend the money is, is really the easy part. Spending the money’s the, the easy part. But where is it gonna actually create value for the clients? I think that is a big, big question. You talked to as many GCs as I do, I hear just constantly daily frustration of-

[00:29:34] Irene Liu: Yes

[00:29:34] Mary O’Carroll: … yeah, I see all the announcements. I know my firms are using it. They say adoption is up. I still don’t see any impact to my work, to my bills. So that’s the challenge. It will take time, as we talked about. These firms have been, in very successful business models for a long time, and it is not easy, and they’re all busy.

[00:29:52] And so, where I think, you know, if I were to invest, absolutely, it’s the talent, it’s the sitting down to understand the work, and I think it’s actually much harder at a firm than it is in-house. I mean, in-house, even if you are the most complicated legal department at a company like, like a Google that has within it many, many geographies, products, as well as practice areas Almost, almost like a small law firm, right?

[00:30:18] But most legal departments do some, do one piece of work a lot, you know? They do a lot of contracts.

[00:30:24] Irene Liu: Yeah.

[00:30:24] Mary O’Carroll: They might do some IP. They might have some litigation, but they all do a ton of contracts, and so that’s, like, the place to start, obviously, for most in-house teams. You go to a law firm, and you go to one team, and you’re like, “Maybe we’ll do some stuff for the M&A team,” and after talking to, you know, five associates and two partners, you’re like, “Wow, it’s actually very complicated, very nuanced. They have every type of situation that happens under the sun.” That’s why you call a law firm. Your company may not have a ton of M&A every year, but that’s all these people do day in and day out. And then you go through every specialization and every practice area, you know, throughout a firm, and you start to realize there’s so much area for, opportunity and to kind of reimagine where do you even start?

[00:31:07] You know, do you start with one team, one office, one geography? Do you throw a bunch of people at every, you know, area? I, and I think that’s what everyone’s kind of struggling with right now is like, who, who is gonna get under the hood and change that? And then what does it mean if we start to do something in one hour versus eight hours? Okay. There’s ripple effects, you know?

[00:31:28] Irene Liu: Yeah.

[00:31:28] Mary O’Carroll: It’s, it’s all connected. That’s gonna affect the pricing. Okay, if it affects the pricing, how does it infect, affect the performance evaluation and incentive models for compensation? How does that affect how we hire? What should our structure of the firm look like?

[00:31:43] Irene Liu: Yeah.

[00:31:43] Mary O’Carroll: Do we need more of these different types of people? And all of a sudden, you have all these kinds of really big existential questions. So, I, you know, law firms are are struggling. I think it’s very hard. They have great people that they’ve hired. They’re making the right investments, I think, in the technology and even in the talent. So, it’s just gonna take time, very hard to move a big ship.

[00:32:04] Irene Liu: Yeah, one of the things you just talked about is how they are hiring a lot. So, you are starting to see these new roles also within law firms where they are, like, AI chairs-

[00:32:13] Mary O’Carroll: Yes

[00:32:14] Irene Liu: … essentially. So, do you find, do you think they will also develop large teams that basically service solution engineers each of the different groups, essentially? Is that how the model will evolve, do you think?

[00:32:28] Mary O’Carroll: I think that’s what I believe is happening, yeah, at firms. You know, the- whether they have an innovation office or an AI chief, they are buying tools, they are building tools, they’re developing, systems in-house, and that’s the smart thing to do.

[00:32:46] Irene Liu: So, let’s do a quick lightning round because there– we’ve talked about in-house, we’ve talked about law firms, but, you know, I just wanna provide practical takeaways for folks at the end. So, if you were to invest in each of these organizations, where would you invest first to help them thrive in the age of AI? So, for an, a legal in-house department, where would you invest first?

[00:33:10] Mary O’Carroll: So, for in-house, so for all of these, there’s sort of like three skill sets that you need. It’s the business process re-engineering or optimization, it is the technical expertise in, you know, translating all that into a technology solution, and then there’s the deep legal, substantive kind of understanding of the work that gets done. So, you have those three things. It’s, very rare to find someone who has expertise in all three, so you kind of have to optimize for who you are.

[00:33:39] So, your first question was if you’re an in-house team, again, I don’t think you need as much deep legal specialization ’cause you don’t have that many practice areas. And so, I would really start with focusing on people who have the business process optimization. As I did, I hired a bunch of sort of ex-consultants or business process, people. And you can teach them. All they have to do is sit down with the lawyers and have… you know, spend some time understanding how the work is done. We’re not asking them to do the work. And hopefully they have some technical expertise as well.

[00:34:10] Irene Liu: What about for a law firm? Where would you optimize for them to thrive in this age of AI?

[00:34:14] Mary O’Carroll: Yeah. So, law firms are a bit different in that I do think you really deeply need to understand the life of the law firm attorney and the work that they’re doing. And so, hiring people who are, who have practiced law at a law firm and understand, that work, but perhaps have some, technical interest and expertise. And we’re seeing a lot more of those people who are shifting out of, let’s say, being an associate into these legal engineering roles. So, those are the people, you know, I, I bet they exist within your firm. Look around and say, “Who, who wants to shift into this new role?” And you might find a lot of them.

[00:34:49] Irene Liu: I think that’s such an important point, that it’s talent shifting because there isn’t a deep pool of these kinds of talents.

[00:34:55] Mary O’Carroll: Yes, you have to get them from somewhere.

[00:34:56] Irene Liu: Yeah, exactly. So, you may need to just shift some of the people that are proficient already in there, areas of expertise-

[00:35:04] Mary O’Carroll: Yes

[00:35:04] Irene Liu: that are highly technical-

[00:35:05] Mary O’Carroll: That’s right …

[00:35:05] Irene Liu: and then shift them to this role, which is interesting. What about for an AI native law firm? Where would they need to invest to thrive?

[00:35:12] Mary O’Carroll: So, I, I think it’s probably not different than a regular law firm, although, you know, now AI native law firms have two different sides.

[00:35:20] Irene Liu: Yes.

[00:35:20] Mary O’Carroll: They have the practice of law, and they have the side that’s actually, many of them are building their own, models, tools. And so, for that side of the house, you definitely want people who can be technical builders and to be working closely with the lawyers.

[00:35:36] Irene Liu: What about for an AI legal tech company? We’re seeing such a surge of these AI legal tech companies that’s, never been seen before. I’ve never seen legal tech be so hot.

[00:35:46] Mary O’Carroll: Oh my gosh. It… Yeah. Isn’t it bizarre?

[00:35:48] Irene Liu: It is so interesting.

[00:35:49] Mary O’Carroll: It is the thing now.

[00:35:50] Irene Liu: Yeah. No one wanted to invest in it before, and now everyone wants to invest in it.

[00:35:54] Mary O’Carroll: And everybody wants to start up in it and work in one of them. I think the same as an AI, that, that one half of the AI native law firm where you want folks who can be builders, but who are really good listeners. I think the solutions engineer sort of people who have that background, and they do. They’ve existed at SaaS companies for years. That’s the kind of, person you want more of.

[00:36:16] Irene Liu: Yeah. Well, Mary, thank you so much for this time today. Hopefully we’ll get a deeper talent of those. Well, they already exist. They just need to shift-

[00:36:24] Mary O’Carroll: That’s right …

[00:36:25] Irene Liu: at times. So, thank you again.

[00:36:27] Mary O’Carroll: Yeah, such a pleasure. Thanks, Irene.

[00:36:33] Irene Liu: Thanks again to Mary O’Carroll for the discussion on the legal engineering talents needed in law firms and law departments today. If you enjoyed this episode, please follow the podcast and share it with a friend. Until next time, stay curious and keep learning