The Tipping Point: Predictive Coding
How does innovation make an impact? Sometimes it lands with a bang, like the iPhone. Other times, it builds gradually toward a greater transformation. At some point, greater speed doesn’t just make a technology a bit faster. More convenience doesn’t just make it a bit easier. Something new becomes possible, regional practices go global and niche technologies enter the mainstream.
Predictive coding is a prime example. Based on machine learning, predictive coding systems can identify key documents in a fraction of the time it would take human reviewers to review. Predictive coding in ediscovery software promised massive savings of time and money. But early systems had a reputation for being expensive, difficult to use, and suited for only the largest cases under the tightest deadlines.
Is that view changing? Has predictive coding reached a tipping point? We asked Benjamin Kennedy from our Australian partner, NuLegal, for the view from the other side of the world.
Tell me a little about NuLegal. Who do you work with, and what sort of cases do you work on?
We’ve been around for 7 years now. Our client base is mostly law firms. We have a few on the corporate side, but law firms are our focus. We mostly support litigation and regulatory requests, with some investigations. Some of our clients have litigation support departments, but many don’t. They like NuLegal because we fill that role for them.
We brought Everlaw to Australia about 18 months ago. As soon as we saw it, we realized we could approach predictive coding differently than we had in the past. Everlaw’s predictive coding is always running in the background and learning, it is referred to as continuous active learning (CAL). It doesn’t cost anything to look at the results. That opened up the possibility of using predictive coding in every matter.
What’s the attitude in Australia toward predictive coding?
It’s a hot topic, and there’s a lot of interest in how it drives value. There are two recent cases in Australia, where judges have accepted the use of predictive coding. The Victorian Supreme Court also released an updated practice note this year advising it may direct parties to use the technology. It put the industry on notice that this is something lawyers need to understand.
We’ve been pushing this technology for five years. Before Everlaw, we used an on-premises vendor with a more traditional predictive coding system. We had a tough time convincing clients to use it because their SMEs always had to take time away from the project to train the system. Then, in a lot of cases, new documents would emerge, the understanding of the issue changed and they had to revisit the training process again.The whole model was a little broken. They had to pay up front and go through all these machinations to see any value, without being sure they’d get any.
With Everlaw, predictive coding is always running in the background. Clients are already working with the documents. They can quickly see if Everlaw is picking up what’s relevant and what’s not. They don’t have to be experts. I think that’s enticing many of them to take a closer look.
It also helps that firms have an answer now when corporations ask them, “Where is the benefit”? A firm we recently worked with had one of their corporate clients ask them the end dollar value of predictive coding as opposed to other review process. Without hesitation, the lawyer said about $200K (AUS). When a lawyer can be that confident, there’s a good chance their clients will follow.
Do Australian law firms feel threatened at all by predictive coding?
Mostly they don’t. They’re likely seeing their revenue attacked on several fronts, and I don’t think they expect to make it up on document review. Their clients are increasingly putting pressure on them to reduce review costs. One firm told us recently about quoting $150K for document review, and their client said no, it would have to be $50K. That kind of response challenges firms to say, “How are we going to do that?” The option they chose was to use predictive coding combined with other search techniques to get closer to the $50K mark. A lot of firms are feeling that sort of pressure, and responding similarly.
Separate but related, I think law firms are feeling increased pressure to show innovation. If a customer is looking at two firms—one innovative, another traditional, it’s probably going to give more consideration to the one trying something new—especially if it’s innovating to get results sooner and keep costs in line.
How much does usability matter with predictive coding?
A lot. When you’re not using continuous active learning, like Everlaw does, there’s a whole overhead in getting TAR up and running—and that’s before you see any benefits. To a large degree, Everlaw has taken away that barrier. We’re now having conversations about predictive coding with firms that never would have entertained it in the past. Small suburban firms are getting their hands on matters with 200,000 docs that they need to review before filing a claim. They’re comforted to know they can leverage technology and not have to review every doc. With Everlaw, predictive coding is being adopted by folks who don’t litigate often and don’t have a review team to throw at the task. I think we’ll see a lot more of that in the future.
What has the feedback been from clients who’ve used other kinds of predictive coding?
Continuous active learning is pretty new. In Australia, most people’s experience has been with predictive coding models that require you to dedicate a subject matter expert to train the system for two weeks. Those who have that experience see the difference immediately. They see they don’t have to invest all that time, that there are models running in the background that can just give them, say, the top 1,000 docs with hot characteristics based on the review undertaken to-date. Looking at the results of a model that updates with review efforts aligns with a developing understanding of what is relevant and what isn’t.
How do you explain the benefits of predictive coding to those who haven’t used it before?
Prioritization is the place to start. Why wouldn’t you want to review the documents with known interesting characteristics before documents with known irrelevant characteristics? Then there’s the point on costs. I focus on total return of using Everlaw—that being using predictive coding with other methods to find and quickly review documents. During a review supported by CAL, you’re going to see a diminishing return on the number of newly identified relevant documents. That’s a good thing. It is a flag for when to stop reviewing—maybe when you’re finding 1 key doc in every 5,000. If you’re reviewing in a linear manner, you may find relevant docs all the way along the line. Reviewing with predictive coding encourages clients to draw a line in the sand. I don’t pitch the cost-benefit alone, but the savings in time and getting to the relevant documents sooner to make decisions sooner. That seems to resonate with many firms to take a closer look at the technology.
