The development of generative AI as an emerging technology has been almost unprecedented. The popular large language model (LLM) ChatGPT-3.5 gained over 1 million users in just five days after its release in 2022, and today boasts over 180 million users worldwide. It has served as the shining example of how advancements in generative AI can move at a blistering pace.
As this technology grows more pervasive in the legal industry, it holds major promise for the future practice of law. For example, many organizations are already using it to reduce document review times, summarize documents in bulk, analyze sentiments contained in documents, write first drafts of arguments, and more.
The key to ensuring generative AI is used to better the legal practice is for organizations and AI providers to build clear guidelines for how this technology should be integrated. This framework serves as an overview of Everlaw’s commitment to responsibly building generative AI products for the legal profession.
Overview of EverlawAI
Many legal technology platforms are coming out with their own generative AI systems in an effort to put this technology responsibly in attorneys’ hands, and Everlaw is at the front lines of this.
EverlawAI Assistant focuses on two main feature areas in the platform.
One area, EverlawAI Assistant Review Assistant, aims to assist users in document review. It can provide information such as:
Batch document summaries
Summaries by topic within a single document
Coding suggestions
Sentiment analysis
People and organization extraction
Dates and numbers extraction
Page and line references
The other area, EverlawAI Assistant Writing Assistant, lives in Storybuilder, and can provide information such as:
Draft summaries
Document references in drafts
Some EverlawAI Assistant tasks require users to specify a series of information, while others just take the click of a button. Depending on the task a user asks EverlawAI Assistant to perform, Everlaw prompts the LLM with task-specific instructions, questions, criteria, and relevant data. The level of detail that Everlaw uses is essential for the high stakes, high precision use cases that are relevant to legal professionals. This grounds the LLM in the facts at hand, rather than relying on embedded, possibly irrelevant, knowledge.
EverlawAI Assistant cites source documents in its results for users to reference when checking their work, so they’re able to ensure accuracy and verify outputs.
In developing EverlawAI Assistant, we leverage LLMs from a third-party service provider who is evaluated from technical, legal, privacy, and security perspectives. When a user inputs data into EverlawAI Assistant, their data is temporarily sent to the AI service provider, which is currently OpenAI, for processing through an SSL encrypted service.
OpenAI only uses the customers’ data to provide this service to Everlaw. They are contractually prohibited from using this data to train or fine-tune their models or service, and do not retain this data once the processing is completed.
Everlaw’s rate of innovation is one of our core differentiators, with new features and improvements released on a monthly cadence. While EverlawAI Assistant’s features and abilities are likely to grow and change rapidly, we are committed to keeping the concerns of our customers, including those about data privacy and security, at the forefront. Please view our product page and support documentation to stay on top of the latest EverlawAI Assistant updates.
Creating Ethical Generative AI
At the core of Everlaw’s approach to this technology is our generative AI principles: control, confidence, and transparency, privacy, and security.
Control
Users can opt in and opt out of using EverlawAI Assistant. We always let users know when they’re using generative AI, which is indicated with a clearly-recognizable icon, denoted here:
If they choose to use it, our generative AI features are designed to ensure that users can quickly and easily verify AI outputs.
Confidence
Generative AI models can provide immense value, but they can also make mistakes. We want to ensure that users can develop confidence in their results. To that end, where possible, we’re focused on creating generative AI that is accountable and trustworthy for the legal professionals who rely on it.
That includes tailoring generative AI features to specific use cases that we believe perform reliably, requiring the AI to cite specific, immediately verifiable passages of text from users’ evidence as justification for its responses, and ensuring that users have clear access to any evidence provided as context to the AI, so that they may perform a more comprehensive validation if they aren’t confident in its results.
Although risk and security are going to remain a concern for organizations in the generative AI era, as the technology becomes unavoidable, discovering ways to mitigate that risk will be imperative.
Transparency
Transparency is of the utmost importance when dealing with generative AI. Although much about these systems is still unknown, companies like Everlaw should be held to a high standard when it comes to demonstrating how customer data is used and where it goes once it’s input into the LLM. Organizations that have specific policies with respect to data use and data retention can mitigate certain data-related risks and provide clients with information on exactly how their data will be used.
When a generative AI task is requested, customer data is sent to our third-party LLM provider, which is currently OpenAI, for processing. The LLM, in response, returns the output, which is displayed in Everlaw. As part of our zero data retention agreement, the LLM provider does not retain customer data.
For example, when a user requests a deposition summary, Everlaw will send a prompt to our LLM provider. To promote accuracy and reduce the risk of hallucinations, that prompt will include contextually relevant information and customer data. OpenAI will rely on that information when generating its response, but will not retain any of it once the response is passed back to Everlaw.
For Everlaw’s part, we only keep customer data for as long as customers store it on our platform. Once a customer deletes that data off our platform, we do not retain it unless we’re required to do so by applicable law.
Protecting the privacy and security of customer data is central to Everlaw’s mission. Everlaw has implemented the appropriate safeguards to protect customer data against any accidental or unlawful destruction, loss, alteration, unauthorized disclosure of, or access to. In addition, we will only partner with AI service providers that have at least comparable safeguards.
Everlaw maintains an up-to-date list of all data subprocessors and provides notice in advance of adding any additional subprocessor, whether for generative AI tools or any other use case.
Similarly, Everlaw’s data collection and processing activities for generative AI features are designed to conform with all collection and processing commitments in customer agreements.
Build vs. Buy
The rapid development of generative AI technology and LLMs is fueled by a number of different companies wanting to get in on the action. The legal profession, Everlaw, and our users have been able to benefit from this competition and take advantage of the incredibly powerful tools being developed today.
At Everlaw, we believe the biggest benefits can be gained from leveraging leading third-party LLM tools. As such, Everlaw does not develop generative AI systems or LLMs independently. We also do not train or fine-tune LLMs developed by others.
Our technological evaluation helps ensure the best LLM is leveraged for a specific task, while our development approach allows for enough flexibility to shift between third-party service providers as needed to ensure the most complete experience for our users.
In addition to technological alignment, Everlaw’s vendor vetting process includes thorough evaluation by our technical, legal, and GRC teams. We continuously monitor our AI system’s performance, especially around accuracy, and ensure our third-party vendors are using best practices to keep customer data secure. Everlaw also monitors changes in law, regulations, and even standing orders from individual courts to ensure we’re building an AI product that customers can trust.
Our Commitment to Zero Data Retention
While many consumer generative AI tools, such as ChatGPT or Anthropic’s Claude, leverage user inputs to continue to train their models, usage of Everlaw’s generative AI technology ensures that data submitted and responses received are only used to serve the users’ experience within Everlaw – they do not stretch across customers or between customers, and customer data is not retained by LLM providers or used to train or fine-tune their models.
Everlaw’s current LLM provider, OpenAI, may access customer data in order to complete a user request, but OpenAI does not retain this data. OpenAI is contractually committed to zero data retention, and is prohibited from using Everlaw customer data to train their models or improve their services.
To the extent that Everlaw engages with additional LLM providers in the future, we anticipate continued commitment to a zero data retention approach.
In addition to our commitment to zero data retention when interacting with LLMs, Everlaw customers can delete case materials from Everlaw, including the inputs and outputs, at any time.
Protecting Against Data Injection and Exfiltration
Part of the confidence Everlaw offers users of EverlawAI Assistant is protection against data exfiltration and data injection, which are growing concerns among organizations.
Data injection is a cyberattack that involves inserting unauthorized data into a system. It is a concern for legal organizations due to the potential of a data injection attack to reveal confidential client information.
Exfiltration, also known as data theft, is the intentional and unauthorized transfer of data. In the age of generative AI, where a constant stream of data is required to train LLMs, this has become a major concern. In the legal industry specifically, organizations are entrusted to safeguard customer data, and putting it in the hands of third-party LLMs requires a great amount of trust, and also carries the potential risk of exfiltration.
Since customer data is not used to train or fine-tune the third-party LLMs, the LLMs have no inherent proprietary knowledge about a customer’s case that can be exfiltrated through prompt or data injection.
Additionally, every existing task in EverlawAI Assistant is run as a one-time generation request, or for actions such as bulk document summarization, a series of singular requests. The LLM provider does not retain customer data once the request is completed. This eliminates the possibility of a compromised model attempting to elicit proprietary or confidential data from the user.
To protect your data further, EverlawAI Assistant does not offer features or functionality that allow link-following or web-browsing based on interactions with the LLM, helping ensure your information remains protected within Everlaw’s closed-loop system. Additionally, many of our tasks do not take user input, which further mitigates the risk of a malicious user trying to use prompting techniques to exfiltrate sensitive data.
Traceability of Generative AI Usage
Whenever an Everlaw user requests a generative AI task, Everlaw logs the request, as well as the input, output, and prompt where applicable. Everlaw also logs additional data associated with the request, such as the user creating the generation, the time, and the document or documents affected.
This information allows a customer to track generative AI usage across their organization, for both management and billing purposes, as well as for potential troubleshooting and customer support needs. Such information will be made available through a user-facing dashboard and can also be fetched by developers on our servers for auditing and tracing purposes.
EverlawAI Assistant demonstrates accountability by showing users exactly how it came to its conclusion, and the steps it took along the way. For example, when asked to generate coding suggestions for a set of documents, EverlawAI Assistant will provide rationale for the suggestion and offer citations directly in the document text as a way of showing its work.
Building for Accurate Generative AI Outputs
Everlaw’s approach to building generative AI tools is aligned to the competencies where LLMs are the strongest, such as fluency and reasoning, and avoids those most likely to lead to potential issues, such as relying on an LLM’s embedded knowledge.
For example, a task like identifying potential gaps in evidence based on supporting context from case materials is far less likely to result in inaccurate outputs than a broader task, such as asking an LLM-based chat bot to identify all instances of fraud or misrepresentation across a document corpus.
Everlaw strives to provide generative AI tools that are accurate, reliable, and defensible, and each feature and functionality is built with these requirements at their core. However, because Everlaw leverages LLMs trained by third parties, we are not privy to the underlying training dataset and cannot manage bias issues at the foundational model level.
Instead, we incorporate assessments of accuracy in our decisions of which LLMs to leverage within EverlawAI, and evaluate the accuracy of the end-to-end pipeline powering a feature.
All of our generative AI features utilize the grounding technique, in which contextually relevant information for a task is included in the LLMs context window as part of a request.
For example, a request to generate a statement of facts will include relevant information from key documents, as identified by the user, on which to base that statement. While this does not fully eliminate hallucinations, it does dramatically curtail them.
Users can further verify outputs through verification pathways that are built into the user interface, allowing them to verify the output’s accuracy and completeness. In our statement of facts example, for instance, each affirmative statement is followed by the case document intended to support it, much like one might cite to the record in their own drafts. Those documents are accessible in a single click, allowing users to trace back the statement to its primary source in order to verify the output’s accuracy and completeness.
Ultimately, however, we encourage users to perform human verification of all generative AI outputs, and have designed our features and functionality to make such verification a natural part of users’ workflows.
Reducing Bias in Generative AI Outputs
Prompt-based mitigations are one strategy for reducing potential bias. However, given the domains in which Everlaw’s generative AI tools are used, primarily litigation and investigations, such approaches also risk obfuscating bias inherent in case documents themselves – a factor that could be an important issue in a matter. Therefore, as any prompt or system message-based bias mitigation solutions could compromise accuracy on tasks, Everlaw does not utilize this approach.
Regulation and Compliance
Everlaw’s development approach allows us to shift between third-party service providers and/or update features if needed to reflect changing legal, regulatory, and business requirements, and ensure a best-in-class experience for our users.
Because we leverage LLMs trained by third parties and do not incorporate additional training or fine-tuning, we are not well positioned to manage copyright and IP infringement risks at the foundational model level.
Our grounding technique means that the output of the LLM is grounded in its input, and not on the underlying training dataset. We’re designing our generative AI to be as accurate as possible, which means that if, for example, the customer data includes a passage from a book, the summary output by the AI Assistant may include that passage as well.
Everlaw’s goal with EverlawAI Assistant is to help users increase the speed and effectiveness of document review and story building, while allowing them to focus on top litigation priorities. Our generative AI features help identify and extract relevant portions of evidence that the customer provides and accurately reproduces that content with a summary of both the content and relevance.
Data Security and Privacy
Everlaw prioritizes data security and privacy with enterprise-class security protocols and privacy standards because we understand that hosting sensitive customer data on a SaaS ediscovery platform requires an established compliance program and rigorous controls. Everlaw’s generative AI implementation is designed to align with compliance requirements, such as those under FedRAMP, CPRA, GDPR, and other applicable privacy regulations.
Everlaw’s security and compliance program is holistic and part of our core philosophy. It demonstrates our commitment to ethics and our company values, as well as compliance with our security, privacy, and confidentiality commitments to customers, and applicable laws and regulations. Our program is built on top of federal guidance on effective compliance programs because we believe that security, privacy, and compliance go hand-in-hand.
Additionally, due to the fact that we use LLMs trained by third parties, Everlaw’s extensive vendor management program, which requires a security and privacy assessment, verifies that appropriate technical and organizational measures are in place and to meet Everlaw’s privacy and security requirements.
Conclusion
This framework is a commitment to Everlaw’s customers and partners that we have the measures in place for users to have a secure and effective experience with EverlawAI Assistant.
This technology is a tool that attorneys and legal professionals can use to work more efficiently and achieve their goals. With a deliberate approach, and using models that have the appropriate checks and balances in place, organizations can responsibly leverage generative AI to position themselves at the forefront of the changing legal landscape.
Built with our Generative AI Principles in mind, EverlawAI Assistant is created to handle your data thoughtfully and responsibly.
Glossary of Key Generative AI Terms
Generative AI is a branch of AI that focuses on the creation of new content based on learned data patterns. These data patterns are determined when models are trained using machine learning fed by massive amounts of data.
Large Language Models are GenAI models that specialize in reading and writing human-readable, natural language text. Think ChatGPT — you pass it a prompt (e.g. a question) and it returns a response.
Training an LLM involves feeding it data to help it learn and understand language.
Fine-Tuning involves adjusting the parameters of a pre-trained LLM, such as ChatGPT, to a specific task or domain. This is done using input-output pairs of representative examples of the desired behavior.
Context Window is the maximum number of words an LLM can process when generating or understanding language. The bigger the context window, the more text the LLM can make sense of.
Exfiltration, also known as data theft, is the intentional and unauthorized transfer of data.
Data Injection is a cyberattack that involves inserting unauthorized data into a system.
Zero Data Retention is a principle where no user data is retained beyond the needs of the immediate task. For example, Everlaw’s zero data retention agreement with OpenAI means no user data is stored or used to train OpenAI’s LLMs or for any other purpose, and is deleted immediately after the requested task is completed.
Input is any text, information, or other content the customer provides or makes available to EverlawAI Assistant for processing.
Output is what LLMs generate in response to inputs. These are created using next word prediction, which is a form of intuition LLMs use to infer one word after the other.