Discovery & Review Services

Job Description: Review of 88GB of email data for an internal investigation - Time Frame: 14 days

Law In Order’s expert review team worked closely with our client to reduce their reviewable set of potentially relevant documents from 747,000 to 6,500 documents in 48 hours. The responsive documents were uploaded the next day with training provided to the team who, with the assistance of our experts, completed a review of those remaining documents in less than 10 days. Once the review was complete, our experts undertook various quality assurance processes and provided our client’s review team with an exported hyperlinked list of documents within 24 hours. In total, the review of the documents was completed in less than 14 working days saving our client a significant amount of time and money.

Job Description: 778 GB of data to be reviewed - Cost Savings: $285,000 AUD

The Challenge

Our client—a large energy company—was involved in litigation over an environmental matter. Only one lawyer was assigned to the case, overseen by one independent consultant—an especially small team, given the 778GB of data collected for the case. In an effort to save costs, both sides agreed on a list of keywords to help identify relevant documents.

The Solution

De-duplication culled the client’s original count of 6.6 million documents down to 3 million documents. The list of keywords culled the document count significantly further, down to only 157,000 documents. However, with overly inclusive keywords like “environment” being used, it was evident there were still many non-responsive documents left in the collection to sort out—and that meant a lot of work remained for only one person to review.

Law In Order proposed using Relativity Assisted Review.

Justin Smith, Global Head of eDiscovery at Law In Order, oversees the data processing services and assisting legal departments by offering technical solutions and workflows. Justin has over 14 years experience in the areas of dispute resolution, eDiscovery, project management, and software development. He understands the complexities of unstructured data and offers solutions to make electronically stored information available for search, retrieval and review in a legally defensible manner.

“I am a great advocate of Assisted Review. We always put this option forward for any matter that has a high volume of documents,” said Justin. “For this matter, there were a number of reasons it was a great option for our client.”

Most notably, Law In Order calculated that if the one lawyer had to review all 157,000 documents linearly— strictly to identify relevant documents—the review would take 1,570 hours to complete. With the lawyer open to alternative processes, the decision was made. The team would use Technology Assisted Review.

The Technology Assisted Review Process

To begin the workflow, Our team used the overly inclusive keyword-responsive document set of 157,000 documents.

“We generally don’t use keywords in addition to Technology Assisted Review, but in this scenario, it actually fit the bill. It’s important to have a decent amount of both non-relevant and relevant documents to train the software—and after applying the keywords, it still looked like we were left with a good mix of relevant and non-relevant documents,” said Justin.

The lawyer completed three review rounds—reviewing a random sample of 1,000 documents during each round—to train the software on responsive versus non-responsive. For each round, six to seven percent of the documents were identified as responsive.

From there, the lawyer tested Relativity’s accuracy by conducting a QC round. For this fourth round, Justin and his team took a statistical sample of documents based on a 95 percent confidence level and a two percent margin of error. This resulted in a sample of 2,226 documents—and the results were consistent, as six to seven percent of the documents in the sample were coded as responsive.

In addition to the consistent results, the lawyer overturned only 174 of the 2,226 documents in the QC round. In other words, he disagreed with the software’s coding decisions only 7.8 percent of the time, further demonstrating the computer’s accuracy. Moreover, he made an interesting discovery while reviewing the overturns.

“As he double-checked the overturns to see what led the computer to make an incorrect decision, he actually ended up agreeing with the software the majority of the time,” said Justin. “He realized his initial decisions were wrong.”

With this realization and the already low overturn rate in mind, the lawyer felt the computer had achieved a consistent level of accuracy for this case—even after just one QC round—and decided to stop review.

Using the logic it learned from reviewing nearly 5,000 documents—three training rounds and one QC round— Relativity categorized the remaining 152,000 documents. In the end, 27,122 documents were marked responsive, 117,635 were non-responsive, and 300 remained uncategorised.

“Because of the limited resources, the legal team had very little time to complete this review. But, they were able to get good results quickly using Assisted Review,” said Justin.

The Outcome

Relativity Assisted Review saved our client a considerable amount of money. Without this, the client would have likely hired a junior lawyer—generally billing at $200 AUD/hour—to manually review all 157,000 keyword-responsive documents. Assuming the junior lawyer reviewed strictly for relevance at a rate of 100 documents per hour, it would have taken 1,570 hours to complete the first-pass review and would have cost the client $314,000 AUD.

However, by using Relativity Assisted Review, the client’s senior lawyer was able to complete the first-pass review at an estimated cost of $29,000—a savings of approximately $285,000 AUD for our client.

“This matter is a great example of the efficiency and cost savings realised by the use of Assisted Review,” said Justin. “Our client was very pleased with the results.”

Contact the team

Download our full case study

Job Description: Novice eDiscovery users required to assess 40GB Lotus Notes Files - Time Frame: 1 week

Our client with no eDiscovery experience had to assess 40 GB worth of Lotus Notes e-mail files. They wanted a quick, low cost option to access the information on hand in order to evaluate the scope of the issues and determine their approach. We ran the files through Venio which gave lawyers access to the data within hours. As the client needed to ascertain how connected the e-mails were in relation to the matter – they wanted a clear picture of the parties involved, our Solutions Consultants ran keyword searches and showed the social network map that visually shows who this person sent emails to which proved to be a great tool to the legal team. The use of Venio allowed for the legal team to complete scoping in 2 days and the Discovery completed within 7 days. The job was delivered earlier than expected and under budget.

Job Description: Regulator request for information, unknown data size & scope - Time Frame: 10 days

The Challenge

Our client with no eDiscovery experience had to assess 40GB worth of Lotus Notes e-mail files. They wanted a quick, low cost option to access the information on hand in order to evaluate the scope of the issues and determine their approach.

The Solution

We ran the files through Venio which gave lawyers access to the data within hours. As the client needed to ascertain how connected the e-mails were in relation to the matter – they wanted a clear picture of the parties involved, our Solutions Consultants ran keyword searches and showed the social network map that visually shows who this person sent emails to which proved to be a great tool to the legal team.

The Outcome

The use of Venio allowed for the legal team to complete scoping in 2 days and the Discovery completed within 7 days. The job was delivered earlier than expected and under budget.

Contact the team

Job Description: Fast discovery deadline - Time Frame: 3 days

The Challenge

Our client had 1.2TB of data they needed to assess to determine the scope of discovery.

The Solution

We received 1.2TB of data on three hard drives.

The Outcome

The data was ingested and indexed into Venio FPR within 3 days, allowing the lawyers to view, search and analyse the documents to accurately determine the best course forward in the matter.

Contact the team

Management Services

Job Description: Last minute request for eTrial - Time Frame: 48 hours

The Challenge

Law In Order received a call at 5:00 pm on a Friday with an urgent request for quotation for a court ordered eTrial involving the provision of evidence display solution to be managed by Counsel. There were two parties, 5 barristers and 5 lawyers. Law In Order had already worked with the plaintiff to provide an eCourtbook. The trial was schedule to commence on Monday.

The Solution

Law In Order responded with a quotation within 10 minutes of the request and received the go ahead from the client to proceed by 5:40 pm.

The Outcome

Law In Order worked through the weekend to ensure all the logistics were secured and set up the eTrial ready for proceedings to begin the following Monday at 10 am.

Relativity

Job Description: Rising business demand in 2017 meant larger and more complex jobs.

 

We had been relying on a legacy solution for processing our data into Relativity. When we took on an enormous job involving a company splitting into two entities, we knew our existing processing solution was not equipped to handle a project of this magnitude.

“Our team was manually mapping data fields and managing multiple imports and exports,” said Justin Smith, Global Head of eDiscovery at Law In Order. “Handling multiple data footprints was becoming tedious, and our storage costs to manage these systems were adding up.”

We had been considering Relativity Processing. This huge job — 8.5 TB of data with nearly 32 million documents — would be the perfect opportunity to put it to the test before full adoption.

DataFootprintIMage.png

 

Go Big or Go Home

Our client tasked us with analyzing its assets and identifying which documents belonged to the entity of the company that was sold off.

Phillip Buglass, Lead Consultant at Law In Order, strategised about the best way to get the data in front of review teams as quickly and accurately as possible.

“We were collecting and receiving continuous amounts of large data, so we couldn’t just hit and go,” Phillip explained. “With a broad collection of 32 million documents, we needed to cull junk data and only promote data relevant for review.”

Law In Order partnered with the Relativity Customer Success team to get the 8.5 TB of case data imported into Relativity Processing.

Then, devising a plan of attack to navigate the enormous data set, the eDiscovery team started an early case assessment (ECA) workflow to promote reviewable documents to reviewers. By doing so, the team parsed out specific file types within personal storage files—like emails, calendar items, and contacts—and applied de-duplication to reduce the amount of data from the get-go.

Law In Order’s prior solution involved loading data in phases, manually mapping fields, and continuously importing and exporting data, according to Justin.

“By the time our data was ready for review, we would have already quadrupled the data,” Phillip said. “With Relativity, we can just point and shoot, and the data is ready for attorneys to start reviewing immediately.”

Reduced by 75%

Using Relativity Processing and ECA reduced our overall data footprint by 75 percent with a 25 percent cost savings. In the second week of the project, Justin and Phillip had the following conversation over Skype:

Graphic.png

“The project’s results gave us the confidence we needed to adopt the entire Relativity platform for all our jobs,” Justin said. “We can continue to deliver top quality results to our clients in a fraction of the time.”

Contact the team

Job Description: Our client received a tranche of disclosure from another party totaling around 55,000 documents.

 

The Challenge

Our client received a tranche of disclosure from another party totaling around 55,000 documents. Their aim was to review all 55,000 documents, but to prioritise the review to find documents similar in nature to their own key documents first.

The Solution

Relativity has various tools to identify similar documents.  Documents can be grouped by near textual duplicates, clustered by concepts or fed into a Technology Assisted Review (TAR) workflow.

In this matter where the client wanted to review all documents, but prioritise the order of review, the best solution was to deploy Relativity’s active learning workflow.

Active learning learns from decisions made on documents as they are coded in the review workflow. It uses these decisions to continuously deliver documents to a review queue based on what the software believes to be the next most similar document to those already coded as relevant.

As the review progresses, the review queue is continually updated based on the decisions being made.

In this matter we were able to take the documents that the client had already identified as key documents in their own disclosure and use these as a pre-coded set to train the system and kick start the active learning review queue.

The Outcome

The active learning project showed its value from the start.

Using only a very small seed set of around 260 key documents, the project returned a mean relevance rate of 45% for the first 2,000 documents. By the second 2,000 documents, the mean relevance rate had fallen to 24% and by the third set of 2,000 documents the mean rate had fallen to just 12%. In overall terms, the mean relevance rate fell from an initial peak of 63% (for the first 200 documents) to 6.8% by the end of the project, with the last 20,000 documents having a mean rate of just 1.54%.

The below graph illustrates how using active learning enabled the review team to review the most relevant documents sooner.

OverallMeanRelevance_Diagram.jpg

Contact our team

Download the full case study

 Can't find the resource you need?

Get in contact with the Law In Order team or call us on 1300 004 667.

Get in Touch