Selling Relativity Analytics to Internal Stakeholders · • Bridgestone Americas, Inc. v. Int....

© kCura LLC. All rights reserved.

Selling Relativity Analytics to Internal Stakeholders


• Michael Longstreth, Reed Smith

• Jesse Murray, Uber

• Jorge Romero, Foley & Lardner

The San Francisco Steering Committee


Who are you?


• Relativity Analytics Adoption

• Overview of Relativity Analytics

• Articulating the Value of Relativity Analytics

• Common Objections

• Takeaways

• Selling Analytics: Scenarios

Agenda


Relativity Analytics Adoption


246Customers with Analytics

Subscriptions


Current RA Subscription Mix by Client Type

3525

116

747

4

30

55

0

20

40

60

80

100

120

140

160

Corporation Government Law Firm Litigation Service Provider

Subscription No Subscription


Over

1.6 BillionDocuments Analyzed


Cumulative GBs Indexed through Analytics

0

100,000

200,000

300,000

400,000

500,000

600,000

700,000

800,000

900,000

1,000,000

1,100,000

1,200,000

Ma

r-09

Ma

y-0

9

Jul-0

9

Se

p-0

9

Nov-0

9

Jan

-10

Ma

r-10

Ma

y-1

0

Jul-1

0

Se

p-1

0

Nov-1

0

Jan

-11

Mar-

11

Ma

y-1

1

Jul-1

1

Se

p-1

1

Nov-1

1

Jan

-12

Ma

r-12

Ma

y-1

2

Jul-1

2

Se

p-1

2

Nov-1

2

Jan

-13

Ma

r-13

Ma

y-1

3

Jul-1

3

Se

p-1

3

Nov-1

3

Jan

-14

Ma

r-14

Ma

y-1

4

Jul-1

4

Se

p-1

4

Nov-1

4

Jan

-15

Ma

r-15

Ma

y-1

5

Jul-1

5

Se

p-1

5

Nov-1

5

Jan

-16

Ma

r-16

1,125,561


Analytics – Average Monthly GBs Indexed By Year

1,0333,664

7,393

12,342

18,532

35,924

56,609

0

10,000

20,000

30,000

40,000

50,000

60,000

2010 2011 2012 2013 2014 2015 2016

2010 2011 2012 2013 2014 2015 2016


• Sedona Principles:

– Number 11 – use of “electronic tools and processes, such as data sampling,

searching, or the use of selection criteria, to identify data reasonably likely to contain

relevant information.”

– Commentary to Principle 11 - possible to use technology to search for ‘concepts,’

which can be based on ontologies, taxonomies, or data clustering approaches, for

example.”

Trend: Using Technology to the Fullest


• Sedona Conference Cooperation Proclamation:

– Leading jurists, trial attorneys, corporate counsel, government lawyers, and others are

signing onto “The Cooperation Proclamation”. By doing so, they are pledging to…

• Reverse the legal culture of adversarial discovery that is driving up costs and

delaying justice

• Help create “toolkits” of model case management techniques and resources for

the Bench, inside counsel, and outside counsel to facilitate proportionality and

cooperation in discovery

• Help create a network of trained electronic discovery mediators available to

parties in state and federal courts nationwide, regardless of technical

sophistication, financial resources, or the size of the matter.

Trend: Openness and Transparency


Overview of

Relativity Analytics


• Concept searching

• Categorization

• Clustering

• Keyword expansion

Structured Conceptual

• Email threading

• Textual Near Duplicate ID

• Language identification

Relativity Analytics Features


What is it?

• Identifies and arranges emails that were part of a single thread or conversation.

What is it used for?

• Allows you to:

– Easily see the order of each email in a thread.

– See which emails are inclusive (i.e. have unique content).

– Identify email duplicate spares (i.e. emails with the same content).

How will it help me?

• Sort and organize emails by thread for more intuitive review.

• Saves time if only reviewing the non-duplicative inclusive emails. ◊

Email Threading


2/24/99 11:25 a.m. 4/29/99 6:45 p.m. 4/30/99 9:03 p.m. 4/30/99 7:00 p.m. 4/30/99 7:22 p.m. 4/30/99 10:24 p.m. 5/1/99 12:57 a.m.

Barry Pearce

Richard Sage

& Mark Elliott

Maria Nartey

Bob Crane

& Jeff Harbert

?


What is it?

• Identifies documents with highly similar text and places them into relational groups.


• Allows you to:

– Use near dupe groups in searching or filtering.

– Conflict check coding decisions amongst near dupes prior to production.


• Saves time by identifying very similar documents prior to the start of review. You can also use

the near dupe groups for review and QC. ◊

Textual Near Duplicate Identification


Version A Version B

Can you spot the difference?


What is it?

• Determines a document’s primary language and up to 2 secondary languages.


• Allows you to see how many languages are present in your collection, and the percentages of

each language by document.


• Easily filters documents by language and batch out files to native speakers for review.

• Determines if translation is needed. ◊

Language Identification


What is it?

• Searches using a sentence, paragraph, or entire document

• Returns documents related to the concept of the query


• Searching for documents based on ideas instead of absolutes

• More natural querying


• Find documents even if terms differ. ◊

Concept Searching


?

?


What is it?

• A classification method that uses example documents which are defined by a reviewer


• Prioritizing review by using previously coded data as examples

• Identifying documents which are related to the hot documents identified by expert reviewers


• Prioritize review by a time or reviewer standpoint

• Sort through a large volume of data quickly

• Run a quick production after validating results ◊

Categorization


Price Fixing:

8 documents ◊


What is it?

• Use the power of the conceptual index to identify groups of conceptually related documents.


• This can be used as a tool for investigation, analysis, review, or QC.


• Investigate a large unknown dataset

• Cull out non-relevant documents quickly

• Speed up a linear review by batching conceptually related documents together ◊

Clustering


Custodian Name

Custodian Name

Custodian Name

Custodian Name

keyterm

date range

filetype


What is it?

• Uses the concept space to allow users to submit terms and returns conceptually related words


• Investigating the language of the workspace using known keywords


• Allows you to find code words

• Assists in expanding the keyword list

• Familiarize yourself with the language of the case. ◊

Keyword Expansion


Articulating the Value of

Relativity Analytics


• Audiences identify with use cases over features

• More memorable for your internal client

• More natural method of “selling”

Articulating Value – Use Cases


• Email threading

• Foreign language identification

Use Case Feature

• Narrowing the review set

Use Case Features


• Near duplicate identification

• Cluster visualization

Use Case Feature


• Quality control

Use Case Features


• Keyword expansion

Use Case Feature


• Quality Control

• Investigation

Use Case Features


• Clustering

• Categorization

Use Case Feature


• Investigation

• Quality control

• Organizing large sets of data

Use Case Features


Common Objections


• Messaging with analytics in e-discovery has focused on large case wins.

• There are a number of uses for a majority of cases. For example:

– Batching - Reviewing conceptually related documents increases review speed

– Production prep – Analytics can help to avoid mistakenly producing privileged docs

– Keyword sampling – Address keyword issues to help identify other potentially relevant

documents

– Threading – Only review inclusives and reduce the volume of email

“Analytics is best only for the largest cases.”


• Email threading will save clients money on almost every case

• Other workflows that can help save your client money:

– Batching by Clusters

– Categorization

– Relativity Assisted Review

“Analytics is too expensive”

Confidential - kCura 2015


• Da Silva Moore, et al. v. Publicis Groupe, No. 11 Civ. 1279 (ALC)(AJP), 2012 WL 607412

(S.D.N.Y. Feb. 24, 2012).

– First court to explicitly approve the use of TAR in e-discovery

– “What the Bar should take away from this Opinion is that computer-assisted review is an

available tool and should be seriously considered for use in large-data-volume cases where

it may save the producing party (or both parties) significant amounts of legal fees in

document review. Counsel no longer have to worry about being the ‘first’ or ‘guinea pig’ for

judicial acceptance of computer-assisted review.”

“Analytics is not defensible”



• Dynamo Holdings Ltd. P’ship v. Comm’r of Internal Revenue, Nos. 2685-11, 8393-12 (T.C.

Sept. 17, 2014).

– Tax court petitioner sought permission to use predictive coding

– First time the tax court sanctioned the use of TAR

– Court recognized that it is widely accepted and does not cause an undue burden

– Update – July 2016

• Tax Court denied IRS motion to compel taxpayers to produce all ESI that hit on search terms.

• The court held that the taxpayers satisfied the reasonable inquiry standard when they

responded using predictive coding.




• Bridgestone Americas, Inc. v. Int. Bus. Machs. Corp., No. 3:13-1196 (M.D. Tenn. July 22,

2014).

– Defendant objected to use of predictive coding because it would change the original case

management order

– Screening of search terms already completed

– Rule 26 requires discovery be tailored “by the court to be as efficient and cost

effective as possible.”

– Magistrate allowed plaintiff to switch horses in midstream

– Openness and transparency are critical and expected




• Progressive Cas. Ins. Co. v. Delaney, No. 11–CV–00678, 2014 WL 3563467 (D. Nev. July

18, 2014).

– Production to the FDIC by Progressive. Keyword culling was done to narrow population

from 2 million to 565,000 documents that hit on key words.

– Progressive used TAR without defendant’s agreement and without leave of court. Court

rejected the use of TAR because Progressive failed to inform.

– No discovery about discovery – exceeds the scope of Rule 26 – a reasonable search is

required

– Progressive ordered to produce all 565,000, but permitted to apply a privilege filter




Takeaways


• Document sets continue to grow along with the pressure to drive down cost.

• Relativity Analytics can save your case team money.

• Analytics should be used on every case (QC, Review speed, organizing large sets of

documents, narrowing down the set).

• Be an advisor. Be prepared to explain your workflow and what happens next.

• Know the use cases and your own success stories.*

Takeaways


Selling Analytics: Scenarios


Scenario #1

You have a discovery deadline quickly

approaching. You were on target for your

deadline until you were just dropped with 100

GB of data to review. How will you get

through this data in time for your deadline?


Scenario #1 – Solution

You have a discovery deadline quickly approaching. You were on target for your

deadline until you were just dropped with 100 GB of data to review. How will you get

through this data in time for your deadline?

1. Use your existing coded documents as examples in a categorization set.

2. Create a search to find documents categorized as Responsive with a rank higher than 80.

3. Batch these out to second level review.

4. Create a search to find documents categorized as Not Responsive and rank greater than

80, or where they are not categorized.

5. Batch these out to first level review.

The key is prioritization.


Scenario #2

The case team received 5 paragraphs from a

subject matter expert depicting potential

conversations among three people that

corporate counsel believes to be important.

How will you find these types of conversations

between these three custodians?


Your attorney received 5 paragraphs from a subject matter expert

depicting potential conversations among three people that corporate

counsel believes to be important. How will you find these types of

conversations between these three custodians?

1. Create a search to find these three custodians’ documents.

2. Create a categorization set.

3. Add each paragraph as an example, using Text Excerpts.

4. Run categorization against the documents in Search #1.



Scenario #3

You need to QC your production to make sure

no privileged documents go out the door.

How will you speed up this process to be as

efficient as possible?



You need to QC your production to make sure no privileged documents

go out the door. How will you speed up this process to be as efficient

as possible?

1. Run Textual Near Duplicate Identification against all documents.

2. Run a search using metadata fields to find privileged documents.

3. Include Textual near duplicates on the search and filter down to find

inconsistencies.


Scenario #4

You’ve already coded your own documents,

and you just received a production from

the opposing counsel. You’ve been data

dumped! How will you find the relevant

documents that you need?


Scenario #4 – Solution (Option A)

You’ve already coded your own documents, and you just received a

production from the opposing counsel. You’ve been data dumped! How

will you find the relevant documents that you need?

Option A:

1. Cluster the received production data.

2. Evaluate the clusters to see if there is any junk that can be quickly

eliminated from review.


Scenario #4 – Solution (Option B)

You’ve already coded your own documents, and you just received a

production from the opposing counsel. You’ve been data dumped! How

will you find the relevant documents that you need?

Option B:

1. Use your existing documents as examples in a new categorization set.

a. Designation, Issues, Privilege, Hot Docs, etc.

b. Use the production as the “Documents to be Categorized” search

2. Cluster the received production documents.

3. Use Pivot and compare each cluster with the categorized issues.


One More Thing…


• New iPad app separate from Relativity

Binders

• Mobile-friendly design for saved

searches, views, document lists, and

coding layouts as they exist in Relativity

• All coding on mobile is synchronized to

and viewable in Relativity

Mobile Beta Program


• Experience the new app before it

becomes publicly available.

• Explore new features added every

month.

• Guide the development of the product –

a direct-line to Product Management.

• Email [email protected] to

participate.

Join the Beta!

mailto:[email protected]


• Administer Relativity from your

smartphone

• Case strategy and construction

• Reporting dashboards

• Support for additional devices and

operating systems

In the Future


Relativity Project Management

Specialist Certification

75-question test

Take it online or in-person

Sign up by August 16 to save 50%


One More “One More Thing”…

Selling Relativity Analytics to Internal Stakeholders · • Bridgestone Americas, Inc. v. Int....

Documents

Transcript of Selling Relativity Analytics to Internal Stakeholders · • Bridgestone Americas, Inc. v. Int....