TBA Law Blog


Posted by: Joel Henry & Michael Pasque on Jun 1, 2016

Journal Issue Date: Jun 2016

Journal Name: June 2016 - Vol. 52, No. 6

Why Technology Assisted Review Is a Must for Data-Intensive Cases

“The fact is, we’re looking for a very small number of very evil needles in a very large haystack, …”

— Charles Clarke, Member of British Parliament, and Home Secretary (2004 – 2006)

Introduction

The practice of law continues to change, albeit slowly. Nowhere does this change impact the legal field more profoundly than the volume of data created by clients, lawyers, courts and the public. Society has gone well beyond being “data rich” to being “data swamped.” Computers, phones, cars, even household appliances, generate enormous amounts of data every day. The practice of law requires attorneys to account for much of this data. They must understand what the data is and how to use it, or face significant risks including sanctions, ethical violations or even disbarment. Yet even the tech-aware attorney can become overwhelmed.

A simple discovery request might include a benign-looking and well-intended request such as: “All emails sent, received or otherwise exchanged by Acme Corporation employees relating to Joe Smith.” However, when a lawyer sends this request on to a client or the technician who handles their technology, such a request becomes a nightmare. Each employee sends or receives an average of 125 emails per day. If Joe Smith’s controversy spanned six months and involved 10 Acme employees, the volume of email to search would be in the neighborhood of 225,000 emails. This fails to count any email attachments, which also need to be examined.

Sheer volume alone hides some nasty challenges. For example, some of those emails may have been archived (stored on backup disks), or come from another email account an employee used such as a web-based email provider. Extracting email from archives and collecting from external email servers will keep your poor technician busy for days on end. And the job isn’t over once emails have been collected. Now they must be reviewed to prevent disclosure of privileged, or otherwise confidential, information (i.e., Social Security numbers, health data and human resource issues).

Manually reviewing such a mountain of email can’t be performed efficiently, especially when reviewers spend a majority of their time sifting through email that has nothing to do with Mr. Smith. Typically, 90 percent of the email collected for review has nothing at all to do with him. One could simply perform a search on the email with the words “Joe” and “Smith,” which would likely reduce that 225,000 email mountain significantly. However, keyword searches, which utilize single words or even combinations of words, both over-collect emails and miss many others. For example, if one employee had a husband named Joe, such a search would return all those emails. Alternatively, if employees referred to Joe Smith as JS, such a search would find none of those emails.

Technology provides the ability to continually create and store data that can be revised and shared at a moment’s notice. However, the ability to save all that data and then find something in it becomes a challenge. In the days of paper records, a law office had no choice but to manually review every piece of paper related to the case at hand. With the technology of today, we can now search these documents faster and more accurately. Using technology to assist document review allows us to reduce one of the most fallible links in the discovery chain: humans. Review that requires humans to spend days reading documents results in errors — lots of them. There is no doubt that machines will make errors too, but human review guided by smart technology reduces the probability of missing otherwise potentially relevant or privileged documents.

Legal Framework

The duty to preserve suffers the most profound impact of this data deluge. Just as an attorney must preserve a vehicle involved in an accident, client and other data must be preserved in the electronic data world. Unfortunately many lawyers struggle to understand technology and therefore see data preservation as a form letter to a client or a technologist. This is not the case, as data preservation requires a hold letter that specifies what to preserve.

Tennessee state courts generally have not imposed sanctions for spoliation when destruction of evidence occurred absent destruction of evidence for an improper purpose. In Bellsouth Advertising & Publishing Corp. v. Abebe, the Tennessee Court of Appeals, citing this requirement, did not impose sanctions for a party’s destruction of documents pursuant to the client’s document retention practices.[1]

A legal hold letter may result from current or reasonably anticipated litigation, audit, government investigation or other such matter that suspends the normal disposition or processing of data. Legal holds may encompass procedures affecting email, document storage, database records, social media content and even text messages. Accessibility of this data may be reasonable or not reasonable at all — try retrieving your text messages from six months ago.

The obligation to preserve evidence arises when a party has notice that the evidence is relevant to litigation or when a party should have known that the evidence may be relevant to future litigation. Identifying the boundaries of the duty to preserve involves two related inquiries: When does the duty to preserve attach and what evidence must be preserved?[2] It is not enough to wait until litigation has commenced to start preserving data; preservation begins much sooner than that. This becomes very important as companies often have data-retention policies that would eliminate the data before litigation could commence to require data preservation.

The duty to preserve extends to those employees likely to have relevant information, including the retention of all relevant documents or tangible things in existence when the duty attaches.[3] While this initially seems like a large task, “a party need not preserve all backup tapes even when it reasonably anticipates litigation” as doing so would cripple large clients.[4] Instead the duty to preserve extends to those employees likely to have relevant information, and not the entire corporate database.[5]

Federal Rules of Civil Procedure. More data results in more problems, in that the likelihood of inadvertent disclosure of privileged information to opposing counsel increases. Fortunately, in 2015 the Federal Rules of Evidence were amended, providing much needed help. In fact, some experts see these changes as an attempt to streamline litigation.[6]
Changes to Rule 26 will affect the e-discovery industry. Many e-discovery vendors routinely charge fees in excess of $100,000, a cost placing these services out of reach for most attorneys. Those who can afford such cost find e-discovery processes to be the opposite of streamlining the entire discovery phase of litigation. The new Rule 26 explicitly makes “proportionality” a key concept within the Rule, and lists six proportionality factors to consider.[7] The amount in controversy and the expense of the proposed discovery may very well drive e-discovery costs down as well as pressure the e-discovery market to provide lower cost alternatives that make such discovery possible in those cases not able to afford pricy solutions. Without a formula or judicial rulings on the balance between the value of a case and e-discovery costs, the real impact of this rule remains unclear.

Rule 37 changes aim to curb harsh sanctions for spoliation of electronically stored information. Many parties fear sanctions that could lose a case and thus spent inordinate amounts of money to save every conceivable form of electronic information from every possible data repository that might remotely be needed. To alleviate this concern, the new Rule 37 provides sanctions “only upon finding that the party acted with the intent to deprive another party of the information’s use in the litigation.”[8] By raising the bar for imposing sanctions, the new Rule intends to “resolve a circuit split” where some circuits held negligent destruction of electronic information sufficient to impose sanctions.[9] This higher bar for sanctions should bring less worry and cost to the preservation process.

Rule 502 generally provides for protection of traditional “subject-matter waiver” standards when the protected information is inadvertently provided to the opposing party.[10] To clarify, the committee directly addressed the cost/benefit problem, commonly referred to as proportionality, when discussing Rule 502, responding to the “widespread complaint that litigation costs necessary to protect against waiver … have become prohibitive due to [disclosure concerns].”[11] While some courts previously held that even unintentional disclosure of protected information constituted waiver, Rule 502 now expressly protects this information. This is certainly helpful to those weary of having to manually review every document or email in order to ensure protection of privileged information.

Tennessee Rules of Civil Procedure. Tennessee Rule 26.02, and specifically the Committee Comments accompanying 26.02, provide much more information on the scope, proportionality and process of producing, and not producing, electronically stored information.

Rule 26.02(1) allows the responding party (the producing party) to avoid producing electronically stored information if it can be shown that “the information is not reasonably accessible because of undue burden and cost.”[12] If shown, the court may still order discovery of this information if the requesting party shows that the likely benefit of the discovered information outweighs the estimated burden or expense.[13] The court must take into consideration almost all of the same factors specified in FRCP 26(b)(1), namely “the amount in controversy, the resources of the parties, the importance of the issues, and the importance of the requested discovery in resolving the issues.”[14] The court shall provide conditions for discovery required under this exception.[15]

Preserve

  • Identify data sources and protect from deletion
  • Determine the need for meta-data and native format

Collect

  • Identify most important issues, custodians and data
  • Prepare data for review — OCR, organize, select

Review

  • Understand the volume of data vs. timeframe and cost
  • Select the best technology to meet the project needs

Advisory Commission Comments to Tennessee Rule 26.02 suggest more responsibility lies with the responding party, namely to “identify, by category or type, the sources containing potentially responsive information that it is neither searching nor producing. The identification should, to the extent possible, provide enough detail to enable the requesting party to evaluate the burdens and costs of providing the discovery and the likelihood of finding responsive information on the identified sources.”[16] The comments go on to outline a process by which both requesting and responding parties work together to evaluate the cost and burden of producing electronic information as well as what happens should this cooperation break down. If a dispute arises, a meeting must take place between the parties prior to the court being called upon to resolve the issue. Once the court becomes involved, the responding party must present arguments as to burden and cost that can be disputed by the requesting party by conducting discovery to test these factors. Such testing might include requiring the responding party to obtain a sample of the electronic information from the data repositories deemed burdensome or inaccessible, or allowing both counsel to inspect sources or data, or obtaining expert witness depositions from those familiar with the responding party’s IT systems. Tennessee Commission Comments greatly assist the parties and the bench in working through these conflicts. The Comments specifically outline the party’s responsibilities as:

The responding party has the burden as to one aspect of the inquiry — whether the identified sources are not reasonably accessible in light of the burdens and costs required to search for, retrieve, and produce whatever responsive information may be found. The requesting party has the burden of showing that its need for the discovery outweighs the burdens and costs of locating, retrieving, and producing the information.[17]

Even if the source of data is not reasonably accessible, the requesting party may still compel discovery by showing good cause and balancing the costs versus benefit of the specific discovery. The factors included in making the ultimate decision include:

  1. the specificity of the discovery request;
  2. the quantity of information available from other and more easily accessed sources;
  3. the failure to produce relevant information that seems likely to have existed but is no longer available on more easily accessed sources;
  4. the likelihood of finding relevant, responsive information that cannot be obtained from other, more easily accessed sources;
  5. predictions as to the importance and usefulness of the further information;
  6. the importance of the issues at stake in the litigation; and
  7. the parties’ resources.[18]

This rule and associated commentary  put those practicing in Tennessee ahead of the rest of the country in that many such decisions occur in the absence of understanding the technical issues or the true cost of e-discovery.

One interesting Tennessee case resulted in cost shifting, which has become rare. In CNX Gas Co., LLC v. Miller Petroleum Inc., CNX requested e-mails, in native format, of Miller Petroleum’s counsel.[19] Miller provided an affidavit from its IT personnel detailing the costs required to review the emails for attorney-client privilege or attorney work product, remove those emails, and then produce the electronic data.[20] The trial court ruled these costs unduly burdensome, and the Tennessee Court of Appeals agreed because Miller had provided the trial court with evidence of the increased cost of reviewing and producing this data.[21] The appellate court noted that the trial court did not articulate the reasons for its decision, but the party opposing discovery need only show specific facts demonstrating the undue burden.[22] Here, the affidavit provided by Miller’s IT department was enough for the trial court’s cost shifting order to be within the court’s discretion.

Authentication. Rules of evidence require authentication, which can be especially challenging with digital evidence. Such authentication can be done using metadata; however, asking for metadata with a discovery request should be done sparingly as gathering metadata associated with every document or email requires significantly more work and expense.

The term “metadata” seems to float around digital evidence and electronic discovery like pollen in the spring. However, what exactly constitutes metadata seems to escape many who use the term. Metadata can be thought of as data about data. For example, documents contain words and sentences — clearly this is data. However, metadata contains data about that document, including when it was created and modified, by whom, on which computer, and when. Metadata can be thought of as a set of unique library catalog cards, one for each document or email you create, read, modify or copy. Thus, just as a catalog card provides background information on a book, metadata provides background information on documents, emails, messages, pictures, blog posts, website visits and a host of other electronic items and actions.

While metadata can be used to authenticate, admissions will always be more common. Only when someone fails to admit involvement in creating, editing or viewing a document, or sending or receiving an email will metadata be needed to authenticate. Of course data processing and electronic discovery vendors will be very happy to provide metadata at attractive rates — attractive to themselves.

Ethical Standards

ABA Rules. American Bar Association Model Rule 1.1 of Professional Conduct expresses in a few words the lawyer’s duty to represent all clients competently. In 2013, the ABA accepted a proposal of the ABA Commission on “Ethics 20/20” to modify one of the comments to this rule in order to make clear that a lawyer must continuously maintain familiarity with technological change in order to comprehend the manner in which technology may affect a particular representation. The added language reads, “To maintain the requisite knowledge and skill, a lawyer should keep abreast of changes in the law and its practice, including the benefits and risks associated with relevant technology, engage in continuing study and education and comply with all continuing legal education requirements to which the lawyer is subject.”[23] Lack of knowledge as to digital evidence cannot excuse a lawyer from such diligence. What’s a lawyer to do? The California Rules of Professional Conduct may supply an answer. Recent changes in California rules included this ominous passage:

Attorneys who handle litigation may not ignore the requirements and obligations of electronic discovery. Depending on the factual circumstances, a lack of technological knowledge in handling e-discovery may render an attorney ethically incompetent to handle certain litigation matters involving e-discovery, absent curative assistance under rule 3-110(C), even where the attorney may otherwise be highly experienced. It also may result in violations of the duty of confidentiality, notwithstanding a lack of bad faith conduct.[24]

This may be the future across the nation. Regardless, the volume of data will not shrink in the future, nor will the importance of some of the needles in the growing haystack of data. The outcome of cases will turn on these needles, as will the ability of lawyers and law firms to compete with those who leverage technology to reduce costs.

Even the most tech-savvy lawyer, using software to find that needle, will likely require the expertise of technology professionals — data from multiple sources, in multiple formats, doesn’t simply appear within a software product without retrieval and conversion. The ABA addressed this issue in a formal opinion, providing that a lawyer who engages a non-lawyer (or lawyer) to provide outsourced services is required to ensure that person’s compliance with Rules 5.1 and 5.3.[25] Therefore, utilizing help from outside vendors to complete a complicated e-discovery task is not unethical. It is, however, the supervising lawyer’s obligation to ensure that those tasks are delegated to individuals competent to engage in them.

Tennessee Rules

Tennessee has no passage like the California Rule. However, Tennessee Rules of Professional Conduct 1.1 and 1.2 would apply. Rule 1.1 requires an attorney to “provide competent representation to a client.”[26] Competent representation includes the “skill, thoroughness, and preparation reasonably necessary for the representation.”[27] Knowledge of, or access to someone who is knowledgeable in, electronic discovery would likely be required to meet this rule.

Rule 1.2, governing the scope of representation, might provide both a safe haven and a risky set of requirements on Tennessee lawyers. Counsel may delineate technical aspects of e-discovery to the client IT specialists and thus avoid errors that could negatively impact the case or the lawyer. However, an attorney must still be knowledgeable regarding the law around e-discovery and the technology involved, or engage another lawyer who is.

This article provides an overview of e-discovery and a host of practical, hands-on advice for those performing e-discovery. For a more detailed review of e-discovery in Tennessee, see the excellent paper by Frazier and Jenkins.[28] For even more in depth information about e-discovery in Tennessee, see the book Electronic Discovery in Tennessee: Rules, Case Law and Distinctions by W. Russell Tauber.[29] This text provides an encompassing review of Tennessee e-discovery legal information. The author continues to be very active in the field of e-discovery law and an excellent resource for Tennessee lawyers.

Data Review

Document review continues to employ methods that date back to the invention of paper — manual review. “Even assuming that the profession had the time and resources to continue to conduct manual review of massive sets of electronic data sets (which it does not), the relative efficacy of that approach versus utilizing newly developed automated methods of review remains very much open to debate.”[30] New associates fear and loath the traditional review of documents, namely sitting in a room with other young lawyers reviewing reams of documents, electronically or in paper form, for days on end. Not only is this effort mentally exhausting but in the age of computers it requires the review of exponentially greater amounts of documents than ever before. Even if it were affordable, manual review of that amount of data leads to error rates of 35-40 percent.[31]

Technology assisted review (TAR) includes a software component that goes beyond simple display of electronic data within the review process. Even though the legal profession views manual review as the gold standard compared to other forms of review, the use of TAR produces accurate, efficient results while still maintaining compliance with applicable discovery rules. “One point must be stressed — it is inappropriate to hold TAR to a higher standard than keywords or manual review. Doing so discourages parties from using TAR for fear of spending more in motion practice than the savings from using TAR for review.”[32] Thus TAR should be viewed on an even playing field with manual review.
Just as when you buy a new computer, there are many choices when it comes to TAR. The three primary types of software available utilize Continuous Active Learning (CAL), Simple Active Learning (SAL) or Simple Passive Learning (SPL). In a study performed by two of the most renowned experts in e-discovery, any active learning process (CAL or SAL) greatly outweighs passive learning (SPL), especially when compared to traditional review techniques.[33]

Active learning differs from passive in that active learning software provides the user with the most likely relevant documents to review while passive selects documents randomly or allows the user to select documents. Active learning leverages user actions to provide increased accuracy and efficiency of the searching and tagging performed by a user while passive learning treats all data the same and provides data to the user that better represents the entire data set (based on random sampling the user sees the data set as a whole). Both processes continually learn from reviewer actions. Both methods use statistical measurements to determine when to let the software mark the remaining documents. After the software marks a set of documents, the user reviews these and corrects any mistakes. The cycle continues until statistical measurements reach pre-set goals, often set as a part of the cost/benefit analysis. Once the software has done enough analysis, the software then marks documents yet to be reviewed by the user. In this way many documents get marked, but the user reviews only a portion of them.

Two very simple terms can be used to convey the effectiveness of TAR: precision and recall. Precision is the percentage of correctly marked documents within all marked documents. Recall is the percentage of correctly marked documents within all documents in the dataset. Essentially, precision measures how accurately a TAR technology uses reviewer markings to mark unreviewed documents. For example, if a user marks 100 documents relevant, and the software marks 1,000 of the remaining 9,000 documents relevant, then precision would be the percentage of the 1,000 documents the software marked that the reviewer confirms as being relevant. Recall works similarly by calculating the number of relevant documents within the remaining unmarked 9,000 documents the software should have marked relevant. An attorney can utilize these measurements to judge the progress of document review and substantiate both the use and accuracy of TAR.

As more time is spent to get closer to that ever-elusive high precision/high recall result, an exponential amount of money is spent. However, the amount of time spent trying to get one more percentage point higher recall or precision may not be proportional to the value of the case. Thus, courts often reiterate that their rulings regarding proportionality for one particular case should not be applied to other cases.[34]

In early 2012, Federal Magistrate Judge Andrew Peck opined one of the first legal affirmations for TAR, stating that “judicial opinion now recognizes that [TAR] is an acceptable way to search for relevant ESI in appropriate cases.”[35] It has been three years since that ruling was published, and courts have generally approved the use of TAR in discovery.[36] It is important to note that while courts approve of the usage of TAR, it is up to the parties to determine when TAR is appropriate. Courts generally have not required TAR in discovery where the party has shown that the benefits do not outweigh the cost.[37]

Many lawyers fear TAR as they assume they must be statisticians or mathematicians to use it. While it is necessary to understand how the technology works, a lawyer need not understand the detailed statistics or programming behind TAR. The choice in technology lies with the producing party as courts resist dictating the use of TAR or what type of TAR to use. Thus, the system used should be dependent on the specifics of the data within the case at hand, which can only be determined on a case-by-case basis.

Production

Data can be produced in one of four formats: native, near-native, near-paper or paper. Each has its benefits and drawbacks, so the process used for one case may not fit the next.

A request for production may include metadata and, if so, such metadata should be delivered with the document in native format. This metadata acts much like a digital fingerprint and can be imperative in authenticating the document. Native production refers to the form ordinarily used by the producing party to store and revise the document. For example, with Microsoft Word, documents would be stored in .doc or .docx format. Production in native format delivers metadata but presents challenges when applying a Bates stamp, performing redaction and controlling privileged or confidential data. Additionally, if the native format requires special software, such as a Computer Assisted Drawing (CAD) package, the opposing party will have no feasible way to view the native file.

The near-native production keeps as much of the original data as possible, but places all documents in a commonly read file format more easily used by the opposing party. For example, CAD drawings can be converted to PDF files. This also makes Bates stamping and redaction easier to perform.

Near-paper production simply produces an electronic file with each document presented as if it were printed and scanned into an electronic file. For example, a group of emails can be exported from Outlook into a single, multi-page, PDF file. Converting to this format allows for Bates stamping, redaction, removal of confidential information and control of metadata. Traditionally lawyers would collect the data to be produced, print it, and then scan it back into a PDF, a costly and inefficient process. Software now allows this process to be performed seamlessly by the user when extracting data, thus decreasing cost and making this an often preferred method of production for data.

Lastly, the paper format is just as it sounds, production completely in paper. As in near-paper, this provides the ability to completely control redaction, Bates stamping, and removal of confidential information. However paper will not include any metadata and, unlike all methods above, will not allow the party to electronically search the information.

Technology that Fits

Currently, discovery extends to any non-privileged matter relevant to any party’s claim or defense, making it very broad and far reaching. Proposed changes to the Federal Rules of Civil Procedure include a change to Rule 26 limiting discovery to be “proportional to the needs of the case.” The motivation for this change stems from an effort to decrease cost and increase efficiency in the age of growing data volumes. The current interpretation of Rule 26 has led to increased costs and delays as firms struggle to sift through large amounts of data even though the expense often far outweighs the benefit to either party. Opinions vary on the impact of such changes but most seem to agree that the proportionality standard will require substantiation beyond legal arguments — namely based on technical and resource expenditures. Most attorneys struggle with these types of arguments, requiring technical experts to weigh in as to the limits and accuracy of the software used.

Many cases and controversies settle without data volume becoming a factor, and without a thorough examination of documents and emails. A case with 1,000 pages of documents and email may appear too small for technology, yet paralegals and lawyers struggle to keep the content of that data organized mentally, especially when handling dozens of matters simultaneously. A case with 10,000 pages requires more than the human memory. However, a matter valued at $200,000 may not warrant a five-figure technology investment. What technology fits a case?

Practicing attorneys can answer this question without the need to become technology geeks. Simple tools that merely organize data into a table of contents linking file names to file locations provide very little value. Tools that truly assist in analyzing, relating and understanding the content in a dataset provide the real assistance needed. Software such as Encase, Logikull, iPro, and START:Review fill this space. The current market offers varying solutions, from cloud-based data review to on-site software. With each vendor having their own methods to perform the complicated analyzation required, this allows lawyers to shop around, finding the right features, at the right price, to find those needles in their haystack of data.

Conclusion

E-discovery is an area of the law that every lawyer should embrace and understand enough to work in. Big data is not going away, and sooner or later the time will come that requires the use of complicated software to navigate a case. Understanding how TAR can help you wade through a seemingly impossible task is the first step in embracing e-discovery, and is your obligation to effectively serving your client by providing them more bang for their buck.

Notes

  1. M2010-01020-COA-R3CV, 2011 WL 1642478 (Tenn. App. Apr. 28, 2011).
  2. Zubulake v. UBS Warburg LLC, 220 F.R.D. 212, 216 (S.D.N.Y. 2003).
  3. Id. at 217; Fed. R. Civ. P. 34(a).
  4. Id. (emphasis added).
  5. Id.; Fed. R. Civ. P. 34(a).
  6. Andrew J. Kennedy, “Amended Federal Rules: Streamlining Litigation,” 41 Litig. News 2 (2016).
  7. Fed R. Civ. P. 26(b)(1).
  8. Fed R. Civ. P. 37(e)(2).
  9. Supra note 6.
  10. Fed. R. Evid. 502(a)-(b).
  11. Fed. R. Evid. 502 Advisory Committee Notes.
  12. Tenn. R. Civ. P. 26.02(1).
  13. Id.
  14. Id.
  15. Id.
  16. Id, advisory comm. cmt. 2009.
  17. Id.
  18. Id.
  19. E2009-00226-COA-R3CV, 2011 WL 1849082, 2 (Tenn. App. May 11, 2011).
  20. Id. at 11.
  21. Id. at 10.
  22. Id. at 10-11.
  23. ABA Model R. of Professional Conduct, 1.1, cmt. 1 (2013) (emphasis added).
  24. State Bar of Calif. Standing Comm. on Prof’l Responsibility & Conduct, Formal Op Interim No. 11-0004 (2015).
  25. ABA Standing Comm. on Ethics & Prof’l Responsibility, Formal Op. 08-451, (2008).
  26. TN R S CT Rule 8, RPC 1.1.
  27. Id.
  28. Roe Frazer & Marc Jenkins, “The Future of E-discovery in Tennessee,” 1 Belmont L. Rev. 181, 182 (2014).
  29. Tauber, W. Russell, Electronic Discovery in Tennessee: Rules, Case Law and Distinctions, (2012).
  30. The Sedona Conference, “The Sedona Conference Best Practices Commentary on the Use of Search and Information Retrieval Methods in E-Discovery,” 8 Sedona Conf. J. 189, 199 (2007).
  31. Id.
  32. Rio Tinto PLC v. Vale S.A., 306 F.R.D. 125, 129 (S.D.N.Y. 2015).
  33. G. V. Cormack & M. R. Grossman, “Evaluation of machine-learning protocols for technology-assisted review in electronic discovery,” Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval, 153-162, (ACM 2014).
  34. Moore v. Publicis Groupe, 287 F.R.D. 182, 193 (S.D.N.Y. 2012) adopted sub nom. Moore v. Publicis Groupe SA, 11 CIV. 1279 ALC AJP, 2012 WL 1446534 (S.D.N.Y. Apr. 26, 2012); Rio Tinto, supra note 12.
  35. Id. at 183.
  36. Green v. Am. Modern Home Ins. Co., No. 14–CV–04074, 2014 WL 6668422 at 1 (W.D.Ark. Nov. 24, 2014); Aurora Coop. Elevator Co. v. Aventine Renewable Energy–Aurora W. LLC, No. 12 Civ. 0230, Dkt. No. 147 (D.Neb. Mar. 10, 2014); Edwards v. Nat’l Milk Producers Fed’n, No. 11 Civ. 4766, Dkt. No. 154: Joint Stip. & Order (N.D.Cal. Apr. 16, 2013); Bridgestone Am. Inc. v. IBM Corp., No. 13–1196, 2014 WL 4923014 (M.D.Tenn. July 22, 2014); Fed. Hous. Fin. Agency v. HSBC N.A. Holdings Inc., 11 Civ. 6189, 2014 WL 584300 at 3 (S.D.N.Y. Feb. 14, 2014); EORHB Inc. v. HOA Holdings LLC, No. Civ. A. 7409, 2013 WL 1960621 (Del.Ch. May 6, 2013); In re Actos (Pioglitazone) Prods. Liab. Litig, No. 6:11–MD–2299, 2012 WL 7861249 (W.D.La. July 27, 2012) (Stip. & Case Mgmt. Order); Global Aerospace Inc. v. Landow Aviation LP, No. CL 61040, 2012 WL 1431215 (Va.Cir.Ct. Apr. 23, 2012).
  37. In re Biomet M2a Magnum Hip Implant Prods. Liabl. Litg., 2013 WL 1729682 & 2013 WL 6405156; Kleen Prods. LLC v. Packaging Corp. of Am., 2012 WL 4498465.

Dr. Joel Henry DR. JOEL HENRY concentrates on the intersection of law and technology, utilizing his education and experience in both to practice law, consult, teach and advance legal technology solutions through writing and speaking on the subject. He continues to teach and research as a professor of computer science and adjunct professor of law at the University of Montana while serving as president of Agile Data Solutions Inc, a company he started three years ago.

 

Mike Pasque MIKE PASQUE is a candidate for J.D. in 2016 at the Blewett School of Law at the University of Montana. He has worked at the Office of Legal Counsel for the University of Montana, and as a specialist with Agile Data Solutions, where he assists clients in overcoming their e-discovery challenges.