Legal Commentary

Document Review: People or Machines?

By Patrick J. Lamb

One of the great battles law firms have fought with their clients is over e-discovery.  Law firms have wanted to have their people do the document review because they then capture the profit associated with that review.  Smaller law firms that do not have the manpower to conduct internal reviews have urged use of outside resources, and clients generally push for the approach that is the cheapest while still being effective.  My partner Hugh Totten just shared with me a study, Document Categorization in Legal Electronic Discovery: Computer Classification vs. Manual Review, published in the Journal of the American Society For Information Science and Technology (January 2010).  The study concludes:

This study is an empirical assessment of two methods for
identifying responsive documents. It set out to answer the question of whether there was a benefit to engaging a traditional human review or whether computer systems could be relied on to produce comparable results.

On every measure, the performance of the two computer
systems was at least as accurate (measured against the original
review) as that of a human re-review. Redoing the same
review with more traditional methods as was done during the
re-review had no discernible benefit.

There may be other factors at play in determining legal reasonableness,
but all other things being equal, it would appear that employing a system like one of the two systems employed in this task will yield results that are comparable to the traditional practice in discovery and would therefore appear to
be reasonable.

The use of the kind of processes employed by the two
systems in the present study can help attorneys to meet
the requirements of Rule 1 of the Federal Rules of Civil
Procedure: 'to secure the just, speedy, and inexpensive
determination of every action and proceeding.'

If we are not there already, we will soon be at a point were determinations of which must be produced will largely be handled via computers.  Indeed, given the error rate reported for human production, it is easy to envision a motion attacking that manner of selecting documents for production, at least in large cases.

Full post as published by In Search of Perfect Client Service on January 15, 2010 (boomark / email).