Playing Moneyball in the Compliance Department

Ryan McConnell, Daniel Trujillo, and Katelyn Richardson

Corporate Counsel

05-31-2012

It's about getting things down to one number . . . we'll find the value of players that nobody else can see. People are overlooked for a variety of biased reasons and perceived flaws. Age, appearance, personality. Bill James and mathematics cut straight through that. Billy, of the 20,000 knowable players for us to consider, I believe that there's a championship team of 25 people that we can afford. Because everyone else in baseball undervalues them. Like an island of misfit toys.

-Peter Brand (played by Jonah Hill); Moneyball (2011).

In the movie Moneyball, Brad Pitt plays Billy Beane, the innovative manager of the Oakland Athletics baseball team who used statistical data about player performance to lead the team to 20 straight wins in the 2002 season. Beane analyzed how pitchers released the ball, where the ball crossed home plate, which pitches drew the most swings, and how often players got on base. Moneyball is a baseball movie, but more broadly it illustrates how empirical data and statistics have changed the way we analyze and solve problems.

Baseball franchises are but one of many kinds of organizations using statistics to improve performance. Some legal departments have begun gathering statistical data to evaluate the effectiveness of their compliance programs. Several years ago, lawyers would talk at compliance conferences about the elements of an effective compliance program under the United States Sentencing Guidelines. “You have to assess the program,” a panelist would say, but there was not always a clear, data-driven way to do so. The guidelines have not changed significantly, but technology has changed the way companies assess their compliance programs. Across the globe, in-house compliance lawyers are using empirical data and statistics to effectively manage allegation programs, document production, due diligence, compliance audits, and training.

Allegation programs can now be driven by data. Ethics helpline software such as Ethics Point provides compliance lawyers the number, source, and type of allegations of compliance failures. The software tracks allegations from the initial complaint to resolution. A compliance lawyer can instantly find out how many anonymous allegations the company receives as compared to allegations where employees identify themselves—the number of anonymous allegations may say something about how the employees view the compliance program and how likely the employees are to report allegations within the company. On an annual basis, empirical compliance allegation data can tell in-house counsel where to allocate additional compliance resources, such as training and audits, to address risk.

Document production and review in response to compliance allegations has changed as well. Instead of an army of outside lawyers responding to government subpoenas, software with self-learning algorithms can search for keywords and patterns in documents to cull through volumes of information and find only the most relevant documents. These search tools can analyze more than 1,000 file types, including audio and video, originating from over 400 data sources in more than 150 languages. This type of e-discovery software saves companies money by allowing both inside and outside counsel to work more efficiently.

Empirical data has also changed the way companies provide training to their employees. No longer do companies provide the same generic training to every employee—training is now risk based. Compliance departments analyze information about company employees and evaluate risks posed by different job positions and environments. For anticorruption compliance, training efforts may focus on employees who pose the highest bribery risk based on their interactions with government officials or their role as a financial gatekeeper. Data protection training is likely to focus on employees who have access to and transfer sensitive data. Trade control compliance training may focus on employees in the supply chain or who manage products and information. Employees in less-sensitive positions may receive more generic online training, as opposed to focused live training. With technology, companies can electronically track how many employees are trained and get feedback on the training to help improve future sessions. Empirical data and technology have made training large employee populations a manageable task.

Data analysis has also changed the way companies conduct due diligence and audits. Today, due diligence is focused on the risks posed by different types of intermediaries. For anticorruption compliance, risk-based due diligence looks at intermediary interactions with foreign officials, the operating environment, amount and length of the contract, and background information and references. For instance, marketing agents or local content partners receive increased scrutiny depending upon the country of operation; law firms and lower-risk agents receive less scrutiny. Compliance audits focus on the locations deemed riskiest by empirical data to ensure that compliance and financial controls are effective. Data analytics software can supplement and enhance regularly scheduled compliance audits for risky locations. In between audits, this software can scan company records for indicia of fraud in real time. This risk-based due diligence using empirical data has allowed compliance departments to better manage risk and allocate resources efficiently.

Law firms are also collecting data to help their clients address compliance risk. Baker & McKenzie LLP partnered with Ethisphere to provide compliance lawyers access to Fortune 500 analytical compliance data and corporate criminal settlements. These databases provide lawyers with a way to empirically analyze compliance problems. In-house lawyers can analyze what peer companies are doing about their gift and entertainment policies, or facilitation payments. By analyzing empirical data, companies can make informed choices about their policies based on company and industry risk. Analyzing corporate settlements provides companies with insight about compliance changes the U.S. Department of Justice has required to resolve a criminal investigation.

Moneyball revealed the way data analytics has changed baseball—and the impact on in-house legal departments is changing the way companies address risk. Data analytic tools allow in-house lawyers the ability to gauge the effectiveness of their compliance program using real-time information. Technology and data analysis will never replace a compliance lawyer’s good judgment and awareness of a business’s culture and operating environment. But as technology provides new metrics for companies to address risk, companies not gathering and analyzing Moneyball-like data will find it difficult to maintain an effective compliance program.

Ryan McConnell is a partner at Baker & McKenzie LLP in Houston and a former federal prosecutor. Daniel Trujillo is deputy general counsel and director of compliance at Schlumberger, Ltd. Katelyn Richardson is a third-year law student at the University of Houston Law Center.