Font Size:

Compliance

What Computer Models Can—and Can't—Do

They can compile and sift compliance data, but humans need to look for patterns.

By Ryan McConnell, Dianne Ralston, and Charlotte Simon All Articles 

Corporate Counsel

January 1, 2013

On November 14, a week before the usual Black Friday shopping madness, the U.S. Department of Justice and the Securities and Exchange Commission gave companies a free gift by releasing long-awaited guidance on the U.S. Foreign Corrupt Practices Act. The guidance repackaged a number of the enforcement actions, opinion releases, and other source materials on the FCPA. The guidance also warned companies that compliance programs addressing the FCPA should focus resources on the areas that pose the greatest risk. The guidance states that "assessment of risk is fundamental to developing a strong compliance program" and explains that one-size-fits-all programs "no longer measure up to the DOJ's standard."

The concept is simple and intuitive. Execution, however, is difficult. A "risk-based" approach depends on using empirical data to address compliance risk. How to use this data to discern trends and risks has been a significant challenge for compliance lawyers. Nate Silver, in his recent book, The Signal and the Noise—Why So Many Predictions Fail But Some Don't (Penguin Press HC, 2012), discusses sorting through empirical data and identifying signals that enable better decision making. And in his wildly successful book (and later movie) Moneyball , Michael Lewis chronicled how statistical modeling led to better decision making in baseball roster management.

This got us thinking—can techniques developed in fields as diverse as weather forecasting and baseball management be applied to developing a risk-based compliance program?

As Silver notes, weather forecasting has improved remarkably over the past few decades, with sophisticated computer systems and mountains of historical weather data. East Coast residents had almost a week's notice before superstorm Sandy struck in late October—roughly the same advance notice forecasters provided for the path of Hurricane Katrina. Both predictions caused a number of residents to evacuate, but others ignored the forecasts.

Twenty or 30 years ago, the type of advance warning given for Katrina or Sandy was impossible. But current computer models and empirical data have created remarkably reliable weather forecasting models—models that use past data to make fairly reliable predictions of future outcomes. Similarly, the Oakland Athletics' remarkable 20 straight wins in the 2002 season was the result of statistical analysis of player performance to set the roster for the upcoming game. Importantly, Silver points out that weather forecasting models, even with the benefit of decades of data and some of the most powerful computers in the world, are imperfect because even the most trivial bug or data distortion in a model can have profound effects.

So how do compliance lawyers sort through the noise to create a valuable risk-based program? This risk-based approach could, for instance, focus FCPA training efforts on job descriptions and locations where the individuals trained have some duty that implicates the FCPA, such as finance, sales/marketing, operations, or supply chain. But how do you identify which jobs and which regions pose the greatest risk?

First, Silver counsels that the analysis is only as good as the integrity of the data—how large is the sample, and how reliable is the information? In our training example, how accurate are the job code descriptions? How do you decide which jobs pose the most compliance risk?

Evaluating calls to a compliance department's ethics hotline highlights other sampling issues. Evaluating the number or frequency of ethics line calls may speak to specific subject-matter risk, but what if the company has only recently begun to capture this information, or if it excludes certain geographic markets, or if it does not have a reliable way to sort and track the data (for instance, by weighting severity and type of issues)? You will only get half the story.

Second, Silver cautions against overreliance on empirical data—particularly where you do not have a long history of data collection and analysis. As Silver points out, the problem with computers developing weather models is that they can't see. They don't know that a fog can clear depending upon the direction the wind blows.

Overreliance on data to develop compliance models creates the same problem. Technology companies have developed a variety of solutions to track compliance-sensitive transactions and relationships.

A browser or device that allows javascript is required to view this content.

Subscribe to Corporate Counsel

TOP JOBS