Can Computer Models Form Risk Compliance Programs?

Ryan McConnell, Dianne Ralston, and Charlotte Simon

Corporate Counsel

01-16-2013

On November 14, a week before the usual Black Friday shopping madness, the U.S. Department of Justice and the Securities and Exchange Commission gave companies a free gift by releasing long-awaited guidance on the U.S. Foreign Corrupt Practices Act. The guidance repackaged a number of the enforcement actions, opinion releases, and other source materials on the FCPA. The guidance also warned companies that compliance programs addressing the FCPA should focus resources on the areas that pose the greatest risk. The guidance states that "assessment of risk is fundamental to developing a strong compliance program" and explains that one-size-fits-all programs "no longer measure up to the DOJ's standard."

The concept is simple and intuitive. Execution, however, is difficult. A "risk-based" approach depends on using empirical data to address compliance risk. How to use this data to discern trends and risks has been a significant challenge for compliance lawyers. Nate Silver, in his recent book, The Signal and the Noise — Why So Many Predictions Fail But Some Don't (Penguin Press HC, 2012), discusses sorting through empirical data and identifying signals that enable better decision making. And in his wildly successful book (and later movie) Moneyball, Michael Lewis chronicled how statistical modeling led to better decision making in baseball roster management.

This got us thinking — can techniques developed in fields as diverse as weather forecasting and baseball management be applied to developing a risk-based compliance program?

As Silver notes, weather forecasting has improved remarkably over the past few decades, with sophisticated computer systems and mountains of historical weather data. East Coast residents had almost a week's notice before superstorm Sandy struck in late October — roughly the same advance notice forecasters provided for the path of Hurricane Katrina. Both predictions caused a number of residents to evacuate, but others ignored the forecasts.

Twenty or 30 years ago, the type of advance warning given for Katrina or Sandy was impossible. But current computer models and empirical data have created remarkably reliable weather forecasting models — models that use past data to make fairly reliable predictions of future outcomes. Similarly, the Oakland Athletics' remarkable 20 straight wins in the 2002 season was the result of statistical analysis of player performance to set the roster for the upcoming game. Importantly, Silver points out that weather forecasting models, even with the benefit of decades of data and some of the most powerful computers in the world, are imperfect because even the most trivial bug or data distortion in a model can have profound effects.

So how do compliance lawyers sort through the noise to create a valuable risk-based program? This risk-based approach could, for instance, focus FCPA training efforts on job descriptions and locations where the individuals trained have some duty that implicates the FCPA, such as finance, sales/marketing, operations, or supply chain. But how do you identify which jobs and which regions pose the greatest risk?

First, Silver counsels that the analysis is only as good as the integrity of the data — how large is the sample, and how reliable is the information? In our training example, how accurate are the job code descriptions? How do you decide which jobs pose the most compliance risk?

Evaluating calls to a compliance department's ethics hotline highlights other sampling issues. Evaluating the number or frequency of ethics line calls may speak to specific subject-matter risk, but what if the company has only recently begun to capture this information, or if it excludes certain geographic markets, or if it does not have a reliable way to sort and track the data (for instance, by weighting severity and type of issues)? You will only get half the story.

Second, Silver cautions against overreliance on empirical data — particularly where you do not have a long history of data collection and analysis. As Silver points out, the problem with computers developing weather models is that they can't see. They don't know that a fog can clear depending upon the direction the wind blows.

Overreliance on data to develop compliance models creates the same problem. Technology companies have developed a variety of solutions to track compliance-sensitive transactions and relationships.

But the best these tools can do is help compliance lawyers make more informed decisions. For instance, a tool to track gifts may tell you that an operating unit took an official from a state-owned company to dinner and spent an amount exceeding the limit set by the company policy. The tool will never tell you, however, whether the dinner was improper entertainment because it cannot reveal the intent of the employee in organizing the dinner or what was discussed there.

Another problem with technology, as Silver points out, is that computers are so literal-minded that they are unable to recognize patterns when subjected to even the slightest degree of manipulation — even changing a few letters can throw them off. Not so with humans, who can "rapidly parse through any distortions in the data in order to identify abstract qualities." Silver concludes in his chapter on baseball statistical analysis that even sporting organizations that are leaders in statistical analysis, such as the Oakland Athletics, rely heavily on scouts to analyze the human factor.

A technology tool cannot promote a culture of integrity, or show employees how good ethics makes the business stronger. That comes from people who believe in the organization and understand how to use risk data to promote the compliance program and its messaging.

The new FCPA guidance notes that taking a risk-based approach is particularly critical with respect to due diligence procedures for assessing third-party relationships. Using empirical data to evaluate third-party risk may be even more challenging depending upon the reliability of the data obtained from the third party to assess risk and, for initial due diligence purposes, the lack of historical data about the third party. Risk-based systems address this lack of historical data with more stringent due diligence based on the risk of the third party. A sales agent who works on commission in a country with historically high levels of corruption should receive more scrutiny than a visa agent who will be paid according to a publicly available fee schedule in a country with historically lower levels of corruption.

The longer a company operates in a particular environment with different third parties, the better it should get at evaluating the risks of entering into those relationships. The sample set will get larger and the risk analysis better.

The Justice Department and SEC guidance makes clear that companies have to assess risk and adopt a "risk-based" approach. Eventually, this may mean that companies spend less on compliance as their programs get more efficient at addressing risk. Risk models will change as compliance programs become more efficient and gather more empirical data. The success of a risk-based model, however, will ultimately depend not on technology tools, but on the compliance lawyer's ability to successfully analyze risk data and sort the signals from the noise. That lawyer must be adaptive, creative, and look beyond the data to see organizational and industry trends and risks. By helping us understand the limits of technology and how to use data, Nate Silver can make us all better compliance lawyers.

Ryan McConnell is a partner at Morgan, Lewis & Bockius in Houston and a former federal prosecutor. Dianne Ralston is deputy general counsel at Schlumberger Ltd., where she focuses on mergers and acquisitions. Charlotte Simon is an associate at Baker & McKenzie in Houston and former law clerk to U.S. District Judge Keith Ellison in Houston.