The Fox-y Science of Risk-Based Compliance Forecasting
Last November, the U.S. Department of Justice and Securities and Exchange Commission released their much-anticipated guidance [PDF] on the U.S. Foreign Corrupt Practices Act. Chapter 5 discussed compliance extensively and focused on a risk-based approach. The guidance notes that, “DOJ and SEC will give meaningful credit to a company that implements in good faith a comprehensive, risk-based compliance program, even if that program does not prevent an infraction in a low risk area because greater attention and resources had been devoted to a higher-risk area.”
To implement a risk-based compliance program, companies have to identify activities that pose greater risk and allocate resources to mitigate the risk. This involves predicting the future based on the past or forecasting. For corruption, it includes analyzing potential corruption touch points in a given market, and then making a forecast about the potential for a compliance failure. A risk-based program would devote more compliance resources (for example, audit, training, and personnel) to that market.
Dr. Philip Tetlock is an expert on forecasting. In 2005, he wrote a book called Expert Political Judgment: How Good Is It? How Can We Know? which shows readers that "what experts think is much less important than how they think." To figure out what makes a good forecaster, Tetlock puts experts into two categories—foxes and hedgehogs—from the title of an Isaiah Berlin essay on Russian novelist Leo Tolstoy: "The Hedgehog and the Fox." Foxes and hedgehogs think differently, and how they think guides their predictions. Tetlock points out that the fox knows "many little things," considers varying approaches, is skeptical of deductive approaches to prediction, and accepts contradictions and uncertainty. The hedgehog, on the other hand, knows "one big thing" and looks for patterns or rules to support this view.
Tetlock found that foxes make better forecasters because they were more willing to consider opposing views and different information and more reluctant to assign a subjective probability of an event occurring. His analogy of the fox versus the hedgehog has stimulated discussion on how to improve forecasting everywhere from Nate Silver’s recent book The Signal and the Noise to a recent New York Times column by David Brooks.
Brooks’s column discusses how Tetlock’s work led to a formal forecasting tournament at the Intelligence Advanced Research Projects Agency. The agency invests in high-risk, high-payoff research programs that have the potential to provide the U.S. with an overwhelming intelligence advantage over future adversaries. Brooks notes that five teams entered, from places like M.I.T., University of Michigan, and University of Maryland—and that a team put together by Tetlock and his wife, the decision scientist Barbara Mellers, destroyed the competition the first two years of the tournament and surpassed the benchmarks by 60 percent. Tetlock had a team of super foxes that could turn hunches into probabilities and used an algorithm to weight the best performers.
Tetlock points out that most of us are fox-hedgehog hybrids. But by understanding how we make decisions and why we should change our mind with new information, we can become better forecasters. Tetlock will speak at the University of Houston’s annual Ethics and Compliance Symposium on June 6. He will provide some basic lessons on forecasting and how compliance professionals can think like foxes. And thinking like a fox is a key ingredient to a risk-based compliance program and efficient allocation a resources to mitigate compliance risk.
Ryan McConnell is a partner at Morgan Lewis in Houston and former federal prosecutor. He teaches criminal procedure and corporate compliance at the University of Houston Law Center. He is a fox that plays a hedgehog on the Internet. Send your ideas of how to create a risk-based approach to compliance to him at firstname.lastname@example.org.