By Jose Tabuena, JD, CFE, CHC
Douglas W. Hubbard, who developed Applied Information Economics as a practical application of scientific and mathematical methods to complex decision making, goes out further on a limb when it comes to measurement. According to Mr. Hubbard:
“Anything can be measured. If something can be observed in any way at all it lends itself to some type of measurement method. No matter how ‘fuzzy’ the measurement is, it’s still a measurement if it tells you more than you knew before.”
In the business world this has a ring of truth. Hubbard made a career out of measuring the sorts of things many thought were immeasurable. He writes of being surprised at how often clients dismissed a critical quantity—something that would affect a major new investment or policy decision—as completely beyond measurement.
For the auditor, compliance professional, and others charged with evaluating (i.e., measuring) the effectiveness and value of compliance program activities, Hubbard’s treatise, How to Measure Anything: Finding the Value of Intangibles in Business,3rd Edition, is a worthy read. His text includes an accompanying website that provides practical examples worked out in detailed spreadsheets. The book discusses how to measure those things in your business that until now you may have considered “immeasurable,” including technology ROI, organizational flexibility, customer satisfaction, technology risk, and even techniques that can be applied to compliance program effectiveness.
His observation that even intangible things can be measured (though he is not claiming they should) is a profound one. Hubbard lucidly explains why things that may seem immeasurable are actually not. He includes inspirational examples of where seemingly impossible measurements were resolved with surprisingly simple methods—for example, how in ancient Greece, a man was able to reasonably estimate the circumference of the Earth by looking at the lengths of shadows in different cities and then applying basic geometry.
There are also key points beyond just measuring things that should resonate with those tackling the compliance effectiveness quandary. A salient reason we should care about measurement is that it can inform important decisions. How much should we budget to enhance our compliance program? What features of our program should be modified? Moreover, he explains why you likely already have more information than you realize and that you don’t need as much data as you think.
Decision makers usually have imperfect information (i.e., uncertainty) about the best choice for a decision. Hubbard insists that decisions should be modeled quantitatively because such models have a favorable track record when compared to unaided (i.e., I know it when I see it) and/or purely qualitative “expert” judgment. Ultimately good measurement informs uncertain decision making. And Hubbard provides a comprehensive framework for measurement that can be applied universally to a host of business issues. The crux of the approach is that if you measure what matters, you make better decisions.
A thought experiment to try, which Hubbard calls a “clarification chain” is to imagine “if we didn’t do this, would there be an impact, and how would we notice the difference?”
Undertaking this methodology forces clarity in considering the objectives you are trying to achieve. When computing the value of information, you may learn that you have been measuring all the wrong things. If your “program” is providing a service the value of which cannot easily be measured, maybe you need to reconsider what you are trying to achieve. Some kind of observable consequence must be present if it really matters (even if dictated by laws and regulations). Measuring things just because they are easy to measure is ultimately useless.
A thought experiment to try, which Hubbard calls a “clarification chain” is to imagine “if we didn’t do this, would there be an impact, and how would we notice the difference?” For example, a safe work environment has been shown to relate directly to safe employee behavior; similarly, a climate for customer service is known to predict customer satisfaction. For compliance programs, if we care about an “intangible” that we call culture or ethical climate, because it impacts certain things—such as perceptions that your supervisor and company sets a good example of ethical behavior, or that employees do not fear retaliation for reporting misconduct—we should be able to measure such outcomes.
Direct application of these ideas for measuring intangibles can prove relevant with what seems to be a new approach by regulators to oversee culture. The Financial Industry Regulatory Authority 2016 Regulatory and Examination Priorities Targeted Exam letter notably focused on culture. FINRA stated plans to assess five indicators of a firm’s culture: (i) whether control functions are valued within the firm; (ii) whether policy or control breaches are tolerated; (iii) whether the company proactively seeks to identify risk and compliance events; (iv) whether immediate managers are effective role models of firm culture; and (v) whether sub-cultures that may not conform to overall corporate culture are identified and addressed.
As described in its February 2016 Targeted Exam Letter, FINRA requested firms submit eight categories of information related to the organization’s cultural values, stating “We will formalize our assessment of firm culture to better understand how culture affects a firm’s compliance and risk management practices.” Significantly, FINRA is, “particularly interested in how your firm measures compliance with its cultural values, what metrics, if any, are used, and how you monitor for implementation and consistent application of those values throughout your organization.”
The industry is still waiting for the results of FINRA’s review and observations from this information. It will be most interesting to learn how FINRA makes use of the collected information, and how it assesses whether cultural values are actually guiding business conduct or acting more as a feelgood exercise. This now makes an opportune time for FINRA, industry auditors, and compliance professionals to lay the groundwork for objective, risk-based analysis that can provide a data-driven assessment of culture’s efficacy.
The compliance profession for some time has advocated for more stringent methodologies and measurements to address the major issue of compliance program effectiveness—a significant measure as it not only can determine whether a company faces indictment or the amount of a sentencing penalty, but more practically on the question of whether the program is having any impact at all.
A 2012 report by the Ethics Resource Center (now the Ethics & Compliance Initiative), observed that “the means by which organizations measure the effectiveness of their programs still vary, and in some cases organizations can be lulled into a false sense of security by evaluations or public rankings that may not be empirically based or reliable.” The report encouraged discussion and analysis, including consideration of possible outcome measures by which firms could demonstrate the impact of their programs (e.g., observed misconduct, frequency and nature of reporting, fear of retaliation, direct measurement in risk areas where this is possible).
It is certainly a positive sign that along with FINRA, the Justice Department has brought the measurement challenge to the forefront in hiring a compliance counsel and issuing possible “metrics.” The trend of the government to provide more guidance has continued with the DoJ stating its plans to release a set of sample questions to give companies an idea what investigators and prosecutors are concerned with.
The evaluation of culture and compliance effectiveness are in fact empirical issues. The elements of a compliance program and vague indicators should not be taken on faith. Whenever practical, tactics based on studies by social scientists should be field-tested using randomized controlled trials to estimate their economic benefits.
Perhaps it is now a brave new world with law enforcement and regulators, working closely with compliance professionals (including auditors and lawyers) to add meaningful rigor to measuring the intangibles of culture and program effectiveness? Eventually it may result in numbers and not just adjectives and color codes to reveal what is working.
Originally published in Compliance Week
The Center for Board Excellence is offering for free download 
About one year after revelations surfaced that Volkswagen AG rigged its diesel cars to cheat emissions tests, it has somehow emerged as “the world’s biggest automaker” in the first six months of 2016, outselling Toyota during that period, according to a 
The National Association of Corporate Directors has made available a free 
eSignLive by Vasco has made available for downloading a new white paper that discusses the best security practices for implementing e-signatures and evaluating vendors. (See the download form below.)
London-based venture firm 
eSignLive by Vasco
In recent months, a proliferation of ransomware attacks has affected everyone from personal-computer and smart-phone owners to hospitals and police departments, 
Exterro