Meaningful Metrics:

Using ERM to Inform Strategy

+1 617 530 1210 | logicmanager.com | info@logicmanager.com ©LogicManager, Inc.

1
Introduction
Risk and compliance professionals face unparalleled
levels of scrutiny today as a result of the See-Through
Economy, an age where social media and new
technology have empowered stakeholders to impact
a company’s reputation. Not only are governance
professionals responsible for protecting their
organization, but they also need to provide evidence
that their programs are effective at managing risk. At
the very minimum they must display they’re meeting
the expectations of examiners, regulators, and their own
board of directors.

We all collect metrics, but many risk managers aren’t collecting the information they need and aren’t
allocating responsibility for information-gathering and communication. This leaves risk teams drowning
in data and unable to easily sift through the noise. Often what holds risk managers back from meaningful
metric-collection is unavailable data that’s expensive to gather, uncertainty in who is responsible for
collection and reporting, and an unequal focus on lagging and leading indicators.

ERM programs need metrics. Without them, it’s highly unlikely that the value of a company’s ERM program, or
the degree to which previously unidentified risks have been mitigated, can be demonstrated.

Table of Contents
2 Defining and Identifying Useful Metrics
5 Contextualizing Metrics
11 Presenting Metrics
14 Case Studies

1
Chapter 1
Defining and Identifying
Useful Metrics
First, let’s explore some common metrics for examining and
ultimately presenting on the effectiveness of your ERM program.

2
Leading and Lagging Indicators

Leading and lagging indicators are great for measuring the effects of your company’s efforts, as well as
predicting upcoming events. Rather than focusing on one or the other, these indicators are best when
measured equally.

Leading Indicators
Leading indicators are preventative measures used to predict potential risks rather than react to them. A best-
practice leading indicator points towards the root cause of a risk, as it takes into account the effectiveness of
control activities focused on preventing the risk from occurring.

Example
A leading indicator could be a customer complaint or a customer satisfaction survey. This
feedback is then the catalyst for bringing about preventative activities to ensure a customer
does not cancel their service.

Lagging Indicators
On the other hand, a lagging indicator follows or reacts to an event, as it essentially measures what already
went right or wrong. Lagging indicators are the result of previous performance and ultimately confirm that a
trend is either occurring or is about to occur.

Example
A lagging indicator within a risk assessment could be the number of customers who have
canceled their service since the deployment of a new customer service policy. By recognizing
this trend in the data, you could then revisit the policy to see what aspects of it caused
customers to cancel.

By taking both lagging and leading indicators into account, an organization is creating a full picture of a risk,
from the controls taken to avoid the risk, to the consequences that would affect the organization if the risk
occurred.

3
KRIs and KPIs

Most frequently, the metrics used to evaluate components of a business are classified as either Key Risk
Indicators (KRIs) or Key Performance Indicators (KPIs).

What’s the difference?

Key Risk Indicators (KRIs)

KRIs are focused on an organization’s risk activities and
measure the level of risk a company is exposed to. KRIs tend to
be leading indicators that often act as warning signs and predict
future outcomes.

Key Performance Indicators (KPIs)

While KRIs are looking towards the future, KPIs look to the past
to analyze the outcomes and trends of previous performances.
Because KPIs look at results that have already happened, they
tend to be lagging indicators.

Both KRIs and KPIs are measurable, comparable, and reportable. They are quantifiable ways to measure
the upsides and downsides of risk, and they should both be utilized when considering any organizational
decision or venture.

Analyzing KRIs and KPIs over an extended period of time will show actual and/or probable deviations from
a given standard or goal. By utilizing these risk metrics, companies can improve their understanding of just
how likely achieving their next strategic objective is.

4
Chapter 2
Contextualizing Metrics
When it comes to measuring metrics, think quality over quantity.
An effective metric provides a combination of both insight and
value. While it may seem smart to monitor as much information
as possible, this approach has many downsides, including an
overload of information and difficulty prioritizing next steps.

Additionally, it’s important to have a tangible context for metrics. A

metric should reflect and measure an organization’s strategic goals
in order to tell a meaningful story.

5
Design Effective Risk Assessments

When metrics are collected in individual silos, they’re

disconnected from the bigger picture and inaccessible
to senior-level decision makers. The best way to
prioritize metrics is with enterprise risk assessments.

As a foundation, risk managers should be assessing

risks on a common scale and criteria to ensure their
assessments are comparable across departments. This
information enables risk managers to understand their
most critical risks and the corresponding mitigation
activities. An additional assessment of the mitigation
activities governing a particular risk, often called
assurance, allows process owners to understand the
effectiveness of a control.

Having more numerical options for employees to choose from when assessing risk is a good start, but
in actuality, even a 1-10 scale can present opportunities for miscommunication. Therefore, it’s equally
important to standardize your evaluation criteria, or in other words, provide guidelines for what makes a 7 a
7, as opposed to a 9.

7-8 9 - 10
Serious Major

• Financial: Negative impact on net income – • Financial: Negative impact on net income –
$15 million to $20 million over $20 million
• Financial: Alternative financing (debt), sale • Financial: Catastrophic impact on financial
or restructuring of the organization could be statements (e.g., critical contractual ratios
required are no longer met)
• Operational: Inability to remain competitive • Operational: Long-term impairment of
(e.g., lagging customer service, operational critical functions make the organization
inefficiencies) vulnerable to forced sale of merger
• Regulatory: Regulatory penalties are • Regulatory: Regulatory agencies seize
required control of assets or are granted absolute
decision-making authority

6
Risks Identified vs. Risks Mitigated

It’s important for risk professionals to ensure controls are linked to the most critical risks. Creating these links
will help identify gaps in the control environment and point to where key risks are left unmitigated.

The number of critical risks without corresponding controls is often one measure of how proactively risks are
governed. To identify this KRI, sort your risk universe to only those risks that exceed a predetermined level or
tolerance. Then determine the percentage of remaining risks that are not linked to at least one control.
GAP

Total Number Mitigated

Q1 Q2 Q3

While this can be a helpful tool in identifying gaps in your control environment, this metric is not an accurate
indicator of how effectively these risks are managed. To measure effectiveness, risk monitoring must be
implemented.

7
Implement Monitoring Activities

The mere existence of controls is not sufficient to demonstrate a risk is managed effectively, and is not
indicative of a mature risk management program. Regulators, examiners, and the board are instead pushing
for evidence of control effectiveness, which is best accomplished by instituting regular, recurring monitoring
activities that indicate if your controls are actually working.

Consider the prior example of the percentage of risks with attached controls. A risk manager could rightfully
report that they have controls in place for X% of risks. While that may seem satisfactory on the surface, the
sole existence of those controls provides no indication of how effectively they manage the risks.

If your organization is concerned with the risk of inappropriate sales practices by its sales force, it may
implement a mitigation plan that involves additional training.

To examine the effectiveness of this control, one metric that could be collected is the percentage of
employees that take part in training. If the training was unsuccessful, the risk team could look at this metric
and determine whether the inefficacy stems from lack of participation or an ineffective training program.

A combination of monitoring activities and metrics is the crucial third piece to the risk assessment process.
This combination reveals the effectiveness or ineffectiveness of a risk management program, allowing
companies to allocate their limited resources in more efficient ways. It is also the foundation of quantifiable,
reliable, and reportable risk metrics, and can reveal patterns that otherwise would not have been visible to
risk managers.

Example
You want to ensure you have the appropriate number of staff and are retaining
valuable workers.

Consider applying these monitoring steps:

1. Administer recurring assessments to department managers to collect information on whether
they have enough staff that is also providing value
2. Create a report comparing reported staffing needs and current staff level and value
3. Communicate these findings with People Operations to determine where gaps can be filled and
more value added

8
Lines of Communication
When determining what metrics to collect for your organization, it is important to consult both the board of
directors and senior management. Discussing your organization’s strategic plan will ensure that the risks and
controls being monitored are aligned with your company’s goals. Metrics that are identified at the front lines
of the business must roll-up to support key strategic objectives outlined by senior management.

Linking strategic goals to risks with activities and metrics provides context, and enables effective
communication that empowers front-line employees to support an organization’s strategic plan.

Executive
Management,
Board, Regulators,
Rating Agencies

Department Level

Business Process/
Activity Level

9
Connecting Metrics to Strategic Objectives
The final step is to contextualize KRIs and KPIs with appropriate performance thresholds. This process is
often referred to as creating risk tolerance statements.

Risk tolerances set specific limits on accepted levels of variation and can help an organization achieve their
business objectives. They serve as quantitative statements that enable risk managers to determine if a metric
falls within a range of outcomes that management has deemed acceptable.

Performance outside of the determined range, or outside a set risk tolerance, indicates a failed or over-
performing process that needs to be examined.

Communicating these expectations across a company allows departments and front-line employees to act as
a cohesive unit in advancing towards organizational goals.

10
Chapter 3
Presenting Metrics
Now that you have an idea of what kinds of information you
should be collecting and how to provide some context to those
metrics, let’s talk about how to present your metrics. The key is to
think “less is more” and concentrate on showing information that
resonates with the audience’s priorities.

11
Presenting the Right Metrics
First, it’s always good to narrow the scope of your metrics. You never want to walk into a meeting with a
4-inch binder full of all the information you’ve collected on a particular topic. Metrics should be designed and
reported on for the highest assessed risks, as they pertain to the achievement of your audience’s strategic
objectives.

For risk managers, simply presenting the number of risks is not enough. It is better to report the percentage of
risks above the average risk score that have controls in place. This is an important step in describing the state
of the company’s enterprise risk management program.

Collecting risk information from individuals on the front lines across departments ensures that an
organization is utilizing the most comprehensive information possible.

This information can then be used to generate accurate reports and becomes an important part of change
management by keeping everyone informed and prepared for the risks that threaten key goals.

The Risk Heat Map: Display Your Organization’s Most Critical Risks

12
Presenting to the Board
Senior executives and board members are held accountable for incidents throughout their organizations,
which has increased their awareness and involvement in enterprise risk management. This is driven by
reputational risk spurred by the See-Through Economy, regulations from the SEC and other regulatory
bodies, as well as legislation from Congress, which requires increased involvement and accountability for risk
management.

Boards of directors previously were only responsible for CEO-level activities and decisions. But now, the
accountability is extended down to the threshold of the material impact of the risk, regardless of level. Risk
now needs to be identified at the business process level, where this material activity takes place.

Boards are given a choice between having effective risk management or publicly disclosing their
ineffectiveness in risk management. If they do neither, it is considered fraud or negligence, as lack of
knowledge about a risk is no longer a defense.

It is the risk manager’s responsibility to keep the board informed about the company’s risk health by
providing regular reports on the effectiveness of risk management. These reports should address the various
risks facing the company and those that may impede the company from achieving its goals.

Furthermore, quantitative risk metrics are a good way to showcase the risk manager’s efforts and purpose
within the organization’s risk management program, and how they tie into the greater strategic company
goals.

Thus, for the sake of company health and legitimate board accountability, it is critical that risk managers are
presenting relevant and useful risk metrics.

Present ERM to the Board

Download our eBook for actionable tips on presenting the
effectiveness of your ERM program and getting buy-in from
the board.

13
Chapter 4
Case Studies of Failures
in Risk Management
Why is it so important to collect, analyze, and present on risk
metrics? Besides not being able to prove compliance or the
effectiveness of your risk management program, failure to collect
meaningful metrics can lead to a failure in risk management, which
so often leads to corporate scandal, poor reputation, and public
distrust.

14
Wells Fargo Case Study
Millions of Wells Fargo bank accounts were set up without customer consent,
generating overdraft charges and other fees. Wells Fargo paid $185 million
in penalties for inappropriate sales practices – the highest fine levied by the
Consumer Financial Protection Bureau.

Although the scandal was partially due to widespread cultural failure and fraud,
the crux of the problem was a failure in risk management for these three key
reasons:

1. Employees were incentivized by unrealistic and unattainably high sales quotas.

2. There were no risk assessments done on potential complications related to increased sales
quotas.

3. There was no enterprise risk management system to monitor and escalate risks to
appropriate levels.

Boards are now required by the SEC to report on how their organizations identify risk, set risk tolerances, and
manage risk/reward trade-offs. It’s important to note that enterprise risk management is not only for financial
institutions, but is a modern requirement for all organizations across all industries, public and private.

The Metrics Wells Fargo Needed

Consider the metrics Wells Fargo could have collected to minimize the scandal:

• Number of customer complaints related to charges of fraudulent accounts they did not
authorize.
• Percent of canceled accounts and services over a period of time.
• Percent of fraudulent customer email accounts.

By tracking the number of complaints related to unauthorized accounts, trends could have been identified
at specific branch locations. If certain branches consistently showed high number of canceled services, this
could have been flagged as a potential risk. Closer examination of outliers can reveal opportunities to cross-
train sales representatives and improve cross-selling tactics.

15
Chipotle Case Study
Chipotle introduced a great food industry innovation: fresh, locally-sourced fast food.
However, Chipotle failed to implement the risk management necessary to support
that innovation. ERM and effective metrics are as much about enabling innovation as
they are facilitating compliance, health, and safety. For every innovation, companies
must find the unique risks it introduces, get them covered, and disclose them to
shareholders.

Chipotle experienced widespread salmonella outbreaks which were linked to food

management issues within their supply chain and kitchen preparation techniques. Chipotle also failed to
disclose the inadequacy of their quality controls, which in turn threatened consumer and employee health.
This two-pronged failure in risk management led to shareholders being misled, stock prices dropping, long-
term reputational damage, and a shareholder class-action lawsuit.

The Metrics Chipotle Needed

Consider the indicators Chipotle could have used to manage risks and improve business efficiency:

• Percent of local vendors who underwent Chipotle screening process and training, to evidence
proactive vendor management due diligence
• Frequency of assessments - conducted at all locations - on internal policies and procedures for
vendor materials inspection and food preparation procedures
• Percent of managers who successfully complied with corporate policies and common
regulations
• Percent of incidents by region, where low incident rates equals good vendor management

An ERM system with central governance of these incidents would have clearly shown the systemic nature of
these incidents and should have pointed to inadequate monitoring of internal processes, procedures, and
controls. Furthermore, consider the lagging indicators that Chipotle could have used to manage their root-
cause risks while improving business efficiency:
• Number of employees who underwent proper food training
• Percent of locations that passed monthly health safety checks
• Average scores on customer satisfaction reviews for each branch, which would have gathered
risk and performance information on the front lines across all of their locations
• Frequency and severity of incident reports, aggregated in an enterprise risk management
software across all restaurant locations

16
Proper Risk Management
What would have happened if Chipotle had a proper risk management program? Vendor management
and operations would have been more efficient, which means their food would likely never have been
contaminated. What if Wells Fargo had been collecting the metrics we mentioned? They would have likely
realized they needed more rigid separation of duties in which sales reps did not have the ability to create new
accounts.

Even if outbreaks or fraud had occurred, Chipotle and Wells Fargo could have used ERM software’s reporting
capabilities to evidence its ERM program. This would have avoided many reputational and regulatory
punitive damages because Chipotle could have provided evidence of control activities and guided risk
disclosure.

It’s important to note that proper risk management is neither rare nor impossible. With risk management
protocols – and proof of those procedures – companies position themselves to avoid risk events and
regulatory penalties.

Incidents are bound to occur, but regulators and shareholders are conscious of this fact. What’s important is
preparing your company, to the best of your ability, to manage the chance of those surprises.

Get Your Risk Maturity Score

The Risk Maturity Model (RMM) is a free online assessment
tool that can benchmark the maturity of your ERM
program. Take the free RMM assessment today!

17
Collect Meaningful Metrics with LogicManager
Risk management is more than identifying and assigning controls for risk; it is a preventative and analytical tool that
all companies should be utilizing. From studying past incidents and establishing trends, to enacting strong security
measures to avoid incidents, metrics and monitoring play a central role in mature risk management programs. A
robust solution enables organizations to maintain full disclosure and avoid regulatory action.

Request a demonstration to see how LogicManager can help you communicate across departments, collect
actionable information, and report on your success.

REQUEST A DEMO

AUDIT BUSINESS COMPLIANCE

MANAGEMENT CONTINUITY & DR MANAGEMENT

INCIDENT ENTERPRISE RISK FINANCIAL

MANAGEMENT MANAGEMENT REPORTING (SOX, MAR)

POLICY VENDOR IT GOVERNANCE

MANAGEMENT MANAGEMENT & SECURITY

+1 617 530 1210 | logicmanager.com | info@logicmanager.com ©LogicManager, Inc.

18