**4.1. Step 1 – Ask questions**

IA OM® is like other investigative ventures – the first step in the process is determining what you want or need to know. This chapter focuses on evaluating and understanding the ongoing risk management activities within an organization. As a result, the questions framed for use with IA OM® in this chapter focus on what the organization's executives, program managers, or system-level managers want or need to know about the risk posture of the portion of the organization they are responsible for. As such, the questions to be addressed by IA OM® need not to be restricted to Tier 1 – they can, and ultimately should, be spread across all three tiers so that the managers at each tier have the answers they need to be successful in the organization's risk management program.

IA OM®

subcharacteristics for each area follow a similar numbering scheme where, for example, subcharacteristic11 is equivalent to da11 and so forth. This is done to simplify the

It is important to remember that the identification of organizational assets, processes, and operations are organization-specific. This is one of the strengths of the IA OM® process, since it enables the abstraction of these key activities – specifically identified by the organization as being of interest – into a metric that shows Senior Management where their organization stands with respect to its risk management strategy and policies. If low level technical metrics exist, they can be combined and abstracted into the IA OM® process. Metrics produced through the use of the IA OM® process can be deconstructed into their component areas, allowing Senior Management to identify the areas needing attention. If further improvement in their risk management program is required, the IA OM® and its component measures can be used to track and improve the organization's risk posture over

**Figure 3.** Example analysis of risk monitoring activities and their subcomponents

**4.3. Steps 4 and 5 – Define activity value scales and determine activity values** 

Activity value scales used for IA OM® activities are normally scoped to range between 0 and 1. This makes comparison easy and allows them to be aggregated and rolled-up into measures that are easy to use and understand. Also, these values can be readily seen as percentages to further enhance their understanding. However it is perfectly acceptable to have more important characteristics have values greater than 1 if desired to indicate their

abbreviations used in the equations in Steps 6 and 7.

time.

as an Enterprise Risk Management Metric 219

Examples of questions that might be asked at each Tier are presented in Table 1:


**Table 1.** Example questions by Tier

#### **4.2. Steps 2 and 3 – Identify assets, processes, or operations to measure**

Now that the questions to be answered have been identified, the next step is to identify the assets, processes, or operations that can be measured to obtain answers to those questions. For example, to determine the effectiveness of an organization's common control strategy, you could examine:


An example analysis of Tier 1 risk monitoring activities, their components, and their subcomponents is provided in Figure 3. The common controls assets are presented as Characteristicn in the right hand branch of the figure. These characteristics and subcharacteristics will also be referred to as diagnostic areas, where diagnostic area 1 (da1) is equivalent to characteristic1; da2 is equivalent to characteristic2; and so forth. The subcharacteristics for each area follow a similar numbering scheme where, for example, subcharacteristic11 is equivalent to da11 and so forth. This is done to simplify the abbreviations used in the equations in Steps 6 and 7.

218 Risk Management – Current Issues and Challenges

IA OM® is like other investigative ventures – the first step in the process is determining what you want or need to know. This chapter focuses on evaluating and understanding the ongoing risk management activities within an organization. As a result, the questions framed for use with IA OM® in this chapter focus on what the organization's executives, program managers, or system-level managers want or need to know about the risk posture of the portion of the organization they are responsible for. As such, the questions to be addressed by IA OM® need not to be restricted to Tier 1 – they can, and ultimately should, be spread across all three tiers so that the managers at each tier have the answers they need

to be successful in the organization's risk management program.

Examples of questions that might be asked at each Tier are presented in Table 1:

**Tier 1 Tier 2 Tier 3** 

How well do our mission/business processes align with our organization's risk management strategy?

How well does my mission program align with our organization's risk management strategy?

Which common controls are cost effective?

Now that the questions to be answered have been identified, the next step is to identify the assets, processes, or operations that can be measured to obtain answers to those questions. For example, to determine the effectiveness of an organization's common control strategy,

An example analysis of Tier 1 risk monitoring activities, their components, and their subcomponents is provided in Figure 3. The common controls assets are presented as Characteristicn in the right hand branch of the figure. These characteristics and subcharacteristics will also be referred to as diagnostic areas, where diagnostic area 1 (da1) is equivalent to characteristic1; da2 is equivalent to characteristic2; and so forth. The

**4.2. Steps 2 and 3 – Identify assets, processes, or operations to measure** 

 The common controls called for in the common controls strategy/policy; Their alignment with current organizational goals and objectives; and

Are our information systems properly operating within the risk tolerance of our organization's mission/business processes?

Is the risk posture of this system within established boundaries to support this mission?

How fully am I using the common controls available to me?

**4.1. Step 1 – Ask questions** 

Is our risk management strategy aligned with our organizational goals and objectives?

How well aligned is our risk management strategy with our mission and business programs?

Is our common control strategy effective?

**Table 1.** Example questions by Tier

you could examine:

Their utilization.

It is important to remember that the identification of organizational assets, processes, and operations are organization-specific. This is one of the strengths of the IA OM® process, since it enables the abstraction of these key activities – specifically identified by the organization as being of interest – into a metric that shows Senior Management where their organization stands with respect to its risk management strategy and policies. If low level technical metrics exist, they can be combined and abstracted into the IA OM® process. Metrics produced through the use of the IA OM® process can be deconstructed into their component areas, allowing Senior Management to identify the areas needing attention. If further improvement in their risk management program is required, the IA OM® and its component measures can be used to track and improve the organization's risk posture over time.

**Figure 3.** Example analysis of risk monitoring activities and their subcomponents

#### **4.3. Steps 4 and 5 – Define activity value scales and determine activity values**

Activity value scales used for IA OM® activities are normally scoped to range between 0 and 1. This makes comparison easy and allows them to be aggregated and rolled-up into measures that are easy to use and understand. Also, these values can be readily seen as percentages to further enhance their understanding. However it is perfectly acceptable to have more important characteristics have values greater than 1 if desired to indicate their

relative importance to the organization. Alternatively, the relative importance of different characteristics/subcharacteristics can be accounted for using the weighting factors discussed in Steps 6 and 7.

IA OM®

2 2

*ij ij*

*w da*

2 2

(max[ ])

*ij ij*

*w da*

1

*j*

For the common control example, using organizationally determined weightings and

Weightings Subcharacteristics/

w31 = 1 da31 = 0.42 w32 = 1 da32 = 0.94 w33 = 2 da33 = 0.61

diagnostic areas

2 22 2 2 2

√�

= .65

<sup>3</sup> 2222 22 (1 0.42 ) (1 0.94 ) (2 0.61 ) (1 1.0 ) (1 1.0 ) (2 1.0 )

*Characteristic*

Characteristic3 = √����

Inserting values for the other characteristics, and arranging all of the values into a fishbone diagram for clarity of presentation provides breakdown of the process as shown in Figure 5.

Calculating an IA OM® index provides a concise, high-level assessment of the enterprise risk management posture of the organization. The IA OM index, in this case referred to as

the IA risk management index, or IARMIndex, is defined in terms of:

*ni*

1

*j*

*<sup>i</sup> ni*

*da*

Equation 1 – Calculating diagnostic area (characteristic) values:

*ni*= number of diagnostic criteria for diagnostic area *dai*  wij = weighting factor for diagnostic daij of diagnostic area *dai daij* = the jth diagnostic criterion of the of the ith diagnostic area *dai*

subcharacteristics/diagnostic areas values provided in Table 2:

max [*daij*] = maximum value of *daij*

**Table 2.** Weightings and Subcharacteristics Values

**4.5. Step 7 – Calculate the IA OM® index** 

2. Diagnostic criteria for each diagnostic area

3. Diagnostic criterion value scales

The equation works out to:

1. Diagnostic areas

Thus:

Where:

as an Enterprise Risk Management Metric 221

For activities that can only assume a specific set of values (e.g., Yes/No, or high, moderate, and low), the value scales can be adapted to accommodate them. For example, with a binary set, like Yes and No, it is common to use Yes = 1, and No = 0. Sets like high, moderate, and low could be represented with high = 1, moderate = 0.5, and low = 0. It would also be possible to decide that high = 0.9 (since even high is not definite like "yes"), moderate = 0.5, and low = 0.1 (since low is also not definite like "no"). The decision on value scales is made by the organization and is made to maximize the utility and understandability of the measurements in the context of their organization.

Continuing with the example of the effectiveness of an organization's common controls strategy, value scales for each of the three subcomponents can be defined, and values determined for each of the subcharacteristics assigned as shown in Figure 4. Each subcharacteristic/diagnostic area (daij) will be evaluated separately and then combined in Step 6 to provide an overall value for each characteristic or top-level diagnostic area (dai). These values are then combined in Step 7 to provide an overall IA OM Index Value.

**Figure 4.** Common Control Value Scales Example

#### **4.4. Step 6 – Calculate an asset, process, or operation index**

Steps 3 through 5 have shown how to derive values for each of the assets, processes, and operations identified in Step 2. Each of these values represents a characteristic or subcharacteristic for an asset, process or operation. In this step, any values for subcharacteristics (daij) will be combined with weighting factors and aggregated to provide values for each characteristic (dai).

The weighting factor value (wij) for each subcharacteristic (daij) is assigned by organizational management to represent the organization's determination of the relative importance of each subcharacteristic to the characteristic being evaluated.

Equation 1 – Calculating diagnostic area (characteristic) values:

$$da\_i = \frac{\sqrt{\sum\_{j=1}^{ni} \pi v\_{ij}^2 da\_{ij}^2}}{\sqrt{\sum\_{j=1}^{ni} \pi v\_{ij}^2 (\max[da\_{ij}])^2}}$$

Where:

220 Risk Management – Current Issues and Challenges

measurements in the context of their organization.

**Figure 4.** Common Control Value Scales Example

values for each characteristic (dai).

**4.4. Step 6 – Calculate an asset, process, or operation index** 

each subcharacteristic to the characteristic being evaluated.

in Steps 6 and 7.

relative importance to the organization. Alternatively, the relative importance of different characteristics/subcharacteristics can be accounted for using the weighting factors discussed

For activities that can only assume a specific set of values (e.g., Yes/No, or high, moderate, and low), the value scales can be adapted to accommodate them. For example, with a binary set, like Yes and No, it is common to use Yes = 1, and No = 0. Sets like high, moderate, and low could be represented with high = 1, moderate = 0.5, and low = 0. It would also be possible to decide that high = 0.9 (since even high is not definite like "yes"), moderate = 0.5, and low = 0.1 (since low is also not definite like "no"). The decision on value scales is made by the organization and is made to maximize the utility and understandability of the

Continuing with the example of the effectiveness of an organization's common controls strategy, value scales for each of the three subcomponents can be defined, and values determined for each of the subcharacteristics assigned as shown in Figure 4. Each subcharacteristic/diagnostic area (daij) will be evaluated separately and then combined in Step 6 to provide an overall value for each characteristic or top-level diagnostic area (dai).

Steps 3 through 5 have shown how to derive values for each of the assets, processes, and operations identified in Step 2. Each of these values represents a characteristic or subcharacteristic for an asset, process or operation. In this step, any values for subcharacteristics (daij) will be combined with weighting factors and aggregated to provide

The weighting factor value (wij) for each subcharacteristic (daij) is assigned by organizational management to represent the organization's determination of the relative importance of

These values are then combined in Step 7 to provide an overall IA OM Index Value.

*ni*= number of diagnostic criteria for diagnostic area *dai*  wij = weighting factor for diagnostic daij of diagnostic area *dai daij* = the jth diagnostic criterion of the of the ith diagnostic area *dai* max [*daij*] = maximum value of *daij*

For the common control example, using organizationally determined weightings and subcharacteristics/diagnostic areas values provided in Table 2:


**Table 2.** Weightings and Subcharacteristics Values

The equation works out to:

$$\text{Characteristic}\_3 = \frac{\sqrt{(1^2 \times 0.42^2) + (1^2 \times 0.94^2) + (2^2 \times 0.61^2)}}{\sqrt{(1^2 \times 1.0^2) + (1^2 \times 1.0^2) + (2^2 \times 1.0^2)}}$$

Thus:

$$\text{Characteristic3} = \frac{\sqrt{2.55}}{\sqrt{6}} = .65$$

Inserting values for the other characteristics, and arranging all of the values into a fishbone diagram for clarity of presentation provides breakdown of the process as shown in Figure 5.

#### **4.5. Step 7 – Calculate the IA OM® index**

Calculating an IA OM® index provides a concise, high-level assessment of the enterprise risk management posture of the organization. The IA OM index, in this case referred to as the IA risk management index, or IARMIndex, is defined in terms of:


IA OM®

diagnostic areas

Weightings Characteristics/

w1 = 2 da1 = 0.25 w2 = 1 da2 = 0.25 w3 = 1 da3 = 0.65

The following steps describe the analysis process allowing organizational management to quickly uncover areas needing attention and prioritize which area(s) to address first to

1. The IARMIndex value is examined to determine whether it is within the range

2. If the IARMIndex value is within the risk tolerance of the organization, the current values are included in the trending information and no further action is required until

a. The individual component index values that were aggregated to derive the IARMIndex value are examined and the value(s) furthest from the normalized value of 1.0 (the

b. The selected component index value(s) is then unfolded (if applicable) to find the

c. The component(s)/subcomponent(s) with the lowest value(s) is analyzed and a method

d. At management's discretion, reassess and recalculate the characteristic(s) (and any applicable subcharacteristic(s)) value(s) for the component(s) that was addressed and

4. By looking at trends in the IARMIndex values, and the component index values compiled over time, the organization may determine the overall improvement resulting

5. By looking at the IA OM® mapping and measurement trends in the component values, leadership or management can see which areas have had the greatest improvement, and

Continuing with the example from above, the IARMIndex value = 0.35. Assuming the organization has determined that any value under 0.70 is outside of its risk tolerance, the next step is to determine which component(s) to single out for further analysis. Using the values in Table 4, we find that characteristics da1 and da2 have the lowest values. However, if we also consider the weightings we see that w1 = 1.0 indicates that it is a higher priority item to the organization than w2, suggesting that if we cannot address both areas, priority

determined by the organization to correspond to its risk tolerance;

3. If the IARMIndex value is not within the risk tolerance of the organization:

update the IARMIndex value to assess the impact of the changes made.

values closest to 0) is singled out for further analysis;

subcomponent(s) with the lowest value(s);

from addressing these components.

which areas are most in need of attention.

for improving it is identified and implemented;

**Table 3.** Weightings and Characteristics Values

Results in a value for **IARMIndex = 0.35**

**4.6. Evaluating the IA OM® index** 

the next review cycle is initiated.

obtain the greatest benefit:

as an Enterprise Risk Management Metric 223

**Figure 5.** Fishbone Diagram with All Risk Monitoring Characteristics and Subcharacteristics Values

All of these items are provided in the fishbone diagram provided as Figure 5. A weighting factor can be applied to each characteristic/diagnostic area. As noted above, the weighting factor is an organizationally defined value indicating the relative importance to the organization of the particular characteristic. In this example the IARMIndex is normalized to one (i.e., ranges between zero and one). In all cases, value scales are defined in terms familiar to corporate management.

The process for calculating the IARMIndex is shown in Equation 2. For activity element characteristics, the IARMIndex15 is normalized to one:

Equation 1 – Calculating the IARMIndex value:

$$IARMIIndex = \frac{\sqrt{\sum\_{i=1}^{n} w\_i^2 da\_i^2}}{\sqrt{\sum\_{i=1}^{n} w\_i^2 \text{(max} \{da\_i\})^2}}$$

Where:

*dai =* diagnostic area (characteristic) *n* = number of attributes *w*i = weighting factor for attribute *dai* max [*dai*] = maximum value of da*<sup>i</sup>*

Using the values from above and the organizationally defined weightings as shown in Table 3:

<sup>15</sup> Taken and adapted from Donaldson & Siegel [22] (p. 420).


**Table 3.** Weightings and Characteristics Values

Results in a value for **IARMIndex = 0.35**

222 Risk Management – Current Issues and Challenges

familiar to corporate management.

*dai =* diagnostic area (characteristic)

*w*i = weighting factor for attribute *dai* max [*dai*] = maximum value of da*<sup>i</sup>*

15 Taken and adapted from Donaldson & Siegel [22] (p. 420).

*n* = number of attributes

Where:

Table 3:

characteristics, the IARMIndex15 is normalized to one:

Equation 1 – Calculating the IARMIndex value:

**Figure 5.** Fishbone Diagram with All Risk Monitoring Characteristics and Subcharacteristics Values

All of these items are provided in the fishbone diagram provided as Figure 5. A weighting factor can be applied to each characteristic/diagnostic area. As noted above, the weighting factor is an organizationally defined value indicating the relative importance to the organization of the particular characteristic. In this example the IARMIndex is normalized to one (i.e., ranges between zero and one). In all cases, value scales are defined in terms

The process for calculating the IARMIndex is shown in Equation 2. For activity element

2 2

*i i*

*w da*

2 2

(max[ ])

*i i*

*w da*

1

*i n*

*n*

1

*i*

Using the values from above and the organizationally defined weightings as shown in

*IARMIndex*

### **4.6. Evaluating the IA OM® index**

The following steps describe the analysis process allowing organizational management to quickly uncover areas needing attention and prioritize which area(s) to address first to obtain the greatest benefit:


Continuing with the example from above, the IARMIndex value = 0.35. Assuming the organization has determined that any value under 0.70 is outside of its risk tolerance, the next step is to determine which component(s) to single out for further analysis. Using the values in Table 4, we find that characteristics da1 and da2 have the lowest values. However, if we also consider the weightings we see that w1 = 1.0 indicates that it is a higher priority item to the organization than w2, suggesting that if we cannot address both areas, priority

should be given to characteristic1, reviewing and updating the organization's risk management strategy.

IA OM®

**Wendy W. Ting, Ph.D., CISSP, CISM**, *Department of Defense, United States of America*, Dr. Ting specializes in performance metrics, cross domain information-sharing solutions, information security and systems security engineering. Dr. Ting earned her doctorate at the

[1] Ting WW, Comings DR. Information Assurance Metric for Assessing NIST's Monitoring Step in the Risk Management Framework. Information Security Journal: A Global

[2] NIST. SP 800-37, Revision 1: Guide for Applying the Risk Management Framework to Federal Information Systems. In: Commerce Do, ed.: National Institute of Standards

[3] Wheeler E. Security Risk Management: Building an Information Security Risk

[4] Olivia LM, ed. IT Solutions Series: IT Security Advice from Experts. Hershey, PA:

[5] Straub DW, Goodman S, Baskerville RL, eds. Information Security: Policy, Processes,

[6] DeLuccia IV JJ. IT Compliance and Controls: Best Practices for Implementation.

[7] LeVeque V. Information Security: A Strategic Approach. Hoboken, NJ: John Wiley &

[8] Pironti JP. Changing the Mind-set - Creating a Risk-conscious and Security-aware

[9] Johnson ME, Goetz E. Embedding Information Security into the Organization. IEEE

[10] HM Government. ITIL. 2012 [cited 2012 4/17/2012]; Available from: http://www.itil-

[11] ISACA. COBIT 5: A Business Framework for the Governance and Management of Enterprise IT. 2012 [cited 2012 4/17/2012]; Available from: http://www.isaca.org/

[12] ISO/IEC. ISO/IEC 27001:2005 Information technology -- Security techniques -- Information security management systems -- Requirements. 2012 [cited 2012 4/17/2012];

[13] ISO/IEC. ISO/IEC 27002:2005 Information technology -- Security techniques -- Code of practice for information security management. 2012 [cited 2012 4/17/2012]; Available

[14] ISO/IEC. ISO/IEC 27005:2011 Information technology -- Security techniques -- Information security risk management. Information Technology 2012 [cited 2012 4/17/2012]; Available from: http://www.iso.org/iso/iso\_catalogue/catalogue\_tc/

Available from: http://www.iso.org/iso/catalogue\_detail?csnumber=42103

from: http://www.iso.org/iso/catalogue\_detail?csnumber=50297

Management Program from the Ground Up. Waltham, MA: Syngress 2011.

University of Fairfax and her Master of Science at the University of Maryland.

**6. References** 

Perspective. 2010; 19(5): 253-62.

and Technology 2010: 93.

CyberTech Publishing 2004.

Sons, Inc. 2006.

officialsite.com/

COBIT/Pages/ default.aspx

and Practices. Armonk, NY M E Sharpe Inc. 2008.

Hoboken, NJ: John Wiley & Sons, Inc. 2008.

Culture. ISACA Journal. 2012 2012; 2: 13-9.

Security & Privacy. 2007 (May/June): 16-24.

catalogue\_detail.htm?csnumber=56742

as an Enterprise Risk Management Metric 225


**Table 4.** Component Values and Their Weightings
