2. Non-monotonic models of software reliability and security evaluation

In the course of preliminary acceptance testing and trial operation of information systems, it is important to define the moment when the testing can be considered complete and the system can undergo commissioning procedures. As for high-security software (including software intended for processing of confidential information or software used in critical system applications), current regulatory documents require that the test results be formalized<sup>1</sup> . In these cases, the test completion criteria (documented in test certificates), besides the very fact that the specified requirements are met, also include the values of test confidence parameters and parameters of the achieved level of reliability or correctness considering the specified evaluation accuracy. For these purposes it is reasonable to use mathematical models [1, 2] that are classified in this work in the following way (Figure 1):


Figure 1. The classification of mathematical models of tests.

<sup>1</sup> ISO/IEC 15408–3:2008. IT—Security techniques—Evaluation criteria for IT security—Part 3.

• Test confidence models that allow assessing confidence parameters of the test procedure

detected errors of different types. Considering all this, at the stage of certification, the information systems and software products can be regarded as non-modifiable, while at the stage of acceptance tests, they are defined as modifiable systems. This defines the difference in

2. Non-monotonic models of software reliability and security evaluation

In the course of preliminary acceptance testing and trial operation of information systems, it is important to define the moment when the testing can be considered complete and the system can undergo commissioning procedures. As for high-security software (including software intended for processing of confidential information or software used in critical system appli-

cases, the test completion criteria (documented in test certificates), besides the very fact that the specified requirements are met, also include the values of test confidence parameters and parameters of the achieved level of reliability or correctness considering the specified evaluation accuracy. For these purposes it is reasonable to use mathematical models [1, 2] that are

• Debugging models that allow assessing the software reliability parameters depending on the results of program runs on specified data areas and subsequent program modifications

• Time reliability growth models that allow assessing the software reliability parameters

depending on the time of test considering the corrected program errors

. In these

cations), current regulatory documents require that the test results be formalized<sup>1</sup>

approaches to developing the mathematical test models.

148 Probabilistic Modeling in System Engineering

classified in this work in the following way (Figure 1):

Figure 1. The classification of mathematical models of tests.

ISO/IEC 15408–3:2008. IT—Security techniques—Evaluation criteria for IT security—Part 3.

1

• Program complexity models based on the relationship between the software complexity metrics and program quality, reliability, and safety parameters

It should be noted that the latter three classes of test models are rather well developed2 [3–14]. For example, today, about 200 time models are known, mainly, NHPP models (e.g., [15–21]). At the same time, debugging models (also known as reliability growth models based on input data areas and revisions) are usually related only to Nelson's model and its modifications [22] developed at the dawn of the programming theory and do not reflect peculiarities of the modern team software development methods.

The early stage of testing is the typical scope of application for the debugging models. This is due to the fact that this period of a software system lifecycle is characterized by active modification of the programs aimed at correcting the detected errors. The models described in the literature reflect monotonic (typically, exponential, or logistic) growth of software operation reliability, which is not always true, as, for instance, in the case of implementation of the open-source software, multi-version or multiple replica software developed at different times by absolutely different teams of developers with diverse qualification, different styles, using various technologies and development systems, etc. This chapter is devoted to justification of new non-monotonic models and calculation of expressions of their parameters. We shall assume that the software reliability is a set of properties that characterize the ability of the program to maintain the specified level of availability in specified conditions during the specified period of time.<sup>3</sup> It is important to note that if the level of availability is restricted by security and vulnerability defects, the term reliability shall be equal to the term information security.

Definition of the software reliability is fundamentally different from that of the hardware, mainly, due to the fact that the software is not prone to aging in time. Two characteristics of the software reliability can be mentioned:


A number of debugging models were described in the literature, namely, Nelson's model, matrix model, LaPadula model, and other models [2, 5, 12, 13, 22], that reflect the stepwise monotonic growth of reliability and thus do not take into account the possibility of obvious reliability decrease, for example, due to introduction of global wavefront errors or addition of new functionality. Experience gathered by the test laboratory shows that application of such mathematical models either gives unreliable results or significantly increases the time required

<sup>2</sup> IEEE Std. 1633–2008 (R2016). Recommended Practice on Software Reliability.

<sup>3</sup> GOST 28806–90. Software quality. Terms and definitions.

to assess the software reliability [23]. That is why it is necessary to substantiate a nonmonotonic software reliability model and obtain calculated values of its parameters which are also required to assess its reliability.

According to the abovementioned first property of the software reliability, the process of software modification can be represented in the form of random transitions from one reliability state to another. The moments of transition are modifications of the tested object, which can be described as any changes of the program aimed at correcting the detected errors or developing the program.

We shall define the main software reliability indicator as the level of the program reliability, which represents the probability of its error-free starting for a set of basic data from the specified range. Considering the above said, we have the following software reliability change model:

$$P\_u = P\_0 + \sum\_{j=1}^{u} \Delta P\_{j\prime} \tag{1}$$

If we view software as a modifiable system, the change of the software reliability level after j

where Pj�<sup>1</sup> is the probability of error-free operation of the software after (j-1) revision,

� � is the probability of detection of software errors after (j-1) revision, Aj is the revision efficiency factor that characterizes the decreased probability of error as the result of j revision, and Bj is the revision negativeness factor that characterizes reliability decreases due to j

Proceeding to the recurrent expression and considering the maximum level of reliability to be

, we can obtain the software reliability evaluation model:

where P<sup>0</sup> is the initial level of reliability, P<sup>∞</sup> is the maximum level of reliability (0 ≤ P<sup>0</sup> < P<sup>∞</sup> ≤ 1),

The obtained expression (Eq. (3)) takes into account the possibility of uneven reliability growth of the tested object and the general trend of ΔPj growth decrease when the level of reliability Pj increases. However, when the model is presented in this way, it is generally monotonic since it does not take into account the different effects produced by fundamentally different types of modifications, for instance, changes of the software in order to correct errors or introduce new functional elements. Besides, the model does not reflect the degree of modification complexity and, consequently, probability of wavefront errors. Obviously, the model represented in this

In order to overcome the drawback described in the previous section, we offer a bigeminal reliability evaluation model based on metrics of the source code modification kij, for example, for error correction and software updates. This metric has no limits (i.e., the complexity metric

ensures comprehensive description of the considered process. Thus, if the revision efficiency

Yu j¼1

<sup>i</sup>¼<sup>0</sup> aikij, we can obtain the main calculated expression of the bigeminal reliability

<sup>1</sup> �<sup>X</sup> 2

i¼1

aikij=P<sup>∞</sup> !

that is most suitable for the software system and development system can be used<sup>4</sup>

Yu j¼1

1 � Aj=P<sup>∞</sup>

� � � BjPj�<sup>1</sup>, (2)

Models for Testing Modifiable Systems http://dx.doi.org/10.5772/intechopen.75126 151

� �, (3)

), which

, (4)

number of revisions can be represented using the following linear operator:

Pu ¼ P<sup>∞</sup> � ð Þ P<sup>∞</sup> � P<sup>0</sup>

form can be regarded as a monotonic reliability growth model [23].

2.1. Bigeminal model of software reliability and security evaluation

Pu ¼ P<sup>∞</sup> � ð Þ P<sup>∞</sup> � P<sup>0</sup>

IEEE Std. 1061–1998 (R2009). Standard for a Software Quality Metrics Methodology.

1 � Pj�<sup>1</sup>

revision.

equal to <sup>P</sup><sup>∞</sup> <sup>¼</sup> Aj

factor Aj <sup>¼</sup> <sup>P</sup><sup>2</sup>

4

evaluation model:

AjþBj

and u is the number of completed revisions.

ΔPj ¼ Aj 1 � Pj�<sup>1</sup>

where P<sup>0</sup> is the initial level of reliability (0 ≤ P<sup>0</sup> < 1), u is the number of completed revisions of the software, and ΔPj is increment of reliability after j revision.

The process of software reliability change can be graphically presented as a stepwise reliability growth function (Figure 2).

Figure 2. Change of the reliability level as a result of revisions.

If we view software as a modifiable system, the change of the software reliability level after j number of revisions can be represented using the following linear operator:

to assess the software reliability [23]. That is why it is necessary to substantiate a nonmonotonic software reliability model and obtain calculated values of its parameters which are

According to the abovementioned first property of the software reliability, the process of software modification can be represented in the form of random transitions from one reliability state to another. The moments of transition are modifications of the tested object, which can be described as any changes of the program aimed at correcting the detected errors or developing

We shall define the main software reliability indicator as the level of the program reliability, which represents the probability of its error-free starting for a set of basic data from the specified range. Considering the above said, we have the following software reliability change model:

Pu <sup>¼</sup> <sup>P</sup><sup>0</sup> <sup>þ</sup>X<sup>u</sup>

the software, and ΔPj is increment of reliability after j revision.

Figure 2. Change of the reliability level as a result of revisions.

j¼1

where P<sup>0</sup> is the initial level of reliability (0 ≤ P<sup>0</sup> < 1), u is the number of completed revisions of

The process of software reliability change can be graphically presented as a stepwise reliability

ΔPj, (1)

also required to assess its reliability.

150 Probabilistic Modeling in System Engineering

the program.

growth function (Figure 2).

$$
\Delta P\_{\circ} = A\_{\circ} \left( 1 - P\_{\circ -1} \right) - B\_{\circ} P\_{\circ -1 \prime} \tag{2}
$$

where Pj�<sup>1</sup> is the probability of error-free operation of the software after (j-1) revision, 1 � Pj�<sup>1</sup> � � is the probability of detection of software errors after (j-1) revision, Aj is the revision efficiency factor that characterizes the decreased probability of error as the result of j revision, and Bj is the revision negativeness factor that characterizes reliability decreases due to j revision.

Proceeding to the recurrent expression and considering the maximum level of reliability to be equal to <sup>P</sup><sup>∞</sup> <sup>¼</sup> Aj AjþBj , we can obtain the software reliability evaluation model:

$$P\_u = P\_\text{\textnu} - (P\_\text{\textnu} - P\_0) \prod\_{j=1}^u \left( 1 - A\_j / P\_\text{\textnu} \right) . \tag{3}$$

where P<sup>0</sup> is the initial level of reliability, P<sup>∞</sup> is the maximum level of reliability (0 ≤ P<sup>0</sup> < P<sup>∞</sup> ≤ 1), and u is the number of completed revisions.

The obtained expression (Eq. (3)) takes into account the possibility of uneven reliability growth of the tested object and the general trend of ΔPj growth decrease when the level of reliability Pj increases. However, when the model is presented in this way, it is generally monotonic since it does not take into account the different effects produced by fundamentally different types of modifications, for instance, changes of the software in order to correct errors or introduce new functional elements. Besides, the model does not reflect the degree of modification complexity and, consequently, probability of wavefront errors. Obviously, the model represented in this form can be regarded as a monotonic reliability growth model [23].

#### 2.1. Bigeminal model of software reliability and security evaluation

In order to overcome the drawback described in the previous section, we offer a bigeminal reliability evaluation model based on metrics of the source code modification kij, for example, for error correction and software updates. This metric has no limits (i.e., the complexity metric that is most suitable for the software system and development system can be used<sup>4</sup> ), which ensures comprehensive description of the considered process. Thus, if the revision efficiency factor Aj <sup>¼</sup> <sup>P</sup><sup>2</sup> <sup>i</sup>¼<sup>0</sup> aikij, we can obtain the main calculated expression of the bigeminal reliability evaluation model:

$$P\_{\rm u} = P\_{\rm \infty} - (P\_{\rm \infty} - P\_0) \prod\_{j=1}^{\rm u} \left( 1 - \sum\_{i=1}^{2} a\_i k\_{\vec{\eta}} / P\_{\rm \infty} \right), \tag{4}$$

<sup>4</sup> IEEE Std. 1061–1998 (R2009). Standard for a Software Quality Metrics Methodology.

where u is the number of completed revisions of the software, a<sup>1</sup> is the efficiency factor of the software revisions aimed at error correction, a<sup>2</sup> is the efficiency factor of the software revisions aimed at introduction of new functions, and kij is the scope of j revision with the purpose of correction or update.

The bigeminal model (Eq. (4)) depends on four parameters (P0, P∞, a1, a2Þ that can be easily calculated with the use, for instance, of the maximum likelihood method.
