3. Test planning and software revision models

This model has two unknown parameters that can be easily found with the help of the least

2.7. Approbation of the non-monotonic software reliability and security evaluation model

The study has shown that the suggested non-monotonic models (Eqs. (4) and (6)) provide high accuracy (σ<sup>j</sup> < 0:001) when the number of revisions exceeds 10 and the number of runs exceeds 50. In order to control the model consistency with the basic data, the Mises criterion was used

<sup>n</sup>n∈ ½ � 0:26; 1:9

Analysis of the effect of the software revision efficiency factor on the model (Eq. (6)) accuracy has shown that the accuracy can increase by an order of magnitude on the condition that revision classes are taken into account. Comparison of the suggested models with the well-

• Possibility of taking into account the software reliability values obtained during the

• Absence of subjective factors, such as programmer's qualification and the level of devel-

• Ease of application since there is no need to calculate probability of all program paths as,

Thus, the study actually substantiates the method of test planning based on utilization of the non-monotonic software reliability evaluation model using the results of runs and revisions. Within the scope of the suggested method, we obtained calculated expressions of parameters of the software reliability evaluation model and estimated accuracy and test planning. The suggested generic non-monotonic model (Eq. (6)) allows considering probable moments of the software reliability decrease typical, for instance, for open-source software development, multiple version software, etc. Accuracy of the generic model depends on how the task of software revision classification is solved. The model can be integrated with software reliability values obtained during the early stages of the software development. Simplification of the model allows reducing it to exponential NHPP models of reliability growth used at the stages of

The main advantage of the suggested non-monotonic models is the possibility to increase accuracy by more than 10% (as the results of introducing revision categories), which is equal

<sup>b</sup>uð Þ¼ <sup>0</sup>:<sup>01</sup> <sup>2</sup>:1, (22)

ω2

known debugging models has demonstrated a number of their advantages, namely:

• Taking into account the possible steep decrease of reliability due to upgrades

<sup>n</sup> is the Mises criterion and <sup>b</sup><sup>u</sup> is the threshold value.

• Possibility of taking into account the revision complexity

• Absence of restrictions for tests and information acquisition

previous stages of development and implementation

for example, in Nelson's model and its modifications [22]

information system operation and upgrade [23].

squares methods.

where ω<sup>2</sup>

(at threshold value of 0.01) [25]:

158 Probabilistic Modeling in System Engineering

opment technology

In the course of the software reliability management, it is necessary to plan the cost of testing in order to achieve the required level of the software reliability. Thus, it is useful to evaluate the trends relevant to the software development and implementation and predict the number of remaining errors and complexity of their correction.

The models (Eqs. (3), (4), (6)) described above can be used to calculate a number of planning indicators. Unfortunately, statistical models of reliability evaluation do not allow predicting the frequency of corrections of a specific type but only use this information. Specific revisions that depend on operating conditions, the achieved level of reliability, requirements for the software reliability, developers' qualification and experience and, consequently, their content may differ. In order to consider the revision types, it is reasonable to use the theory of multiple factor analysis. Since the change of the number of specific corrections is considered within the scope of revisions, the software modification complexity function can be approximated using, for example, a quadratic polynomial in one variable:

$$k\_{\rangle} = \kappa\_0 + \kappa\_1 j + \kappa\_2 j^2,\tag{23}$$

where κ0, κ1, and κ<sup>2</sup> are the polynomial parameters (j ¼ 1, u).

It is easy to demonstrate that the polynomial parameters have the following form:

$$\begin{cases} \kappa\_0 = \frac{30(\sum\_{j=1}^u \widehat{k}\_j - \frac{2}{u(u-1)} \sum\_{j=1}^u \widehat{k}\_j)^2 - \beta\_2 \left(2 + 3u - 3u^2 - 2u^3\right)}{10(u-1)}; \\\\ \kappa\_1 = \frac{6(\sum\_{j=1}^u \widehat{k}\_j - \frac{2}{u+1} \sum\_{j=1}^u \widehat{k}\_{jj} - \beta\_2 (1 - u^2)u}{u(1-u)}; \\\\ \kappa\_2 = \frac{\sum\_{j=1}^u \widehat{k}\_j \frac{u^2 + 3u - 2}{2} - u \sum\_{j=1}^u \widehat{k}\_{jj} - \frac{2}{u-1} \sum\_{j=1}^u \widehat{k}\_{j^2}}{u - (4 - u^2)}. \end{cases} (24)$$

Then, assuming that the estimation Pu of the model parameters and the achieved software reliability level was obtained based on the available test data, we have the following calculated expression of the reliability-level prediction model:

$$P\_{r\eta} = P\_{\bullet} - \left(P\_{\bullet} - P\_{u}\right) \prod\_{i=u+1}^{u+j} \left(1 - \frac{\sum\_{i=1}^{e} a\_{i}k\_{i\circ}}{P\_{\bullet}}\right),\tag{25}$$

by a non-monotonic software reliability growth function utilizing the fuzzy sets of theory in

It is possible to demonstrate that the non-monotonic software reliability growth function looks

where Pn is the probability of successful software run after n revision, a is the revision efficiency factor, P<sup>0</sup> is the initial level of reliability, and P<sup>∞</sup> is the maximum level of reliability.

This model depends on three parameters that can be conveniently calculated with the help of the maximum likelihood method. To create the likelihood function, it is reasonable to use the data recorded during the software tests, namely, the order of revisions, results of the software runs (whether any vulnerabilities were detected or not), and number of runs between the revisions.

The function ln ð Þ Ln is convex and is defined for a convex set; that is why in order to effectively find the maximum of the likelihood function we can use, for example, the modified steepest descent method with the variable increment parameter, which allows obtaining the desired parameters of the model (Eq. (28)). The greatest difficulty of modeling the automation system operational readiness is determined by the fact that the software reliability level has to be

1. Fuzziness of cause-and-effect relationship of the automation system as an ergatic system does not allow clear distinction between successful and unsuccessful revisions.

2. Definition of the amount of revisions as a function of the software metric characteristics does not always line up with reality. Knowledge of the software developers is required. 3. A number of errors appear as the result of shortcomings of the debugging and update procedures. Some errors are automatically eliminated at the final stages of the software

These uncertainties introduce a significant portion of subjectivity to the software reliability evaluation. The fuzzy set of theory allows taking them into account without substantial

Let us present the information on the debugging process in the form of the set X ¼ f g xi , where xi is the software revision (i ¼ 1, n). The number of relevant revisions is defined as

alteration of the model (Eq. (3)). This work is primarily aimed at solving this task.

4.1. Development of a fuzzy software reliability and security model

<sup>m</sup>c<sup>j</sup> ln 1 � <sup>P</sup><sup>∞</sup> <sup>þ</sup> ð Þ <sup>P</sup><sup>∞</sup> � <sup>P</sup><sup>0</sup> ð Þ <sup>1</sup> � <sup>a</sup>=P<sup>∞</sup> <sup>i</sup> � � �

� ln <sup>P</sup><sup>∞</sup> <sup>þ</sup> ð Þ <sup>P</sup><sup>∞</sup> � <sup>P</sup><sup>0</sup> ð Þ <sup>1</sup> � <sup>a</sup>=P<sup>∞</sup> <sup>i</sup> � ��:

It is easy to show that the maximum likelihood function logarithm will look as follows:

P<sup>∞</sup> � �<sup>n</sup>

, (28)

Models for Testing Modifiable Systems http://dx.doi.org/10.5772/intechopen.75126 161

Pn <sup>¼</sup> <sup>P</sup><sup>∞</sup> � ð Þ <sup>P</sup><sup>∞</sup> � <sup>P</sup><sup>0</sup> <sup>1</sup> � <sup>a</sup>

order to take into account the incompleteness of input data.

ln ð Þ¼ Ln

Xn J¼1

þðnj � <sup>m</sup>c<sup>j</sup>

evaluated in conditions of considerable uncertainty, namely:

development and do not require correction.

where <sup>m</sup>c<sup>j</sup> is the number of failures in nj tests and <sup>n</sup> is the number of revisions.

as follows:

where Prq is the required level of the software reliability, u is the number of the last revision, and j is the quantity of planned revisions.

The quantity of revisions required to achieve the desired level of reliability can be calculated using the cyclic recalculation of the expression (Eq. (25)). To this end Pu is calculated using the formula (Eq. (25)); further, in the cycle the value Puþ<sup>j</sup> is defined by increasing j. When the condition Puþ<sup>j</sup> ≥ Prg is met, the cycle stops.

To simplify application of the predictive model, let us assume that Aj ¼ a, which corresponds to the transition from the model (Eq. (6)) to (Eq. (3)). Then, after we reduce the expression (Eq. (25)) and take its logarithm, we will obtain the following expression required to evaluate the number J of software revisions that are necessary to achieve the desired level of reliability:

J ¼ ln <sup>P</sup>∞�Prg P∞�Pu � � ln 1ð Þ � a=P<sup>∞</sup> 0 @ 1 A � � � � � � � � � � � � , (26)

where k k ℵ is the operation of obtaining of the nearest biggest integer ℵ and a is the averaged software revision efficiency factor.

Assuming that revisions do not introduce additional errors (i.e., P<sup>∞</sup> ¼ 1), we can obtain the formula for the number of remaining errors after u revision:

$$N\_{\mu} = \left\| \left( \frac{\ln \left( \frac{1 - P\_{\eta}}{1 - P\_{u}} \right)}{\ln \left( 1 - a \right)} \right) \right\|. \tag{27}$$
