1.2. SRGM background

limit downtime and vulnerabilities that might have unprecedented consequences for the anticipated societal-critical applications, while agility helps to continuously improve services to meet customer needs. In order to meet business objectives, and supported by advances in virtualization technologies, Telecom service providers are recently moving towards service agility, where the aim is to create new or improve the services provided to their users [1]. To this end, network software vendors must have the capability to continuously deliver and

These requirements mean that development teams must be able to produce quality software in very short time periods. This calls for accurate ways of planning team resource requirements or software size (number/complexity of features) ahead of time to avoid failing to meet customer needs. A useful approach for this assessment is predicting the number of defects during the requirements and design phase, which may allow for time to take preventive actions such as additional reviews and more extensive testing, finally improving software process control

Software quality can be significantly affected by the number of defects and amount/type of testing carried out at the different stages of the software development phase. In this subsection, we examine the defect flow through various development phases and in the field. Once a set of new features is identified, each of them goes through the phases shown in Figure 1. It can be observed that there are a number of activities overlapping in time (x-axis). New defects are introduced during the requirement, design and coding phases while old defects are carried over from previous releases into the current release. The thickness of the arrows indicates the relative numbers of defects expected at each point in time. Defects are found and removed or

integrate software from which such services are composed.

and achieving high software quality [2].

42 Telecommunication Networks - Trends and Developments

Figure 1. Defect flow development, test, field (traditional vs. agile).

1.1. Software development phases

The quality of software is measured in terms of software defects found during the customer operation period. SRGM is usually used to quantify the quality of software products before they are delivered to customers. SRGM is a very well-studied subject, with over 200 SRGMs developed since early 1970s [3–15]. These models can generally be categorized as either being parametric or non-parametric.

Parametric models are based on explicit mathematical expressions and physical interpretation. They were originally designed for system test but continue to be extended to include functional, integration, and other earlier test phases. They are the earliest developed models, are easily understood, and widely used. Most of the frequently used parametric models can be systematically grouped based on the shape of the function (i.e. S-shaped curve or exponential curve). S-shaped curve models use various types of S-shaped distributions such as Gamma, logistic, Weibull functions to match with actual data trends [6, 9], while exponential models (e.g. [3, 4, 7, 10, 11]) consider that defects are found at a constant rate through the different test phases. S-curve models have flexibility in describing different shapes of the trend since they have more than two parameters. Compared with S-shaped models, exponential models are simple with only two parameters and have been successfully demonstrated with practical data from large-scale software development projects [16, 17]. There have been many comparison studies performed and various tools have been developed for evaluating individual models in terms of how well each model fits to the data [18, 19].

Recently, non-parametric SRGMs are being proposed. Such models typically use machine learning techniques considering the inherent characteristics of the software (such as code size, test resources, test duration, complexity, etc.) as input. Although non-parametric models are promising, their use in practical applications is still limited by their need for (big amounts of) input data which is not always available, or whose quality may be unacceptable. Therefore, practical implementations and application of SRGMs are still mainly based on parametric models. Even then, there is no single model which can be used in every situation [20]. Models that work well during the early development phases are typically not efficient in the later

<sup>1</sup> It is worth noting that test phase names vary across projects and organizations.

stages, in which case a complete solution may require more than one model. In addition, the fact that the rate at which defects are found can change overtime requires that even for the same phase, the same model may need to be applied more than once each covering a specific trend. In practice, applying such models in an efficient way can be effort intensive and usually requires an expert.

This chapter discusses advances to the above techniques and approaches using BRACE [21], a cloud-based, fully-automated tool for software defect prediction, reliability and availability modeling and analytics. BRACE includes a number of technical contributions to make software defect prediction more practical for every software development project. First, the tool unifies and automates the entire process of data extraction, pre-processing, core processing and post-processing. Second, the core data analytics engine of BRACE—SRGM—provides a robust, consistent, flexible, fast, statistically sound approach to defect prediction for any defect data set without human intervention. This is achieved by modeling the entire defect trend as a series of piece-wise exponential curves, and incorporating a mechanism to automatically detect when to transition from one curve to the next. Moreover, to enhance the accuracy, whenever available, the algorithm can also take as input feature arrival data (which is a measure of development effort). This allows the enhanced SRGM to provide defect prediction from the project planning phase through the internal testing phase and the customer site test and customer operation periods. Finally, BRACE exposes a user interface which displays the output generated from the core processing. It makes it easier for projects to understand the current progress towards quality improvement. To the best of our knowledge, this is the first attempt to propose a practical software defect prediction and reliability management solution that can be re-used across multiple teams at industrial scale.

1. The pre-processing step is made up of a generic data-collector which uses application programming interfaces (APIs) to collect data from various sources. As example, defect data can be obtained from Jira while other project-related information may be obtained from GitLab. The data is generally obtained as records of defects with the corresponding fields (such as creation date, resolution date, defect type, software release, etc.). Once the data is received for project, a number of processing steps are carried out. These might include—as necessary—filtering out specific defects based on any chosen field, mapping particular field/values, grouping defects based on any of the fields, etc. These processing steps may be generic for all projects, or may also be customized for a particular project. The output of this step is defect numbers aggregated over time (for example defects each day,

) for any desired profile. The profiles are project-specific analysis groupings

Software Quality Assurance

45

http://dx.doi.org/10.5772/intechopen.79839

such as defects related to a given software component, development location, severity, etc.

2. The core processing implements the optimization function which derives maximum likelihood estimates of the model parameters for a given set of defect data. It then—dynamically and automatically—identifies changes in the defect trend as new defect data is added. This is achieved by comparing the goodness-of-fit of the predicted values to actual data points. It eventually generates multiple curves to describe the entire defect trend using a piecewise application of NHPP exponential models. The last curve is used for predicting total defects and residual defects at delivery. Detailed discussions on the SRGM used by the

While defect trend analysis is typically performed with weekly data, as we move towards continuous delivery, it is necessary to investigate the use of daily data. Therefore, the proposed tool has been applied to daily defect data from several projects, and has been able to produce results consistent with those from weekly data. Allowing the use of daily data is important as this allows SRGM to be applied in real time (daily), and hence help detect possible problems earlier.

3-days, week2

2

Figure 2. BRACE main processes.

core processing step are presented in Section 3.

#### 1.3. Chapter overview

The rest of this chapter is organized as follows: In Section 2, we present the design, implementation and use cases of BRACE. We also describe datasets from two example projects that are used as references throughout the chapter. In Section 3, we detail an automated SRGM algorithm which is the core analytics engine of BRACE. The algorithm enhances traditional SRGMs by enabling accurate early defect prediction, which, as mentioned earlier, is a necessity for most projects. Sections 4 and 5 will discuss key post-delivery metrics: customer defects and software availability respectively. The Chapter is concluded in Section 5. Data sets from two large-scale software development projects from telecom products are used to illustrate the effectiveness of BRACE throughout the chapter.
