**Abstract**

The European gas network currently includes approximately 200,000 km high pressure transmission and distribution pipelines. The needs and requirements of this network are focused on risk-based security asset management, impacts and cascading effects of cyber-physical attacks on interdependent and interconnected European Gas grids. The European SecureGas project tackles these issues by implementing, updating, and incrementally improving extended components, which are contextualized, customized, deployed, demonstrated and validated in three business cases, according to scenarios defined by the end-users. Just validation is considered to be a key end activity, the essence of which is the evaluation of the proposed solution to determine whether it satisfies specified requirements. Therefore, the chapter deals with the validation strategy that can be implemented for the verification of these objectives and evaluation of technological based solutions which aim to strengthen the resilience of the European gas network.

**Keywords:** critical infrastructure, European gas network, validation, key performance indicators, resilience, protection

### **1. Introduction**

The European gas network is an important and irreplaceable subsector of European Critical Infrastructure (ECI) [1]. The functioning of this network is constantly affected by threats with a direct but also cascading or synergistic effect [2]. These threats can be of various natures, e.g. meteorological, geological, process-technological, cascading, personnel, cyber or physical [3]. The impact of these threats can result in serious disruption or even failure of the regional parts of the gas network. For this reason, it is necessary to continuously improve the protection system of the European Gas Network, in particular through risk analysis and the consequent strengthening of the resilience through the identification and elimination of the identified weaknesses.

One of the main measures and means to achieve the enhancement of resilience, is through technological solutions, which should address the operational and technical needs of the infrastructure and requirements of the end user, i.e. infrastructure operator [4]. The chapter therefore deals with the validation strategy [5] that can be implemented for the verification of these objectives and the evaluation of technological based solutions which aim to strengthen the resilience of the

European gas network. The main objective of the proposed validation plan, as part of an overall evaluation process, is to study the acceptance of a designed security system aiming to promote resilience [6] of gas critical infrastructures (at strategic, tactical and operational level). For this purpose, it is necessary to collect qualitative information concerning some key criteria of the system which define its performance in the operations. The primary focus of the validation strategy is to assess the functionality and effectiveness of the proposed system. However, the intuitiveness of the individual components as well as the overall exploitation and operationalization potential of the developed solution, should also be evaluated.

The aforementioned validation plan has been developed and verified through continuous interaction with critical infrastructure (CI) operators within the SecureGas project [7]. The project aims to improve the resilience capabilities of the gas CI. The methodology uses a gas CI-contextualized Panarchy loop [8] reflecting a disaster life-cycle management process. The objective is to reduce foreseen risk, optimize the monetary investment, and reduce uncertainties. Providing the CI operators with a detailed validation methodological procedure to assess the added value of security solutions added to their infrastructure is of high value. Within the context of the SecureGas validation and evaluation, the following aspects that are addressed include: performance versus expectation, ease-of-use, understandability, reliability of operations, completeness and reliability of output, functionality, man– machine interface and efficiency. The criteria for validation, i.e. Key Performance Indicators (KPIs) [9], can be clustered into two categories: (1) general criteria that apply to the whole SecureGas system, and (2) specific criteria that apply to individual components of the system.

Such validation plan is fully transferable to other CI operators both of Gas and other sectors (e.g. power, telecommunication). With a slight adjustment of the identified KPIs, it can provide a valuable information on the applicability and usefulness of a security solution for risk mitigation, prevention and response purposes within a CI.

### **2. Validation, verification and evaluation**

In order to understand the activities to be implemented from the validation point of view, definitions of the basic concepts used and are further analyzed below, presenting also several methodological approaches. Therefore, this section provides both a background analysis for validation-verification-evaluation processes and an adequate methodology.

The validation process involves the collection and evaluation of data, from the process design stage through commercial production phase, which establishes scientific evidence that a process meets a determined requirements. Process validation involves a series of activities taking place over the process. Regulatory authorities like European Medicines Agency and Food and Drug Administration have published guidelines relating to process validation [10]. The purpose of process validation is to ensure that varied inputs lead to consistent and high quality outputs. Process validation is an ongoing process that must be frequently adapted as manufacturing feedback is gathered. End-to-end validation of production processes is essential in determining product quality because quality cannot always be determined by a finished-product inspection. Process validation can be broken down into three steps: (1) process design, (2) process qualification, and (3) continued process verification.

The Guide to the Project Management Body of Knowledge (PMBOK guide), a standard adopted by the Institute of Electrical and Electronic Engineers, defines validation and verification as follows [5]:

**129**

control.

*Validation Strategy as a Part of the European Gas Network Protection*

It is often an internal process. Contrast with validation.

legislative rules, guidelines or proposals [18–25].

equipment have occurred [26].

clinical trial transitions.

• Validation: The assurance that a product, service, or system meets the needs of the customer and other identified stakeholders. It often involves acceptance

• Verification: The evaluation of whether or not a product, service, or system complies with a regulation, requirement, specification, or imposed condition.

These terms generally apply broadly across industries and institutions. In addition, they may have very specific meanings and requirements for specific products, regulations, and industries. Some examples: Software [11], Food and drug, Health care [12], Greenhouse gas [13], Traffic and transport [14], Simulation

In the context of the above, validation can generally be classified into five basic

• Prospective validation comprises the missions conducted before new items are released to make sure the characteristics of the interests which are functioning properly and which meet safety standards [17]. Some examples could be

• Retrospective validation is a process for items that are already in use in distribution or production. The validation is performed against the written specifications or predetermined expectations based upon their historical data/evidences that are documented/recorded. If any critical data is missing, then the work cannot be processed or can only be completed partially [10]. Retrospective validation is used for facilities, processes, and process controls in operation use that have not undergone a formally documented validation process. Validation of these facilities, processes, and process controls is possible by using historical data to provide the necessary documentary evidence that the process is doing what it is believed to do. Therefore, this type of validation is only acceptable for well-established processes and would be inappropriate where recent changes in the composition of product, operating processes, or

• Concurrent validation is used for establishing documented evidence that a facility and processes do what they purport to do, based on information generated during actual imputation of the process [26]. This approach involves monitoring of critical processing steps and end product testing of current production to show that the manufacturing process is in a state of

• Cross-validation is an approach by which the sets of scientific data generated

• Re-validation is carried out for the item of interest that is dismissed, repaired, integrated/coupled, relocated, or after a specified time lapse. Examples of this category could be relicensing/renewing driver's license, recertifying an analytical balance that has been expired or relocated, and even revalidating professionals [28]. Re-validation may also be conducted when a change occurs during the courses of activities, such as scientific researches or phases of

using two or more methods are critically assessed [27].

models [15], ICT industry, Civil engineering [16], Economics, Accounting,

and suitability with external customers. Contrast with verification.

*DOI: http://dx.doi.org/10.5772/intechopen.94644*

Agriculture, Arms control.

categories:

*Issues on Risk Analysis for Critical Infrastructure Protection*

individual components of the system.

**2. Validation, verification and evaluation**

validation and verification as follows [5]:

within a CI.

adequate methodology.

European gas network. The main objective of the proposed validation plan, as part of an overall evaluation process, is to study the acceptance of a designed security system aiming to promote resilience [6] of gas critical infrastructures (at strategic, tactical and operational level). For this purpose, it is necessary to collect qualitative information concerning some key criteria of the system which define its performance in the operations. The primary focus of the validation strategy is to assess the functionality and effectiveness of the proposed system. However, the intuitiveness of the individual components as well as the overall exploitation and operationaliza-

The aforementioned validation plan has been developed and verified through

Such validation plan is fully transferable to other CI operators both of Gas and other sectors (e.g. power, telecommunication). With a slight adjustment of the identified KPIs, it can provide a valuable information on the applicability and usefulness of a security solution for risk mitigation, prevention and response purposes

In order to understand the activities to be implemented from the validation point of view, definitions of the basic concepts used and are further analyzed below, presenting also several methodological approaches. Therefore, this section provides both a background analysis for validation-verification-evaluation processes and an

The validation process involves the collection and evaluation of data, from the process design stage through commercial production phase, which establishes scientific evidence that a process meets a determined requirements. Process validation involves a series of activities taking place over the process. Regulatory authorities like European Medicines Agency and Food and Drug Administration have published guidelines relating to process validation [10]. The purpose of process validation is to ensure that varied inputs lead to consistent and high quality outputs. Process validation is an ongoing process that must be frequently adapted as manufacturing feedback is gathered. End-to-end validation of production processes is essential in determining product quality because quality cannot always be determined by a finished-product inspection. Process validation can be broken down into three steps: (1) process design, (2) process qualification, and (3) continued process verification. The Guide to the Project Management Body of Knowledge (PMBOK guide), a standard adopted by the Institute of Electrical and Electronic Engineers, defines

continuous interaction with critical infrastructure (CI) operators within the SecureGas project [7]. The project aims to improve the resilience capabilities of the gas CI. The methodology uses a gas CI-contextualized Panarchy loop [8] reflecting a disaster life-cycle management process. The objective is to reduce foreseen risk, optimize the monetary investment, and reduce uncertainties. Providing the CI operators with a detailed validation methodological procedure to assess the added value of security solutions added to their infrastructure is of high value. Within the context of the SecureGas validation and evaluation, the following aspects that are addressed include: performance versus expectation, ease-of-use, understandability, reliability of operations, completeness and reliability of output, functionality, man– machine interface and efficiency. The criteria for validation, i.e. Key Performance Indicators (KPIs) [9], can be clustered into two categories: (1) general criteria that apply to the whole SecureGas system, and (2) specific criteria that apply to

tion potential of the developed solution, should also be evaluated.

**128**


These terms generally apply broadly across industries and institutions. In addition, they may have very specific meanings and requirements for specific products, regulations, and industries. Some examples: Software [11], Food and drug, Health care [12], Greenhouse gas [13], Traffic and transport [14], Simulation models [15], ICT industry, Civil engineering [16], Economics, Accounting, Agriculture, Arms control.

In the context of the above, validation can generally be classified into five basic categories:


In contrast, evaluation is a systematic assessment of a subject's qualities, using criteria governed by a set of standards. Evaluation involves tests or studies conducted to investigate and determine the technical suitability of an equipment, material, product, process, or system for the intended objective. So evaluation can be formative that is taking place during the development of a concept or proposal, project or organization, with the intention of improving the value or effectiveness of the proposal, project, or organization. It can also be summative, drawing lessons from a completed action or project or an organization at a later point in time or circumstance. [29]

According to the way the evaluation is conducted we can distinguish the following types [30]:


Evaluation can be characterized as being either formative or summative. Broadly (and this is not a rule), formative evaluation looks at what leads to an intervention working (the process), whereas summative evaluation looks at the short-term to long-term outcomes of an intervention on the target group [31]:


Process evaluation is an inductive method of theory construction, whereby observation can lead to identifying strengths and weaknesses in program processes and recommending needed improvements [32]. For this purpose, qualitative methods are most often used, which are defined in the context of evaluation as research methods that emphasize depth of understanding, that attempt to tap the deeper meaning of human experience, and that intend to generate theoretically richer, observations which are not easily reduced to numbers [32]. The most used qualitative evaluation methods include [33]: content analysis, situational analysis, in-house surveys and interviewing.

Content analysis involves studying documents and communication artifacts, which might be texts of various formats, pictures, audio or video [34]. Quantitative content analysis highlights frequency counts and objective analysis of these coded frequencies [35]. Additionally, quantitative content analysis begins with a framed hypothesis with coding decided on before the analysis begins. These coding categories are strictly relevant to the researcher's hypothesis. Quantitative analysis also takes a deductive approach [36].

Situation analysis refers to a collection of methods that managers use to analyze an organization's internal and external environment to understand the organization's capabilities, customers, and business environment. The situation analysis consists of several methods of analysis: The 5Cs Analysis, SWOT analysis and Porter five forces analysis [37]. These analyses help understand the analytical processes by which managers understand themselves, their consumers, and the marketplaces in which they compete.

**131**

*Validation Strategy as a Part of the European Gas Network Protection*

SWOT analysis is a strategic planning technique used to help a person or organization identify strengths, weaknesses, opportunities, and threats related to business competition or project planning [38]. It is designed for use in the preliminary stages of decision-making processes and can be used as a tool for evaluation of the strategic position of an organization. It is intended to specify the objectives of the project and identify the internal and external factors that are favorable and unfavorable to achieving those objectives. Users of a SWOT analysis often ask and answer questions to generate meaningful information for each category to make the tool useful

An interview is essentially a structured conversation where one participant asks questions, and the other provides answers. Interviews can range from Unstructured interview or free-wheeling and open-ended conversations in which there is no predetermined plan with prearranged questions [39], to highly structured conver-

Other commonly used tools and techniques for evaluation purposes [41] can include especially observation, survey questionnaires, case studies, analytical models, expert panel's consultation, cost–benefit analysis (CBA), and multi-criteria

Normally validation, verification and evaluation are performed in a row allowing to estimate the completeness and consistency of the system and examining its

To sum up, verification and validation heavily rely on earlier phases of the project. Verification is a rather technical process in which the main question is whether the system works properly. The validation process covers not only the demonstrations but also earlier meetings and discussions in which the requirements are refined. As already mentioned, verification of developed tool/solution is the process of determining that the system is built according to its specifications. Validation is the process of determining that the system actually fulfills the purpose for which it was intended. Evaluation reflects the value and the acceptance of the system by the

Following the analysis and presentation of validation, verification and evaluation processes, in this section, a holistic (including all those three processes) validation plan, will be analyzed. In principal, an effective validation and evaluation plan,

needs to seek, as clear as possible, answers to the following issues:

sations in which specific questions occur in a specified order [40].

*DOI: http://dx.doi.org/10.5772/intechopen.94644*

and identify their competitive advantage.

technical appropriateness, as depicted in **Figure 1**.

analysis (MCA).

**Figure 1.**

*Quality assurance framework.*

end users and its performance.

**3. Concept of creating a validation plan**

*Validation Strategy as a Part of the European Gas Network Protection DOI: http://dx.doi.org/10.5772/intechopen.94644*

**Figure 1.** *Quality assurance framework.*

*Issues on Risk Analysis for Critical Infrastructure Protection*

following types [30]:

methods of inquiry.

takes a deductive approach [36].

which they compete.

In contrast, evaluation is a systematic assessment of a subject's qualities, using criteria governed by a set of standards. Evaluation involves tests or studies conducted to investigate and determine the technical suitability of an equipment, material, product, process, or system for the intended objective. So evaluation can be formative that is taking place during the development of a concept or proposal, project or organization, with the intention of improving the value or effectiveness of the proposal, project, or organization. It can also be summative, drawing lessons from a completed action or project or an organization at a later point in time or circumstance. [29] According to the way the evaluation is conducted we can distinguish the

• Internal evaluation, carried out by organizations, groups or stakeholders

• External evaluation, carried out by specialists outside the development team, who are not employed within the organization responsible for the project under evaluation and who have no personal, financial or direct interest in the project.

Evaluation can be characterized as being either formative or summative. Broadly (and this is not a rule), formative evaluation looks at what leads to an intervention working (the process), whereas summative evaluation looks at the short-term to

• Formative evaluation takes place in the lead up to the project, as well as during the project, in order to improve the project design as it is being implemented (continual improvement). Formative evaluation often lends itself to qualitative

• Summative evaluation takes place during and following the project implemen-

Situation analysis refers to a collection of methods that managers use to analyze an organization's internal and external environment to understand the organization's capabilities, customers, and business environment. The situation analysis consists of several methods of analysis: The 5Cs Analysis, SWOT analysis and Porter five forces analysis [37]. These analyses help understand the analytical processes by which managers understand themselves, their consumers, and the marketplaces in

tation, and is associated with more objective, quantitative methods.

Process evaluation is an inductive method of theory construction, whereby observation can lead to identifying strengths and weaknesses in program processes and recommending needed improvements [32]. For this purpose, qualitative methods are most often used, which are defined in the context of evaluation as research methods that emphasize depth of understanding, that attempt to tap the deeper meaning of human experience, and that intend to generate theoretically richer, observations which are not easily reduced to numbers [32]. The most used qualitative evaluation methods include [33]: content analysis, situational analysis, in-house surveys and interviewing. Content analysis involves studying documents and communication artifacts, which might be texts of various formats, pictures, audio or video [34]. Quantitative content analysis highlights frequency counts and objective analysis of these coded frequencies [35]. Additionally, quantitative content analysis begins with a framed hypothesis with coding decided on before the analysis begins. These coding categories are strictly relevant to the researcher's hypothesis. Quantitative analysis also

directly involved in the implementation of the project solution.

long-term outcomes of an intervention on the target group [31]:

**130**

SWOT analysis is a strategic planning technique used to help a person or organization identify strengths, weaknesses, opportunities, and threats related to business competition or project planning [38]. It is designed for use in the preliminary stages of decision-making processes and can be used as a tool for evaluation of the strategic position of an organization. It is intended to specify the objectives of the project and identify the internal and external factors that are favorable and unfavorable to achieving those objectives. Users of a SWOT analysis often ask and answer questions to generate meaningful information for each category to make the tool useful and identify their competitive advantage.

An interview is essentially a structured conversation where one participant asks questions, and the other provides answers. Interviews can range from Unstructured interview or free-wheeling and open-ended conversations in which there is no predetermined plan with prearranged questions [39], to highly structured conversations in which specific questions occur in a specified order [40].

Other commonly used tools and techniques for evaluation purposes [41] can include especially observation, survey questionnaires, case studies, analytical models, expert panel's consultation, cost–benefit analysis (CBA), and multi-criteria analysis (MCA).

Normally validation, verification and evaluation are performed in a row allowing to estimate the completeness and consistency of the system and examining its technical appropriateness, as depicted in **Figure 1**.

To sum up, verification and validation heavily rely on earlier phases of the project. Verification is a rather technical process in which the main question is whether the system works properly. The validation process covers not only the demonstrations but also earlier meetings and discussions in which the requirements are refined. As already mentioned, verification of developed tool/solution is the process of determining that the system is built according to its specifications. Validation is the process of determining that the system actually fulfills the purpose for which it was intended. Evaluation reflects the value and the acceptance of the system by the end users and its performance.

## **3. Concept of creating a validation plan**

Following the analysis and presentation of validation, verification and evaluation processes, in this section, a holistic (including all those three processes) validation plan, will be analyzed. In principal, an effective validation and evaluation plan, needs to seek, as clear as possible, answers to the following issues:


All these questions have been taken under consideration and are answered and described in detail as part of the SecureGas validation-evaluation methodological approach. In this four-step methodology (**Figure 2**), a set of business cases (BCs) is used to support the validation, verification and evaluation of SecureGas solution. Three BCs, addressing relevant issues for the gas sector (production, transport and distribution phase of the gas lifecycle, including different infrastructures for each phase) have been identified to ensure the delivery of solutions and services to the end-users. During the BCs implementation, tailor-made scenarios for the CIs will be used for demonstrations on actual sites. The technical components involved will be assessed quantitively (by measuring foreseen KPIs) and qualitatively (by using a set of questionnaires and interviews to the participants in the demonstrations).

#### **3.1 Set the context**

This kick-off step entails all the discussions and reviews with relevant stakeholders for the exact identification of the gaps and the existing capabilities. This step also sets the scope and the objectives of each BC for the SecureGas solution to provide differentiation from current practices and added value to the operational environment of a gas CI.

**133**

Section 4).

*Validation Strategy as a Part of the European Gas Network Protection*

interconnected, interdependent or similar CIs);

Within SecureGas framework, the end-user team consists of the gas CI operators

1.CI operators, managers and administrators, security liaison officers (also from

2.Emergency response authorities (police, fire brigade, civil protection, etc.);

5.Secondary/other security professionals and practitioners (e.g. policy makers,

The SecureGas validation and evaluation process is an essential part of the project's development cycle. The development cycle is user-oriented, which means it relies on the perception, needs and responses by end users. Based on this development cycle, in SecureGas phase 1: "construct/develop", user requirements and specifications are identified leading to conceptual model (CM), concept of operations (ConOps) and high level reference architecture (HLRA). The CM, ConOps and HRLA will be implemented and demonstrated in phase 2: "demonstrate" and

Initial and crucial substeps to achieve an efficient planning and implementation

2.Identify legacy systems and existing infrastructures, integration-data sharing, possible limitations, etc., and collaborate with the technical team to develop a

For the execution of these substeps, some may choose from a set of existing tools and frameworks, e.g. risk and vulnerability assessment and penetration testing (see

The main objectives of the evaluation process will be to study the acceptance of the SecureGas system (at the strategic, tactical and operational levels), assess the performance of its components and the operational potential of the developed solution.

1.Identify CI assets, threats, vulnerabilities, requirements, procedures, etc., in order to prepare the scenario including CI's specific security issues and

3.National Authorities (CI regulatory authorities, ministries, etc.);

participating in the project (DEPA, EDAA, AMBER, ENI). Further to them, the SecureGas technical component providers are actively engaged and directly involved in all phases of the validation plan. External stakeholders have been identified and will be involved only in the BC implementation phase. They will participate and provide feedback for evaluation purposes. The stakeholders/actors participating in the pilot activities may vary among the different BCs however they

*DOI: http://dx.doi.org/10.5772/intechopen.94644*

belong to one of the following groups:

4.Security service providers;

other EU research projects, etc.).

finally validated in phase 3: "validate & exploit".

addressing end users' actual needs.

SecureGas solution tuned to the project's BCs.

*3.1.3 Define the objective of the validation-evaluation process*

of the BC are to:

*3.1.2 Identify requirements and processes*

*3.1.1 Identify end users/teams*

**Figure 2.**

*SecureGas validation-evaluation methodology.*
