Clinical Validation of a Whole Exome Sequencing Pipeline

*Debra O. Prosser, Indu Raja, Kelly Kolkiewicz, Antonio Milano and Donald Roy Love*

#### **Abstract**

Establishing whole exome sequencing (WES) in an accredited clinical diagnostic space is challenging. The validation (as opposed to verification) of an approach that will lead to clinical reports requires adhering to international guidelines and recommendations and developing a robust analytical pipeline that can scale due to the increasing clinical demand for comprehensive gene screening. This chapter will present a step-wise approach to WES validation that any laboratory can follow. The focus will be on highlighting the pivotal technical issues that must be addressed in validating WES and the analytical tools and QC metrics that must be considered before implementing WES in a clinical environment.

**Keywords:** whole exome sequencing, next-generation sequencing, validation, bioinformatics, diagnostics

### **1. Introduction**

The decision as to which type of genetic test should be implemented by a clinical laboratory is largely driven by the type of referrals received by the laboratory and the complexity of patients' clinical phenotypes. In the main, testing has advanced from single-gene to multi-gene panels in which next-generation sequencing (NGS) has offered the technical means of undertaking this approach at low cost and high throughput. However, with the increasing awareness of genetic heterogeneity combined with gene discovery, whole exome sequencing (WES) offers laboratories a more streamlined approach. By implementing a single wet-work pipeline of exome capture coupled with the ability to analyze a virtual gene panel or report on the whole exome, laboratories can perform NGS in a more efficient manner.

Since the inception of NGS over a decade ago, multiple recommendations and guidelines have been published for NGS [1–3]. Using these guidelines, the College of American Pathologists (CAP) and Association for Molecular Pathology (AMP) published their Practical Framework for Designing and Implementing NGS Tests for Inherited Disorders in 2019 [4], and this is available through the CAP website (https://www.cap.org/member-resources/precision-medicine/ next-generation-sequencing-ngs-worksheets).

We adopted this framework to establish a diagnostic NGS service using whole exome sequencing as our capture procedure and analyzing virtual gene panels or WES for reporting purposes.

The framework provides guidance and editable worksheets for the five steps involved in test establishment and validation.


Throughout the validation process, it is essential that the NGS workflow is informed by the real-world local environment in which clinical testing will be performed.

#### **2. Test design: setup**

In view of the diverse range of referrals made to the authors' genetics laboratory (serving the needs of a 400-bed women and children's hospital in the Middle East), a whole exome capture solution was chosen for library preparation. The principal motivation behind this determination was to achieve an efficient workflow that would allow appropriate batching coupled with a time-limited turnaround time (TAT) for all referrals.

The limited number of staff in the authors' laboratory demanded a WES workflow that could be easily automated, twinned with a data analysis package that would allow secure remote access with a strong databasing function. The whole exome solution capture by SOPHiA™ Genetics was chosen for library preparation. This platform allows for the analysis of WES, clinical exome sequencing (CES) and clinical gene panels, together with the identification of single-nucleotide variants (SNVs) and copy number variants (CNVs) using SOPHiA™ DDM software.

#### **3. Assay design and optimization**

The validation pipeline needs to be grounded from the beginning in terms of the requirements of the test, which must take into account the sample types the laboratory will receive and the parameters that need to be satisfied (see **Table 1**).

Routinely, whole blood samples collected in EDTA are received by the authors' laboratory for testing. Therefore, our validation focused only on genomic DNA extracted from whole blood using our standard methods. The baseline validation of the WES data required the inclusion of two HapMap gDNA samples: the NIST control (NA12878) and the commercial control (SG063) supplied by SOPHiA™ Genetics.

The WES capture by SOPHiA™ Genetics was used for library preparation following all the steps as set out by the automated WES 32 reaction protocol. For instrumentation, our validation was restricted to automated library preparation using the PE Sciclone® G3 NGS workstation and sequencing using the Illumina® HiSeq4000 platform.

A critical additional consideration was the need for copy number variant calls to be made. This required a minimum batch number of eight patients and high coverage requirements, which involved restricting the number of samples per Illumina® HiSeq4000 lane to one pool of eight patients.

**119**

file name:

**Table 1.**

*Test requirements and limitations.*

i.Unique sample identifier

ii.Unique patient identifier

iv.Laboratory location identifier

iii.Unique run identifier

subsequent testing.

**4. Test validation**

with differing technologists.

*Clinical Validation of a Whole Exome Sequencing Pipeline*

**Test requirements Must** 

CES Y Clinical panels Y CNV detection Y

DNA from whole blood collected in EDTA Y DNA from external/commercial sources (limitations) Y

Combine different tests (existing or planned) within a sequencing run Y

How deeply does each position need to be covered for accurate variant calling

(if known—otherwise address during test optimization)

WES Y

Necessary sample throughput per month 16 32

Required/expected TAT 3 months 2 months

*WES, whole exome sequencing; CES, clinical exome sequencing; CNV, copy number variant; TAT, turnaround time.*

**have**

>20x >50x

**Nice to have**

Importantly, the naming of the sequence files (.bam,. FASTQ, etc.) should be considered during the early phase of test design and validation. File conventions that are used for the bioinformatic process may be limited in terms of the type of special characters and/or character length. Following recommendations in the CAP/AMP-Guidelines for Validating Next-Generation Sequencing Bioinformatics Pipelines [5], the identity of the sample must be preserved throughout all steps of the bioinformatic pipeline. These authors recommend the following four unique identifiers that should be applied to the sample

It is essential that the file naming convention that is decided upon for validation adheres to the above recommendations and can be universally implemented for all

Test validation mandates a need for accuracy, precision and stability. These assessments must be made in the context of expected clinical workloads and performance. For the authors' laboratory, the sample batch size was set at 16 samples per validation batch and a total of three validation runs performed over differing days

Analytical performance was characterized by the assessment of precision, sensitivity and concordance of variant calls against previously validated data.

*DOI: http://dx.doi.org/10.5772/intechopen.93251*

*Clinical Validation of a Whole Exome Sequencing Pipeline DOI: http://dx.doi.org/10.5772/intechopen.93251*


#### **Table 1.**

*Methods in Molecular Medicine*

1.Test design: setup

3.Test validation

**2. Test design: setup**

(TAT) for all referrals.

**3. Assay design and optimization**

HiSeq4000 lane to one pool of eight patients.

4.Quality management

5.Bioinformatics and IT

involved in test establishment and validation.

2.Assay design and optimization

The framework provides guidance and editable worksheets for the five steps

Throughout the validation process, it is essential that the NGS workflow is informed

In view of the diverse range of referrals made to the authors' genetics laboratory (serving the needs of a 400-bed women and children's hospital in the Middle East), a whole exome capture solution was chosen for library preparation. The principal motivation behind this determination was to achieve an efficient workflow that would allow appropriate batching coupled with a time-limited turnaround time

The limited number of staff in the authors' laboratory demanded a WES workflow that could be easily automated, twinned with a data analysis package that would allow secure remote access with a strong databasing function. The whole exome solution capture by SOPHiA™ Genetics was chosen for library preparation. This platform allows for the analysis of WES, clinical exome sequencing (CES) and clinical gene panels, together with the identification of single-nucleotide variants (SNVs) and copy number variants (CNVs) using SOPHiA™ DDM software.

The validation pipeline needs to be grounded from the beginning in terms of the requirements of the test, which must take into account the sample types the labora-

A critical additional consideration was the need for copy number variant calls to be made. This required a minimum batch number of eight patients and high coverage requirements, which involved restricting the number of samples per Illumina®

Routinely, whole blood samples collected in EDTA are received by the authors' laboratory for testing. Therefore, our validation focused only on genomic DNA extracted from whole blood using our standard methods. The baseline validation of the WES data required the inclusion of two HapMap gDNA samples: the NIST control (NA12878) and the commercial control (SG063) supplied by SOPHiA™ Genetics. The WES capture by SOPHiA™ Genetics was used for library preparation following all the steps as set out by the automated WES 32 reaction protocol. For instrumentation, our validation was restricted to automated library preparation using the PE Sciclone® G3 NGS workstation and sequencing using the Illumina®

tory will receive and the parameters that need to be satisfied (see **Table 1**).

by the real-world local environment in which clinical testing will be performed.

**118**

HiSeq4000 platform.

*Test requirements and limitations.*

Importantly, the naming of the sequence files (.bam,. FASTQ, etc.) should be considered during the early phase of test design and validation. File conventions that are used for the bioinformatic process may be limited in terms of the type of special characters and/or character length. Following recommendations in the CAP/AMP-Guidelines for Validating Next-Generation Sequencing Bioinformatics Pipelines [5], the identity of the sample must be preserved throughout all steps of the bioinformatic pipeline. These authors recommend the following four unique identifiers that should be applied to the sample file name:

i.Unique sample identifier

ii.Unique patient identifier

iii.Unique run identifier

iv.Laboratory location identifier

It is essential that the file naming convention that is decided upon for validation adheres to the above recommendations and can be universally implemented for all subsequent testing.

#### **4. Test validation**

Test validation mandates a need for accuracy, precision and stability. These assessments must be made in the context of expected clinical workloads and performance. For the authors' laboratory, the sample batch size was set at 16 samples per validation batch and a total of three validation runs performed over differing days with differing technologists.

Analytical performance was characterized by the assessment of precision, sensitivity and concordance of variant calls against previously validated data.

#### *Methods in Molecular Medicine*

Inter-run and intra-run data were achieved by replicate analysis of two HapMap gDNAs, the NIST sample, NA12878, and the commercial control supplied by SOPHiA™ Genetics, SG063, as well as four well-characterized clinical samples previously reported by accredited laboratories. The remaining samples included a representative group of the clinical samples received by the authors' laboratory (see **Table 2**).

The complete NGS workflow should be included in the validation, from library preparation to bioinformatic analysis to report generation, which is highlighted below.

• Sample collection and DNA extraction. Genomic DNA is extracted and purified from blood samples using either the Gentra® PureGene® DNA Blood Mini Kit or the QIAsymphony® DSP DNA Midi kit (QIAGEN, Hilden, Germany). DNA quality is initially assessed by NanoDrop™ spectrophotometry.

Genomic DNA preparation. The initial preparation of gDNA used in NGS library preparation is the most critical step in the NGS workflow, and the care and time taken here are key to successful library amplification and sequencing.

High-quality gDNA can be by quantified using a Qubit™ fluorometer followed by sequential dilution with further quantification to the desired input concentration. It is essential to minimize pipetting gDNA volumes of less than 5 μl for dilution. In our study, gDNA is prepared to a working concentration of 40 ng/ μl. After Qubit™ quantification, the integrity of the gDNA can be analyzed using an Agilent TapeStation 4200. Samples with a DNA integrity number (DIN) of greater than 7.5 can proceed to WES capture.

• Library preparation, targeted capture and sequencing. Whole exome sequencing was performed according to the SOPHiA™ Whole Exome Solution 32 Samples User Guide, in combination with the SOPHiA™ Library Preparation and Capture User Guide—automation with PerkinElmer Sciclone® G3 NGS workstation. Each validation run consists of 16 samples that are divided into 2 pools of 8 samples each, as shown in the validation grid in **Table 3**.

The SOPHiA™ WES protocol for library construction subjects genomic DNA (200 ng) to enzymatic fragmentation, end repair and A-tailing. All these steps occur using a Sciclone® G3 NGS workstation. The adapter-ligated DNA is then amplified in a limited way via an eight-cycle PCR protocol.

Post-amplification cleanup of the libraries is carried out using the Sciclone® G3 NGS workstation, and libraries are prepared for quantitation with a dilution factor of 4.

Amplified libraries are analyzed using Qubit™ fluorometer and Agilent TapeStation 4200 to assess the quantity and quality of each individual library. Library DNA fragments should have a size distribution between 300 and 700 bp. Genomic DNA that has been fragmented, end repaired, A-tailed and adapter-ligated can then be considered library DNA, which is ready for pooling and then hybridization and capture. In the case of the SOPHiA™ WES protocol, eight samples are pooled (200 ng of each library) per capture.

Prepared pools are hybridized for 4 h followed by post-capture amplification and cleanup on the Sciclone® G3 NGS workstation.

Final library quantification is performed for each captured library pool using a Qubit™ fluorometer and Agilent TapeStation 4200. Subsequent pools are

**121**

**Sample ID**

VAL-1 VAL-2 VAL-3 VAL-4 VAL-5 VAL-6 VAL-7 VAL-8 VAL-9 VAL-10 VAL-11 VAL-12 VAL-13

Anonymized

Gene-specific

Variant type

validation

patient specimen

Anonymized

Gene-specific

Variant range

Epilepsy gene panel

Single-gene analysis CFTR: deletion of exons 4–8

validation

patient specimen

Anonymized

Gene-specific

Variant type

validation

patient specimen

Anonymized

Gene-specific

Variant type

validation

patient specimen

Anonymized

Gene-specific

Variant type

Paroxysmal Dystonia gene panel Del 16p11.2

chr16:29,656,684-30,190,568

Leukodystrophy gene panel MLC1:c.908\_918delinsGCA

p.(Val303Glyfs\*96)

Epilepsy gene panel WWOX: Deletion of exons 1–5

CNV SNV DEL/

Sensitivity

DUP

CNV

Sensitivity

Sensitivity

validation

patient specimen

Anonymized

Gene-specific

Variant type

validation

patient specimen

Anonymized

Gene-specific

Variant type

validation

patient specimen

Anonymized patient specimen

Baseline validation

Variant type prevalent in gene

Arrhythmia cardiomyopathy gene panel

SCN5A:c.4867C > T p.(Arg1623\*)

Custom panel of 196 genes 200 genomic co-ordinates

SNV DEL/

Sensitivity

DUP

CNV DELINS

Sensitivity

Sensitivity

Anonymized patient specimen

Baseline validation

Variant type

Anonymized patient specimen

Baseline validation

Variant type prevalent in gene

Craniosynostosis gene panel CACNA1H:c.4318\_4319delinsGC p.(Phe1440Ala)

Tuberous sclerosis gene panel TSC2: Deletion of exons 2 to 16

CNV SNV

Sensitivity

(stop)

Inter-run variability Sensitivity

Anonymized patient specimen

Baseline validation

Variant type

Ciliopathy gene panel CCDC39:c.2017G > T p.(Glu673\*) CCDC39: Deletion of exons 14 to 20

Single-gene analysis CFTR:c.1521\_1523delCTT p.(Phe508del)

SG063

Baseline validation

N/A

N/A

NA12878

Baseline validation

N/A

N/A

**Description**

**Purpose**

**Purpose (detail)**

**Specific variant/s of interest**

**Variant type**

N/A N/A SNV CNV

DEL DELINS

Inter-run variability Sensitivity

Intra-run variability Inter-run variability

Intra-run variability Inter-run variability

**Measured metric**

*Clinical Validation of a Whole Exome Sequencing Pipeline*

Inter-run variability Sensitivity

*DOI: http://dx.doi.org/10.5772/intechopen.93251*

Inter-run variability Sensitivity


#### *Clinical Validation of a Whole Exome Sequencing Pipeline DOI: http://dx.doi.org/10.5772/intechopen.93251*

*Methods in Molecular Medicine*

Inter-run and intra-run data were achieved by replicate analysis of two HapMap

• Sample collection and DNA extraction. Genomic DNA is extracted and purified from blood samples using either the Gentra® PureGene® DNA Blood Mini Kit or the QIAsymphony® DSP DNA Midi kit (QIAGEN, Hilden, Germany). DNA

Genomic DNA preparation. The initial preparation of gDNA used in NGS library preparation is the most critical step in the NGS workflow, and the care and time taken here are key to successful library amplification and sequencing.

High-quality gDNA can be by quantified using a Qubit™ fluorometer followed by sequential dilution with further quantification to the desired input concentration. It is essential to minimize pipetting gDNA volumes of less than 5 μl for dilution. In our study, gDNA is prepared to a working concentration of 40 ng/ μl. After Qubit™ quantification, the integrity of the gDNA can be analyzed using an Agilent TapeStation 4200. Samples with a DNA integrity number

• Library preparation, targeted capture and sequencing. Whole exome sequencing was performed according to the SOPHiA™ Whole Exome Solution 32 Samples User Guide, in combination with the SOPHiA™ Library Preparation and Capture User Guide—automation with PerkinElmer Sciclone® G3 NGS workstation. Each validation run consists of 16 samples that are divided into 2

The SOPHiA™ WES protocol for library construction subjects genomic DNA (200 ng) to enzymatic fragmentation, end repair and A-tailing. All these steps occur using a Sciclone® G3 NGS workstation. The adapter-ligated DNA is then

Post-amplification cleanup of the libraries is carried out using the Sciclone® G3 NGS workstation, and libraries are prepared for quantitation with a dilu-

Prepared pools are hybridized for 4 h followed by post-capture amplification

Final library quantification is performed for each captured library pool using a Qubit™ fluorometer and Agilent TapeStation 4200. Subsequent pools are

Amplified libraries are analyzed using Qubit™ fluorometer and Agilent TapeStation 4200 to assess the quantity and quality of each individual library. Library DNA fragments should have a size distribution between 300 and 700 bp. Genomic DNA that has been fragmented, end repaired, A-tailed and adapter-ligated can then be considered library DNA, which is ready for pooling and then hybridization and capture. In the case of the SOPHiA™ WES proto-

col, eight samples are pooled (200 ng of each library) per capture.

and cleanup on the Sciclone® G3 NGS workstation.

pools of 8 samples each, as shown in the validation grid in **Table 3**.

amplified in a limited way via an eight-cycle PCR protocol.

SOPHiA™ Genetics, SG063, as well as four well-characterized clinical samples previously reported by accredited laboratories. The remaining samples included a representative group of the clinical samples received by the authors' laboratory (see **Table 2**). The complete NGS workflow should be included in the validation, from library preparation to bioinformatic analysis to report generation, which is highlighted below.

gDNAs, the NIST sample, NA12878, and the commercial control supplied by

quality is initially assessed by NanoDrop™ spectrophotometry.

(DIN) of greater than 7.5 can proceed to WES capture.

**120**

tion factor of 4.


**123**

**Sample ID**

VAL-26 VAL-27 VAL-28 **Table 2.** *Sample list.*

Anonymized patient specimen

Gene-specific validation *DEL, deletion; INS, insertion; DUP, duplication; SNV, single-nucleotide variant; CNV, copy number variant.*

Variant type

Anonymized patient specimen

Gene-specific validation

Variant type

Anonymized patient specimen

Gene-specific validation

Variant type

Primary Immunodeficiency gene panel TBX1:c.1383\_1421del p.(Ala464\_Ala476del)

Dilated cardiomyopathy gene panel TTN:c.75984\_75985insTACCA p.(Ala25329Tyrfs\*32)

Pediatric cancer gene panel SMARCB1:c.159\_160delinsTAT

CTGGAGGCG (p.Leu54Ilefs\*20)

DELINS

Sensitivity

INS

Sensitivity

**Description**

**Purpose**

**Purpose (detail)**

**Specific variant/s of interest**

**Variant type**

DEL

Sensitivity

**Measured metric**

*Clinical Validation of a Whole Exome Sequencing Pipeline*

*DOI: http://dx.doi.org/10.5772/intechopen.93251*


**Table 2.** *Sample list.*

#### *Clinical Validation of a Whole Exome Sequencing Pipeline DOI: http://dx.doi.org/10.5772/intechopen.93251*

*Methods in Molecular Medicine*

**122**

**Sample** 

**Description**

**Purpose**

**Purpose (detail)**

**Specific variant/s of interest**

**Variant** 

**Measured metric**

**type**

SNV DEL/

Sensitivity

DUP

SNV DEL/

Sensitivity

DUP

DEL CNV SNV CNV

SNV DEL

Sensitivity

Sensitivity

Sensitivity

Sensitivity

**ID**

VAL-14 VAL-15 VAL-16 VAL-17 VAL-18 VAL-19 VAL-20 VAL-21 VAL-22 VAL-23 VAL-24 VAL-25

Anonymized

Gene-specific

Pseudogene

Custom panel of nine genes

validation

(pseudogene)

patient specimen

Anonymized

Gene-specific

validation

patient specimen

Anonymized

Gene-specific

validation

patient specimen

Anonymized

Gene-specific

Variant type

prevalent in gene

Variant type

prevalent in gene

Variant range

validation

patient specimen

Anonymized

patient specimen

Anonymized

patient specimen

Anonymized

Gene-specific

Variant range

validation

(pseudogene)

Gene-specific

Variant range

Custom panel of 196 genes 200 genomic coordinates

Molecular karyotype referral Duplication at 16p13.11,

deletion at 12p31 and duplication at Xp21.1

Single-gene analysis DMD: duplication exons 45–62

Dystrophinopathy gene panel DMD: deletion of exons 8–34

Custom panel of 196 genes 200 genomic co-ordinates

SNV DEL/

Sensitivity

DUP

SNV DEL/

Sensitivity

DUP

CNV

Sensitivity

CNV

Sensitivity

SNV DEL/

Blind analysis

DUP

CNV

Sensitivity

validation

Chromosomal CNV

Variant type

validation

patient specimen

Anonymized

Gene-specific

Variant range

validation

patient specimen

Anonymized

patient specimen

Anonymized

Gene-specific

Variant type

validation

Chromosomal CNV

Variant type

validation

patient specimen

Anonymized

Gene-specific

Variant range

Cholestasis gene panel

Tuberous sclerosis gene panel (2 genes)

TSC2:c.5238\_5255del p.(His1746\_Arg1751del)

Molecular karyotype referral Dup 22q11.21

chr22:18,661,724-21,809,099

Primary ciliary dyskinesia gene panel DNAH5: Gain of

exons 1 to 50 DNAH5:c.5503C > T p.(Gln1835\*)

Inherited cancer gene panel CDKN2A:c.9\_32dup

p.(Ala4\_Pro11dup)

validation

patient specimen

Anonymized

Gene-specific

Variant range

Neuropathy gene panel

validation

patient specimen


#### **Table 3.** *Validation grid.*

diluted to 20 nM (in a total volume of 20 μl) and subjected to sequencing using an Illumina® HiSeq4000 Sequencing platform.

• Sequence analysis: performance metrics. Baseline performance metrics for the WES validation study must involve the analysis of well-characterized reference samples: the NIST sample (NA12878) and the SOPHiA™ Genetics control SG063. The sequence metrics for each sample in the run must be recorded and averages established using the reference samples. Samples must meet the sequencing metrics shown in **Table 4** in order to reach the threshold for clinical reporting.

Analytical sensitivity and specificity must be calculated separately for each variant type (SNV, indel, CNV, etc.). Additional runs may be required to meet acceptable confidence intervals for less frequent variant types of insertions and deletions. For 95% confidence and 95% reliability, 59 variants of each type (and insertion/deletion range) should be analyzed [5]. The variant types that do not have strong confidence intervals must be listed in the test limitations of the clinical report until such time that the desired confidence levels have been achieved.


**125**

**Table 5.**

*Quality management.*

*Clinical Validation of a Whole Exome Sequencing Pipeline*

The worksheets described by Santani et al. [4] set out very clear guidance for all quality aspects that need to be taken into consideration for the test to meet CAP requirements [4]. Through a validation study, the majority of a test's limitations will be discovered and can be recorded against the QC parameters. **Table 5** summarizes

**Section Category Criteria Specific requirement**

**Note that these may vary between tests and laboratories**

barcode, date of collection

weight DNA band

>7.5

<5%

>80% of fragments between 300 and 700 bp

>20 nM

>40×

All worksheets and transfers during bench work are witness checked for accurate specimen identification

Wrong specimen type Whole blood Wrong type of tube Purple top EDTA tube Insufficient quantity ≥0.5 ml Clotting (blood only) No visible clots Insufficient labelling Labelling contains name, DOB,

Expired specimen ≤7 days since collection Expired collection tube Collection tube not expired

Electrophoretic analysis Shows intact high molecular

Cluster density Not taken into account Base quality Q30 ≥ 80

OD 260/280 ratio >1.7

Quantification ≥500 ng

DNA integrity number (DIN)

% reads not assigned to any sample

> Fragment size and distribution

> > Pooled library concentration

Read alignment % Reads aligned to target >90%

Coverage 10% quantile (at this depth 90% target covered at x)

Accurate specimen identity, file names with 4 points of identification

Data transfer to secure analysis platform

Pipeline QC Total reads passing filter >280 M per lane

Control samples Positive control Expected variants found

% reads assigned to sample 8–12%

Distribution of coverage >95% within 25–200×

PCR duplicates <20%

*DOI: http://dx.doi.org/10.5772/intechopen.93251*

quality metrics that need to be addressed.

Specimen quality

DNA quality and quantity

Instrument run QC

> Library preparation

Sample de-multiplexing

> Specimen identity

Data transfer Integrity

**5. Quality management**

Pre-analytical QC (per sample)

Analytical QC (per instrument run)

Analytical QC (per sample)

**Table 4.** *Sequencing metrics.*
