**6. Precision**

Precision has typically been considered as the most important performance characteristic of the method, because it gives customers/clients of the analytical data direct information on the significance or uncertainly of results. Typically, method precision is established from replicate analyses of the same sample. However, methods for predicting precision have recently been published that allow the assessment of precision based on a single chromatogram (Apostol, Kelner et al. 2012).

Method precision defines the capability of the method expressed in its reporting interval (Holme and Peck 1998). Agut et al. (Agut, Segalini et al. 2006) examined different rules and their application to the reporting interval of results and specifications. The best known and simplest rule to implement is that stated in the AMST standard E-29-02. The rule states that the results of analytical measurements should be rounded to not less than 1/20 of the determined standard deviation (ASTM 2005).

For example, bioassays with a standard deviation of 11.8 should adopt a reporting interval larger than 0.59. However, this 0.59 reporting interval is impractical in day-to-day applications due to the inability of bioassays to provide precision that would justify such a reporting interval. Therefore, bioassays with a standard deviation of 11.8 would result in a reporting interval of 1. Similarly, an HPLC assay with a standard deviation of 1.3 for the main peak would result in a reporting interval of 0.1. Reporting intervals for impurities (minor peaks) need to be consistent with reporting intervals for the main peak. In general, STD of equal or less than 2 (in units reported by the method) is required to ensure a reporting interval of one decimal place. The argument can be raised that for low level, minor analytes (for example, the dimer in SEC present at 1%), the requirement for STD to be at or below 2% is too generous. This will result in an RSD of 200% for the peak. In such an instance, this would indicate that the minor peak is well below the detection level, because theoretically the RSD at the LOD level should not exceed 33% (Long and Winefordner 1983; Hayashi and Matsuda 1995)

The table below proposes the nearest reporting intervals based on standard deviations obtained during qualification for protein products.


**Table 2.** Recommended Nearest Reporting Results Based on Standard Deviation

Method precision is closely linked to the concentration of the analyte. The best-known relationship between analyte concentration and RSD is the Horwitz equation (Horwitz 1982; Horwitz and Albert 1997; Horwitz and Albert 1997)

$$\text{RSD} = 2^{(1\text{-}0.5\log\text{C})}$$

where, C is the concentration of the analyte in mg/g.

122 Analytical Chemistry

ICH Q2R1 method types

Testing for impurities

Method Type

**6. Precision** 

**5. Performance characteristics** 

experiments for protein products.

ICH Q6B method types

Purity and impurities

Identification Identity ID √

chromatogram (Apostol, Kelner et al. 2012).

determined standard deviation (ASTM 2005).

\*In some cases may be required by USP <1225>.

Industry method types

Process

In order to produce a reliable assessment of method performance, all necessary performance characteristics should be evaluated in carefully designed experiments. ICH guideline Q2R1 specifies which performance characteristics should be evaluated for validation. However, interpretation of the table for protein products is not straightforward. This is due in part to the fact that ICH Q2R1, Q6B and the industry used different nomenclature to describe the type of methods. The table below details the performance characteristics that should be assessed during qualification, and subsequently during the confirmatory validation

Quantity Titer √ √ √ √ √ \*

Reagents <sup>√</sup> <sup>√</sup> <sup>√</sup> <sup>√</sup> <sup>√</sup>

Assay Potency Potency √ √ √ √ √ √

**Table 1.** Performance Characteristics that need to be Evaluated During Qualification/Validation by

Precision has typically been considered as the most important performance characteristic of the method, because it gives customers/clients of the analytical data direct information on the significance or uncertainly of results. Typically, method precision is established from replicate analyses of the same sample. However, methods for predicting precision have recently been published that allow the assessment of precision based on a single

Method precision defines the capability of the method expressed in its reporting interval (Holme and Peck 1998). Agut et al. (Agut, Segalini et al. 2006) examined different rules and their application to the reporting interval of results and specifications. The best known and simplest rule to implement is that stated in the AMST standard E-29-02. The rule states that the results of analytical measurements should be rounded to not less than 1/20 of the

For example, bioassays with a standard deviation of 11.8 should adopt a reporting interval larger than 0.59. However, this 0.59 reporting interval is impractical in day-to-day applications

Purity √ √ √ √ √ √ Immuno √ √ √ √ √ √ DNA √ √ √ √ √ √ Peptide map √ √ √ Gels √ √ √ √ √ √

Performance characteristics Specificity Linearity Range Precision Accuracy LOD/LOQ

> Based on the Horwitz equation, the precision of the measurement, expressed as RSD, doubles for each decrease of analyte concentration of two orders of magnitude.

> The Horwitz relationship can provide good guidance for method precision targets during method development and qualification. Intermediate precision obtained during these studies should meet the variability derived from the Horwitz equation for each individual analyte. If, during execution of the qualification experiments, the precision of the measurements exceeds values derived from the Horwitz equation, this may indicate that the assay may need to be redeveloped, or that the technology utilized in the assay may not be fully suitable for the intended application.

> Typically, proteins are available for analysis as solutions, with concentrations ranging widely from 1 g/ml (e.g., a growth factor) to 100 mg/ml or higher (e.g., a monoclonal antibody). In such cases, expectations for the RSD of measurements of the main protein analyte in these solutions, based on the Horwitz relationship (e.g., using protein concentration method), will be 16 and 2.8 %, respectively.

As mentioned earlier, the precision of the method is referred to as uncertainty. The uncertainty of results is a parameter that describes a range within which the measured value is expected to lie (Miller and Miller 2000). Intuitively, we associate this parameter with precision. Therefore, method precision has been viewed as the most important performance characteristic. Typically, method precision has been assessed from replicate analyses of the same sample. The work of Hayashi and Matsuda on FUMAI theory (Hayashi, Rutan et al. 1993; Hayashi and Matsuda 1994; Hayashi and Matsuda 1994; Hayashi and Matsuda 1995; Hayashi, Matsuda et al. 2002; Hayashi, Matsuda et al. 2004) demonstrated that the precision of chromatographic methods can be predicted from noise and the height and width of the signal (peak). However, due to the complexity associated with the required Fourier transformation of chromatograms and the parameterization of the power spectrum called for in implementation of this theoretical construct to the determination of precision, the FUMAI theory approach has not been widely applied.

Analytical Method Validation for Biopharmaceuticals 125

uncertainty of results obtained early in the product lifecycle may not be fully applicable to results obtained later. Application of historical validation data always begs a question about the relevance of these data to the current experimental situation, and sometimes requires investigation, which can delay the approval of results. The UBCI approach, therefore, has the capability of providing not only simplicity, but also a greater level of assessment of the

The determination of accuracy for protein purity methods presents significant challenges. Since it is difficult to establish orthogonal methods for proteins to measure the same quality attribute, it is hard to assess the truthfulness of the accuracy measurements. For example, although SEC-HPLC results can be verified by analytical ultra centrifugation (AUC) techniques, these techniques are based on very different first principles, and may not provide comparable results (Carpenter, Randolph et al. 2010; Svitel, Gabrielson et al. 2011). Therefore, in most cases, the accuracy of purity methods for proteins is inferred when other performance characteristics meet expectations, which is consistent with the principles of

Linearity and range are typically assessed in a complex experiment demonstrating a linear change of peak area with analyte concentration. Since most of the methods use UV detection, such linearity experiments can be considered as re-confirmation of the Beer-

The specificity of analytical methods is typically assessed by examining system interference with the detection and quantification of analytes. Part of this evaluation is the determination of protein recovery from the column (Rossi, Pacholec et al. 1986; Eberlein 1995). The recovery determination requires the knowledge of the extinction coefficient for the protein, which can be calculated from its amino acid composition (Pace, Vajdos et al. 1995) or determined experimentally. It should be noted that the extinction coefficient of a protein may change as a function of pH (Eberlein 1995; Kendrick, Chang et al. 1997). Therefore, direct comparison of the recovery in the neutral pH, size exclusion method with the recovery in an acidic reversed-phase separation may not be valid due to differences in the operating pHs of the methods. The difference may not necessarily reflect the actual recovery, but rather shows pH dependent changes of spectroscopic properties of the protein. With such an approach, the specificity of the method can be assessed in every assay, and reflects dynamically the change in status of consumables (columns and mobile phases) and

data validity relative to current practices.

**7. Accuracy** 

ICH Q2R1(ICH 2005).

**9. Specificity** 

hardware.

**8. Linearity and range** 

Lambert law for the particular hardware configuration.

Apostol et al. (Apostol, Kelner et al. 2012) proposed a new approach to assessing the uncertainty of purity analyses that uses a more holistic approach that is called Uncertainty Based on Current Information (UBCI). The model allows for real-time assessment of all performance characteristics using the results of the specific separation of interest. A fundamental, underlying principle of this approach recognizes that the execution of a purity method is always associated with specific circumstances; therefore, uncertainty about the generated results needs to account for both the operational conditions of the method and the hardware. The authors demonstrated that noise levels, instrument and software settings can be linked directly to all method performance characteristics. Such simplification makes it easy to implement this procedure in a daily operation, and can provide a valuable live assessment of uncertainty instead of extrapolating uncertainty from historical qualification/validation studies.

The UBCI model approximates the maximal uncertainty of the measurement associated with the actual conditions of analysis (test). The obtained precision corresponds to the uncertainty under the most unfavorable conditions, including the highest variability of injection, maximal numeric integration error, expected variability of the peak width, and the most unfavorable contribution of the noise. UBCI shows that the uncertainty of results is not only a function of the method (composition of the mobile phase, gradient, flow rate, temperature), but also is influenced by the hardware associated with the execution of the method (pump pulsation, detector range, status of the lamp, etc.), and the software settings used to acquire the output in the form of chromatograms. Information about these parameters can be extracted from individual chromatograms; therefore, the assessment of method performance characteristics (uncertainty) can be performed realtime, which can be considered as a 'live validation' associated with each individual test result.

It is important to note that historical qualification/validation approaches do not take this fundamental principle into account, such that performance drift may occur over time due to hardware differences and even due to differences in analyst skill levels, such that the uncertainty of results obtained early in the product lifecycle may not be fully applicable to results obtained later. Application of historical validation data always begs a question about the relevance of these data to the current experimental situation, and sometimes requires investigation, which can delay the approval of results. The UBCI approach, therefore, has the capability of providing not only simplicity, but also a greater level of assessment of the data validity relative to current practices.
