3. Bias detection without comparators

medicine can be to run linearity experiments and limit the measurement range based on the linearity results. Despite this the effect of random variation on the regression line remains. To

The simplest weighting procedure is to use the standard deviation of variation for each data point of the method comparison study. This requires that the method comparison study is repeated multiple times (20-30 times). This allows us to calculate the standard deviation of measurement for each point (Si). The weighting coefficient will then be the inverse of this

> wi <sup>¼</sup> <sup>1</sup> Si

This weight can then be incorporated into the equations of the method comparison. For

Weighting can often considerably decrease the bias percentage especially at the extremes of measurement compared to non-weighted regression. Weighting by inverse of standard deviation tends to normalize the relative bias at the extremes of measurement while weighting by inverse of variance tends to favor the bias correction for lower ends of measurement (less bias at lower concentrations). The decision for weighting and/or choice of weighting procedure

To estimate the proportional bias, a recovery experiment is needed. The recovery experiments are performed by calculating the amount of recovery when adding a known amount of the analyte to the sample: this is done by dividing the measurement sample into two equal aliquots and performing the measurement for both aliquots. To one of the aliquots, a known amount of target analyte is added (aliquot 1). For the other aliquot (aliquot 2) an equal amount of diluent is added and the measurement is repeated. The recovery percentage can then be

Amount of analyte added to aliquote 1

The recovery or bias percentage is often used in laboratory medicine to state the proportional bias. Most of the regulatory agencies have set critical values for the recovery percentage for different analytes. The advantage of using recovery percentage is that it normalizes to 100

should be based on the assay characteristics and performance requirements [13].

Recovery% <sup>¼</sup> <sup>ð</sup>Analyte amount in aliquot <sup>1</sup>Þ � ð Þ Analyte amount in aliquout <sup>2</sup>

allowing for easier understanding of the scale of bias present [2].

<sup>P</sup>wi Xi � <sup>X</sup> � � Yi � <sup>Y</sup> � � <sup>P</sup>wi Xi � <sup>X</sup> � �<sup>2</sup> <sup>P</sup>wi Yi � <sup>Y</sup> � �<sup>2</sup> � �<sup>1</sup>

2

(27)

(28)

� 100 (29)

rectify this, a solution is to employ a weighting procedure.

example, the r coefficient can be recalculated as:

r ¼

standard deviation:

60 Quality Control in Laboratory

2.8. Recovery percentage

calculated:

Up to this point we have discussed bias detection methods that use a reference material or comparator to assess the presence of bias. While this has been the accepted standard for many laboratory regulatory agencies, there are arguments against this approach to bias detection: first of all, the assumption of method comparison studies is that the reference material (control samples) values are true and do not suffer from imprecisions. The measurement uncertainty is considered to be minimal in these samples. Yet, unless these samples vary considerably from the biologic sample matrix, a degree of measurement uncertainty would exist in these samples which lead to inaccurate estimates of bias and imprecision of laboratory instruments and techniques. On the other hand, running repeated control samples with each run and the need for revalidation of the instrument and techniques after each change in the parameters, requires a considerable investment in terms of time, labor and cost.

Alternatively, the systematic error can be determined by using the patient samples. This can be done by either tracking the results of known normal patients (i.e. those expected to have a result within the reference range based on their clinical and physiologic state) or by following the trend of all the results of an analyte over time. Using patient samples has the advantage of including the inherent biologic uncertainty into the calculation of bias.

#### 3.1. Average of normal (AON)

In this approach the comparator for quality control would the average values of the analyte in normal individuals. This requires us to know the population average and standard deviation for that analyte. If we measure the analyte in a normal individual, we would expect the results to approximate the population average. Deviations of the normal results from the expected reference normal can signal the presence of a systematic error.

In AON, the mean value of normal samples is compared to a mean reference value. The mean reference value should be established by the laboratory based on the population it serves; this is best done as part of the initial validation of an assay when a large size sample of normal individuals is tested to establish the reference ranges. This experiment allows us to calculate the population mean, standard deviation and standard error (SD/√N). We expect the Average of Normals from our analytical run to fall within the 95% confidence interval of the population mean.

$$95\% \text{CI} = \text{Population Mean} \pm 1.96 \text{ Standard Error} \tag{30}$$

With each analytical run, a sample of normal results should be used to calculate the Average of Normals for that analytical run. If the calculate average is beyond the 95% CI of the population then we have detected a systematic error in the analytical run.

In AON method, as the size of the normal sample increases the probability of detecting bias also increases. The size calculations for the AON method are determined by the ratio of the biological variance of the target analyte (CVb) to the variance of the method (CVa) (CVb/CVa) as well the expected probability of detecting the bias. To help with these calculations, one can utilize the Cembrowski nomogram [14] or, alternatively, the methods used in [15]. It is also possible to perform the AON by performing a two-sample independent t-test.

#### 3.2. Moving patient averages

Unlike the AON method, in moving patient averages, all the results of an assay are included in evaluation of bias. The principle for moving patient averages is that the samples tested in a laboratory follow a repeating pattern. This assumption means that the overall biologic and clinical spectrum of patients and individuals tested in the laboratory is constant throughout the analytical runs. In moving patient averages, we expect the average results of an assay for two overlapping subsets of patient to be constant. In this method, for example, an average is calculated on the first 100 patients, should be similar to the average calculated based on the results of patients number 2 to 101, etc.

The moving average can be calculated using exponentially weighted moving average (XM,i). It is important to consider that, in moving patient averages the weight 1ð Þ � r assigned to previous results average (XM,i�1) should be greater than the weight (r) assigned to the most recent results (Xi) (in other words the average of each batch is weighted down by previous averages). This can be stated as:

$$
\overline{X}\_{M,i} = r\overline{X}\_i + (1-r)\overline{X}\_{M,i-1} \tag{31}
$$

from the value of current measurements in the batch. If we assume a value of 1 for r then we

P N j¼1

The control limits of Bull's moving average are set as Xb,<sup>0</sup> � 3%Xb, 0, with Xb, <sup>0</sup> being the target

The advantage of moving averages is that they can filter out outliers' effect thus removing

The moving patient averages algorithms are very powerful for detection of bias: they can routinely identify bias percentages of 1% and more. Most automated hematology analyzers use moving patient averages to check for presence of bias in their assays. However, the patient moving averages algorithms have suffered from implementation problems and are not widely

An extension of the moving patient averages is the application of time series analysis and forecasting for bias detection. In time series analysis the previous trends of the analyte results are used to predict (forecast) the trend in future. If the observed analyte results deviate from the forecasted trend, then a measurement error may exist. In the setting of laboratory medicine, we need to be able to detect bias in short time series and distinguish the measurement error from the noise and chaos stemming from biologic variation. Here, we will introduce the concept of using time series analysis for bias detection but we will not explain the methodol-

In forecast models, a series of data points are used to create one or more projection patterns for future trends. This is done using forecasting models such as ARIMA (Autoregressive integrated moving average). These projections are often correct for very short-term predictions (next 1 or 2 data points), but for forecasting further, the noise and chaos cause the prediction accuracy to fall. However, by examining the correlation of predicted and observed values and documenting its changes as we forecast further into the future, we can determine if the observed pattern represents the deterministic chaotic nature of biologic measurement or if it represents a measurement error; for measurement error we expect the correlation coefficient to remain constant with time; however, with chaos, we expect the correlation coefficient to

There are other approaches using times series analysis that can be helpful in systematic error identification. One of these approaches uses unit root tests such as the Dickey-Fuller test [17]. These tests examine whether a time series is stationary over time, i.e., whether the mean and

0

BBB@

p

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Xj � Xb,i�<sup>1</sup>

1

2

Systematic Error Detection in Laboratory Medicine http://dx.doi.org/10.5772/intechopen.72311

(34)

63

CCCA

N

Xb,i ¼ Xb,i�<sup>1</sup> þ

3.3. Time series analysis and forecasting for bias identification

ogy in depth as it goes beyond the scope of this chapter.

can write the bull's algorithm as:

value for that analyte.

confounding by imprecision.

deteriorate over time [16].

used beyond hematology analyzers [2].

where N is the number of results in the batch.

The weight assigned to current values is usually set between 0.05 and 0.25 with recommended value of 0.1.

The comparator in moving patient averages are the control limits. We expect the weighted patient average to fall within the control limits for that test. Any moving patient average outside of this control limit signifies the presence of a bias. The control limit equation is provided below.

$$\text{Control limits of exponential moving average} = \overline{\mathbf{X}}\_{\mathbf{M},0} \pm L\sigma \sqrt{\left| \frac{r}{2-r} \left[ 1 - \left( 1 - r \right)^{2i} \right] \right|} \tag{32}$$

where L is a constant set based on the confidence level (for 95% CI, L equals 2), and σ is the standard deviation of the current batch.

The moving patient averages can also be evaluated using the Bull's algorithm. In this approach, the moving average (Xb) is calculated for subsets of 20 samples with 19 patient values and one value representing the previous moving average. These values are weighted differently (i.e. more weight is assigned to the previous moving average than the 19 new samples).

The general formula for Bull's moving average can be written as:

$$
\overline{X}\_{b,i} = (2 - r)\overline{X}\_{b,i-1} + rD \tag{33}
$$

where Xb,i is the current moving average, r is the weight for current values (with possible values of 0 < r ≤ 1, usually set to 1), Xb,i�<sup>1</sup> is the previous moving average and D is calculated from the value of current measurements in the batch. If we assume a value of 1 for r then we can write the bull's algorithm as:

$$\overline{X}\_{b,i} = \overline{X}\_{b,i-1} + \left(\frac{\sum\_{j=1}^{N} \sqrt{X\_j - X\_{b,i-1}}}{N}\right)^2 \tag{34}$$

where N is the number of results in the batch.

as well the expected probability of detecting the bias. To help with these calculations, one can utilize the Cembrowski nomogram [14] or, alternatively, the methods used in [15]. It is also

Unlike the AON method, in moving patient averages, all the results of an assay are included in evaluation of bias. The principle for moving patient averages is that the samples tested in a laboratory follow a repeating pattern. This assumption means that the overall biologic and clinical spectrum of patients and individuals tested in the laboratory is constant throughout the analytical runs. In moving patient averages, we expect the average results of an assay for two overlapping subsets of patient to be constant. In this method, for example, an average is calculated on the first 100 patients, should be similar to the average calculated based on the

The moving average can be calculated using exponentially weighted moving average (XM,i). It is important to consider that, in moving patient averages the weight 1ð Þ � r assigned to previous results average (XM,i�1) should be greater than the weight (r) assigned to the most recent results (Xi) (in other words the average of each batch is weighted down by previous

The weight assigned to current values is usually set between 0.05 and 0.25 with recommended

The comparator in moving patient averages are the control limits. We expect the weighted patient average to fall within the control limits for that test. Any moving patient average outside of this control limit signifies the presence of a bias. The control limit equation is provided below.

where L is a constant set based on the confidence level (for 95% CI, L equals 2), and σ is the

The moving patient averages can also be evaluated using the Bull's algorithm. In this approach, the moving average (Xb) is calculated for subsets of 20 samples with 19 patient values and one value representing the previous moving average. These values are weighted differently (i.e. more

where Xb,i is the current moving average, r is the weight for current values (with possible values of 0 < r ≤ 1, usually set to 1), Xb,i�<sup>1</sup> is the previous moving average and D is calculated

Control limits of exponential moving average ¼ XM,<sup>0</sup> � Lσ

weight is assigned to the previous moving average than the 19 new samples).

The general formula for Bull's moving average can be written as:

XM,i ¼ rXi þ ð Þ 1 � r XM,i�<sup>1</sup> (31)

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi r

1 � ð Þ 1 � r <sup>2</sup><sup>i</sup> � h i �

� � �

(32)

2 � r

�

Xb,i ¼ ð Þ 2 � r Xb,i�<sup>1</sup> þ rD (33)

r

possible to perform the AON by performing a two-sample independent t-test.

3.2. Moving patient averages

62 Quality Control in Laboratory

results of patients number 2 to 101, etc.

averages). This can be stated as:

standard deviation of the current batch.

value of 0.1.

The control limits of Bull's moving average are set as Xb,<sup>0</sup> � 3%Xb, 0, with Xb, <sup>0</sup> being the target value for that analyte.

The advantage of moving averages is that they can filter out outliers' effect thus removing confounding by imprecision.

The moving patient averages algorithms are very powerful for detection of bias: they can routinely identify bias percentages of 1% and more. Most automated hematology analyzers use moving patient averages to check for presence of bias in their assays. However, the patient moving averages algorithms have suffered from implementation problems and are not widely used beyond hematology analyzers [2].

#### 3.3. Time series analysis and forecasting for bias identification

An extension of the moving patient averages is the application of time series analysis and forecasting for bias detection. In time series analysis the previous trends of the analyte results are used to predict (forecast) the trend in future. If the observed analyte results deviate from the forecasted trend, then a measurement error may exist. In the setting of laboratory medicine, we need to be able to detect bias in short time series and distinguish the measurement error from the noise and chaos stemming from biologic variation. Here, we will introduce the concept of using time series analysis for bias detection but we will not explain the methodology in depth as it goes beyond the scope of this chapter.

In forecast models, a series of data points are used to create one or more projection patterns for future trends. This is done using forecasting models such as ARIMA (Autoregressive integrated moving average). These projections are often correct for very short-term predictions (next 1 or 2 data points), but for forecasting further, the noise and chaos cause the prediction accuracy to fall. However, by examining the correlation of predicted and observed values and documenting its changes as we forecast further into the future, we can determine if the observed pattern represents the deterministic chaotic nature of biologic measurement or if it represents a measurement error; for measurement error we expect the correlation coefficient to remain constant with time; however, with chaos, we expect the correlation coefficient to deteriorate over time [16].

There are other approaches using times series analysis that can be helpful in systematic error identification. One of these approaches uses unit root tests such as the Dickey-Fuller test [17]. These tests examine whether a time series is stationary over time, i.e., whether the mean and variance are constant over time. In contrast, nonstationary time series will have either a varying mean and/or varying variance over time. Using this approach any departure from stationarity can signal either a drift (proportional bias) and/or a shift (constant bias) or even increase in imprecision over time (difference-stationary nonstationarity) [17]. If the Dickey-Fuller test returns a significant p-value then we can say that the series is stationary, and no significant measurement error is present.

[10] Hanneman SK. Design, analysis and interpretation of method-comparison studies. AACN Advanced Critical Care. Oxford, UK and Waltham, Mass, USA. 2008;19(2):223

Systematic Error Detection in Laboratory Medicine http://dx.doi.org/10.5772/intechopen.72311 65

[11] Guidelines for Quality Management in Soil and Plant Laboratories. No. 74. Food &

[12] Mermet JM, Granier G. Potential of accuracy profile for method validation in inductively coupled plasma spectrochemistry. Spectrochimica Acta Part B: Atomic Spectroscopy.

[13] Mermet JM. Calibration in atomic spectrometry: A tutorial review dealing with quality criteria, weighting procedures and possible curvatures. Spectrochimica Acta Part B:

[14] Cembrowski GS et al. Assessment of "average of normals" quality control procedures and guidelines for implementation. American Journal of Clinical Pathology. 1984;81(4):492-499

[15] Westgard JO, Smith FA, Mountain PJ, Boss S. Design and assessment of average of normals (AON) patient data algorithms to maximize run lengths for automatic process

[16] Sugihara G, May RM. Nonlinear forecasting as a way of distinguishing chaos from mea-

[17] Cheung YW, Lai KS. Lag order and critical values of the augmented Dickey–Fuller test.

Agriculture Org.; 1998

Atomic Spectroscopy. 2010;65(7):509-523

control. Clinical Chemistry. 1996;42(10):1683-1688

surement error in a data series. Nature. 1990;344:734-741

Journal of Business & Economic Statistics. 1995;13(3):277-280

2012;76:214-220
