**4.3 Identify the components of uncertainty**

From equation 14 we see four components that contribute to the combined uncertainty in the corrected BAC. These include: (1) the original duplicate measurement results of the blood alcohol concentration, (2) the reference value (R) representing a traceable unbiased control standard purchased from a commercial laboratory having a certificate of analysis, (3) the mean of the replicate measurements *X* of the traceable control and (4) the correction factor *fdilutor* for the dilutor used in preparing both the controls and blood samples before analysis. We will assume 1 *dilutor f* .

Measurement Uncertainty in Forensic Toxicology: Its Estimation, Reporting and Interpretation 425

C0 (0.082, 0.081 g/dL)0.0815 g/dL A 0.00072g/dL 2 R 0.100 g/dL B 0.0004 g/dL 1 *X* 0.0986 g/dL A 0.0008 g/dL 8 *dilutor f* 10.15 ml B 0.050 ml 10

**Uncertainty <sup>n</sup>**

**Parameter Values Type Standard** 

Table 1. Estimates, standard uncertainties and the number of measurements for the four

Fig. 5. An uncertainty function plotting pooled standard deviation estimates against their

**Blood Alcohol Concentration (g/100ml) 0.0 0.1 0.2 0.3 0.4 0.5**

**SD = 0.0038BAC + 0.000447 n = 13,159 duplicates**

concentration determined from a large number of duplicate blood alcohol results

propagation method of equation 8 yield nearly identical results.

**4.5 Combine the standard uncertainties and compute the combined uncertainty**  We first determine our combined uncertainty using the general method of error propagation found in equation 8 assuming independence amongst the predictor variables. Putting our values determined from equation 16 into equation 8 we obtain equation 17. Since our measurement function is multiplicative we also estimate our combined uncertainty using equation 6 and assuming independence we obtain equation 18. Notice that we have included the actual estimate for fdilutor of 10.15 ml. This will ensure the appropriate value is determined for the uncertainty of the dilutor component. For purposes of bias correction in the measurement function of equation 14, however, we assume the value of fdilutor = 1.0. From equations 17 and 18 we see that both the RSS method of equation 6 and the error

parameters assumed to contribute to the combined uncertainty of blood alcohol

**Standard Deviation (g/100ml)**

**0.000**

**0.001**

**0.002**

**0.003**

**0.004**

**0.005**

measurement

#### **4.4 Quantify the standard uncertainties for the components**

For our example we will assume the values for the four parameters are those shown in Table 1. The uncertainty for the reference value (R) is a Type B uncertainty which comes from the certificate of analysis provided by the vendor preparing the control standard. The uncertainty for the replicate measurements of the control standard is simply the standard deviation determined from n=8 measurements of the control standard. The uncertainty for the dilutor was determined from the certificate of analysis. Since the dilutor is designed to provide 10 ml volume we see a small bias exists. This is not corrected for since the same bias would influence both the control standard measurements as well as the blood samples. For this reason we assume 1 *dilutor f* . The actual value of the *dilutor f* in table 1 (10.15ml), however, will be used to estimate its uncertainty. The uncertainty associated with the blood alcohol results reported in table 1 (0.00072 g/dL) requires some further explanation. The uncertainty associated with these BAC results represents total method uncertainty. This estimate will be determined from a large number of duplicate BAC results generated within the same laboratory over a long period of time (approximately one year). This would include variation from sample preparation, multiple instruments, multiple calibrations, multiple analysts, multiple uses of the dilutor and time. Figure 5 illustrates an uncertainty function generated from duplicate blood alcohol data analyzed in the forensic laboratory of New Zealand. (Stowell, e.tal., 2008) For illustration purposes, we will assume this model is relevant to our example. Each point in the plot represents the standard deviation associated with a single determination and is generated from the following equation for a pooled estimate:

$$
\mu\_B = \sqrt{\frac{\sum\_{i=1}^k d\_i^2}{2k}} \tag{15}
$$

where: uB = the standard deviation for a single measurement of blood alcohol concentration di = the difference between duplicate results for the ith sample

k = the total number of duplicate samples within the bin

Duplicate results are pooled into bins of 0.010 g/dL to generate the uncertainty estimates throughout the concentration range. The result is an estimate of the uncertainty as a function of concentration and reveals the general increase in variation with concentration. Some would advocate the use of a characteristic function rather than an uncertainty function. (Thompson and Coles, 2011) A characteristic function is generated from regressing the variance against the concentration squared. Before estimating our method uncertainty from these functions, we need to determine our corrected BAC result. This is done as follows:

$$\mathbf{C}\_{corr} = \frac{\mathbf{C}\_0 \, \mathbf{R}}{\overline{X}} \cdot f\_{\text{dilutor}} = \frac{(0.0815)(0.100)}{(0.0986)} \cdot \mathbf{1} \, \mathbf{1} = 0.0827 \, \mathbf{g} \, / \, \text{dL} \tag{16}$$

We now use this corrected result to estimate our method uncertainty from the model in figure 5. Based on the linear uncertainty function in figure 5 we obtain a method uncertainty of 0.00076 g/dL. Developing the characteristic function for the same data set yields a method uncertainty estimate of 0.00072 g/dL. Therefore, we will use the value of 0.00072 g/dL for example, as we see in table 1.


Table 1. Estimates, standard uncertainties and the number of measurements for the four parameters assumed to contribute to the combined uncertainty of blood alcohol measurement

#### **Standard Deviation (g/100ml)**

424 Toxicity and Drug Testing

For our example we will assume the values for the four parameters are those shown in Table 1. The uncertainty for the reference value (R) is a Type B uncertainty which comes from the certificate of analysis provided by the vendor preparing the control standard. The uncertainty for the replicate measurements of the control standard is simply the standard deviation determined from n=8 measurements of the control standard. The uncertainty for the dilutor was determined from the certificate of analysis. Since the dilutor is designed to provide 10 ml volume we see a small bias exists. This is not corrected for since the same bias would influence both the control standard measurements as well as the blood samples. For this reason we assume 1 *dilutor f* . The actual value of the *dilutor f* in table 1 (10.15ml), however, will be used to estimate its uncertainty. The uncertainty associated with the blood alcohol results reported in table 1 (0.00072 g/dL) requires some further explanation. The uncertainty associated with these BAC results represents total method uncertainty. This estimate will be determined from a large number of duplicate BAC results generated within the same laboratory over a long period of time (approximately one year). This would include variation from sample preparation, multiple instruments, multiple calibrations, multiple analysts, multiple uses of the dilutor and time. Figure 5 illustrates an uncertainty function generated from duplicate blood alcohol data analyzed in the forensic laboratory of New Zealand. (Stowell, e.tal., 2008) For illustration purposes, we will assume this model is relevant to our example. Each point in the plot represents the standard deviation associated with a single determination and is generated from the following equation for a pooled

> 2 1 2

(15)

*d*

*k* 

*k i i*

where: uB = the standard deviation for a single measurement of blood alcohol concentration

Duplicate results are pooled into bins of 0.010 g/dL to generate the uncertainty estimates throughout the concentration range. The result is an estimate of the uncertainty as a function of concentration and reveals the general increase in variation with concentration. Some would advocate the use of a characteristic function rather than an uncertainty function. (Thompson and Coles, 2011) A characteristic function is generated from regressing the variance against the concentration squared. Before estimating our method uncertainty from these functions, we need to determine our corrected BAC result. This is done as follows:

> <sup>0</sup> 0.0815 0.100 1 0.0827 / 0.0986 *corr dilutor*

We now use this corrected result to estimate our method uncertainty from the model in figure 5. Based on the linear uncertainty function in figure 5 we obtain a method uncertainty of 0.00076 g/dL. Developing the characteristic function for the same data set yields a method uncertainty estimate of 0.00072 g/dL. Therefore, we will use the value of 0.00072

*C R C f g dL <sup>X</sup>* (16)

*B*

*u*

di = the difference between duplicate results for the ith sample k = the total number of duplicate samples within the bin

g/dL for example, as we see in table 1.

**4.4 Quantify the standard uncertainties for the components** 

estimate:

Fig. 5. An uncertainty function plotting pooled standard deviation estimates against their concentration determined from a large number of duplicate blood alcohol results

#### **4.5 Combine the standard uncertainties and compute the combined uncertainty**

We first determine our combined uncertainty using the general method of error propagation found in equation 8 assuming independence amongst the predictor variables. Putting our values determined from equation 16 into equation 8 we obtain equation 17. Since our measurement function is multiplicative we also estimate our combined uncertainty using equation 6 and assuming independence we obtain equation 18. Notice that we have included the actual estimate for fdilutor of 10.15 ml. This will ensure the appropriate value is determined for the uncertainty of the dilutor component. For purposes of bias correction in the measurement function of equation 14, however, we assume the value of fdilutor = 1.0. From equations 17 and 18 we see that both the RSS method of equation 6 and the error propagation method of equation 8 yield nearly identical results.

Measurement Uncertainty in Forensic Toxicology: Its Estimation, Reporting and Interpretation 427

freedom for a probability distribution formed from several independent normal

4 4

*C*

*u u*

*i i i* (20)

*g dL*

(21)

1

The uncertainty terms <sup>4</sup> *ui* can be determined either from the coefficients of variation (CV) or from partial derivatives determined from the measurement function in equation 14. If the CV estimates are used we do not incorporate the sample size n for each term. We will determine the CV estimates for our example. We first compute the combined uncertainty

2 2 22 0.00072 0.0004 0.0008 0.050 0.0011 / 0.0827 0.0815 0.100 0.0986 10.15

4 4 4 44

0.0011

 

0.00072 0.0004 0.0008 0.050 0.0815 0.100 0.0986 10.15 1 7 9

From this computation we see that the effective degrees of freedom can be some non-integer value, in which case the value is generally truncated. Notice also that the uncertainty associated with the reference value (R) has an infinite number of degrees of freedom. This is because it is a Type B uncertainty determined from a certificate of analysis where we assume the uncertainty in the uncertainty estimate (0.0004 g/dL) is zero with correspondingly large degrees of freedom. As a result this term disappears from the computation. Each of the other degrees of freedom is determined from n-1. From these results we would estimate our value from the t-distribution to be: 0.975, 4 *t* 2.776 for estimating a 95% uncertainty interval. Using these results along with our combined uncertainty determined from equation 18 we would obtain a 95% uncertainty interval of:

*Y ku <sup>C</sup>* 0.0827 2.776 0.00067 0.0827 0.0019 0.0808 0.0846 / *to g dL* .

4

0.0827 4.6 4

*eff k*

 

distributions as in equation 20. (Ballico, 2000, Kirkup and Frenkel, 2006)

k = the number of components contributing to the combined uncertainty

2 2 2 2

*dilutor C C R X f Corr dilutor*

Next, we incorporate these results into equation 20 as follows:

*u u u u u C CR f X*

where: veff = the effective degrees of freedom

0

0

<sup>4</sup> *ui* the uncertainty associated with the ith component

<sup>4</sup> *uC* the combined uncertainty

again as in equation 21.

*C*

4

*C*

*u u*

*i i i*

1

*u*

*eff k*

 0 2 2 2 2 2 2 22 0 0 0 2 <sup>2</sup> 2 2 22 2 0.100 0.00072 0.0815 0.0004 0.0815 0.100 11 1 0.0986 2 0.0986 1 0.0986 *Corr dilutor Corr C dilutor C dilutor R X f C <sup>R</sup> <sup>C</sup> CR CR u fu fu u u X X <sup>X</sup> <sup>X</sup> u* 2 2 <sup>2</sup> 0.0008 0.0815 0.100 0.050 8 0.0986 10 0.00065 / *CCorr u g dL* (17)

$$\frac{u\_{\overline{Y}}}{\overline{Y}} = \sqrt{\left[\frac{u\_{\mathbb{C}\_{0}}}{\overline{C}\_{0}}\right]^{2} + \left[\frac{u\_{R}}{\overline{M}}\right]^{2} + \left[\frac{u\_{\overline{X}}}{\overline{X}}\right]^{2} + \left[\frac{u\_{f\_{\text{dulator}}}}{\overline{X}}\right]^{2}}$$

$$\frac{\mu\_{\overline{Y}}}{0.0827} = \sqrt{\left[\frac{\frac{0.00072}{\sqrt{2}}}{0.0815}\right]^2 + \left[\frac{\frac{0.0004}{\sqrt{1}}}{0.100}\right]^2 + \left[\frac{\frac{0.0008}{\sqrt{8}}}{0.0986}\right]^2 + \left[\frac{\frac{0.050}{\sqrt{10}}}{10.15}\right]^2} \tag{18}$$

 0.0827 0.0081 0.00067 / *Yu g dL* 

#### **4.6 Compute the expanded uncertainty and uncertainty interval**

The expanded uncertainty is denoted by the value U and is determined from: *U ku <sup>C</sup>* where k = a coverage factor and uC = the combined uncertainty. The expanded uncertainty is then used to generate an uncertainty interval as

$$
\overline{Y} \pm k\mu\_c \quad \Rightarrow \quad \overline{Y} \pm \mathcal{U} \tag{19}
$$

where: *Y* the unbiased mean measurement result, k = the coverage factor and U = the expanded uncertainty. Notice that *uC* is actually the standard deviation of the mean. This results from the fact that we included the appropriate sample sizes, where available, for each term in equations 17 and 18. Sample size also determines degrees of freedom and whether the normal distribution can be assumed or if the t-distribution should be employed. Sample size should be determined as part of the measurement design to ensure sufficient quality control and statistical power. Coverage factors of k=2 or k=3 are common and represent approximately 95% and 99% uncertainty intervals respectively. Selecting k=2 or 3 assumes large degrees of freedom (sample size ≥ 30). Sample sizes less than 30 should employ the Students t distribution. From table 1 we see that none of the sample sizes exceed ten. However, we could argue that the method uncertainty associated with the duplicate blood alcohol results (0.00072 g/dL), determined from the data in figure 5, was generated from over 11,000 duplicate blood alcohol results. This should clearly justify the use of k=2 or 3 for approximate estimates of the 95% and 99% expanded uncertainty intervals. For our present example, however, we will assume we have the limited number of observations noted in table 1 and illustrate the calculation of what is called the "effective degrees of freedom", which may be necessary in some forensic contexts. For this purpose we employ the Welch-Satterthwaite equation which assumes the estimation of the effective degrees of freedom for a probability distribution formed from several independent normal distributions as in equation 20. (Ballico, 2000, Kirkup and Frenkel, 2006)

$$\nu\_{\it eff} = \frac{\mu\_{\it C}^4}{\sum\_{i=1}^k \frac{\mu\_i^4}{\nu\_i}} \tag{20}$$

where: veff = the effective degrees of freedom

<sup>4</sup> *uC* the combined uncertainty

426 Toxicity and Drug Testing

2 2 2 2

*dilutor C R X f*

*u u u u u nnn n Y X CR f*

0.0827 0.0815 0.100 0.0986 10.15

The expanded uncertainty is denoted by the value U and is determined from: *U ku <sup>C</sup>* where k = a coverage factor and uC = the combined uncertainty. The expanded uncertainty

where: *Y* the unbiased mean measurement result, k = the coverage factor and U = the expanded uncertainty. Notice that *uC* is actually the standard deviation of the mean. This results from the fact that we included the appropriate sample sizes, where available, for each term in equations 17 and 18. Sample size also determines degrees of freedom and whether the normal distribution can be assumed or if the t-distribution should be employed. Sample size should be determined as part of the measurement design to ensure sufficient quality control and statistical power. Coverage factors of k=2 or k=3 are common and represent approximately 95% and 99% uncertainty intervals respectively. Selecting k=2 or 3 assumes large degrees of freedom (sample size ≥ 30). Sample sizes less than 30 should employ the Students t distribution. From table 1 we see that none of the sample sizes exceed ten. However, we could argue that the method uncertainty associated with the duplicate blood alcohol results (0.00072 g/dL), determined from the data in figure 5, was generated from over 11,000 duplicate blood alcohol results. This should clearly justify the use of k=2 or 3 for approximate estimates of the 95% and 99% expanded uncertainty intervals. For our present example, however, we will assume we have the limited number of observations noted in table 1 and illustrate the calculation of what is called the "effective degrees of freedom", which may be necessary in some forensic contexts. For this purpose we employ the Welch-Satterthwaite equation which assumes the estimation of the effective degrees of

0

*Y*

*Y*

*u*

*Corr*

0.00065 / *CCorr u g dL*

*C*

*u*

2 2 2 2 2 2 22 0 0 0 2 <sup>2</sup> 2 2 22

0.100 0.00072 0.0815 0.0004 0.0815 0.100 11 1

 

0.0986 2 0.0986 1 0.0986

0

0

**4.6 Compute the expanded uncertainty and uncertainty interval** 

is then used to generate an uncertainty interval as

0.0827 0.0081 0.00067 / *Yu g dL*

*Corr dilutor*

*C dilutor C dilutor R X f*

*<sup>R</sup> <sup>C</sup> CR CR u fu fu u u X X <sup>X</sup> <sup>X</sup>*

 

2 2 <sup>2</sup> 0.00072 0.0004 0.0008 0.050 2 1 8 10

2

*dilutor*

*Y ku Y U <sup>c</sup>* (19)

 2 2 <sup>2</sup> 0.0008 0.0815 0.100 0.050 8 0.0986 10

(17)

2

(18)

  <sup>4</sup> *ui* the uncertainty associated with the ith component

k = the number of components contributing to the combined uncertainty

The uncertainty terms <sup>4</sup> *ui* can be determined either from the coefficients of variation (CV) or from partial derivatives determined from the measurement function in equation 14. If the CV estimates are used we do not incorporate the sample size n for each term. We will determine the CV estimates for our example. We first compute the combined uncertainty again as in equation 21.

$$\begin{aligned} \frac{u\_{\text{C}}}{\text{C}\_{\text{Corr}}} &= \sqrt{\left[\frac{u\_{\text{C}\_{0}}}{\text{C}\_{0}}\right]^{2} + \left[\frac{u\_{R}}{R}\right]^{2} + \left[\frac{u\_{\overline{X}}}{\overline{X}}\right]^{2} + \left[\frac{u\_{f\_{\text{diar}}}}{f\_{\text{diar}}}\right]^{2}} \\\\ \frac{u\_{\text{C}}}{0.0827} &= \sqrt{\left[\frac{0.00072}{0.0815}\right]^{2} + \left[\frac{0.0004}{0.100}\right]^{2} + \left[\frac{0.0008}{0.0986}\right]^{2} + \left[\frac{0.050}{10.15}\right]^{2}} \\ &= 0.0011 \text{g } / \text{d} \text{L} \end{aligned} \tag{21}$$

Next, we incorporate these results into equation 20 as follows:

$$\nu\_{\text{eff}} = \frac{\mu\_{\text{C}}^{4}}{\sum\_{i=1}^{k} \frac{\mu\_{i}^{4}}{\nu\_{i}}} = \frac{\left[\frac{0.0011}{0.0827}\right]^{4}}{\left[\frac{0.00072}{0.0815}\right]^{4} + \left[\frac{0.0004}{0.100}\right]^{4} + \left[\frac{0.0008}{0.0986}\right]^{4} + \left[\frac{0.050}{10.15}\right]^{4}} = 4.6 \approx 4$$

From this computation we see that the effective degrees of freedom can be some non-integer value, in which case the value is generally truncated. Notice also that the uncertainty associated with the reference value (R) has an infinite number of degrees of freedom. This is because it is a Type B uncertainty determined from a certificate of analysis where we assume the uncertainty in the uncertainty estimate (0.0004 g/dL) is zero with correspondingly large degrees of freedom. As a result this term disappears from the computation. Each of the other degrees of freedom is determined from n-1. From these results we would estimate our value from the t-distribution to be: 0.975, 4 *t* 2.776 for estimating a 95% uncertainty interval. Using these results along with our combined uncertainty determined from equation 18 we would obtain a 95% uncertainty interval of: *Y ku <sup>C</sup>* 0.0827 2.776 0.00067 0.0827 0.0019 0.0808 0.0846 / *to g dL* .

Measurement Uncertainty in Forensic Toxicology: Its Estimation, Reporting and Interpretation 429

One of the most important, yet often overlooked, elements of determining measurement uncertainty is reporting the results. A great deal of thought should be given to this aspect of measurement. The end-user should be consulted to determine exactly what is needed for their application. There should be sufficient information so the results and their associated uncertainty are fully interpretable and unequivocal for a specific application without reference to additional documentation. This will necessitate some textual explanation in addition to the numerical results. One possibility for our blood alcohol example above is: *The duplicate whole blood alcohol results were 0.082 and 0.081 g/dL with a corrected mean result of 0.0827 g/dL. An expanded combined uncertainty of 0.0019 g/dL assuming a coverage factor of k=2.776 with an effective degrees-of-freedom of 4 and a normal distribution was generated from four principle components contributing to the uncertainty. An approximate 95% confidence interval for* 

In addition to the statement, a figure similar to that of figure 3 could be provided which might assist the court in placing the results in some geometric perspective. The format for reporting the results should be considered flexible. As time goes on there will no doubt be

There were a number of assumptions employed in estimating the uncertainty illustrated above. The customer should appreciate these assumptions to allow for full and clear

3. The method uncertainty is probably over estimated due to some "double counting"

6. The confidence interval expresses the uncertainty due to sampling variability only

We would not advocate that these assumptions be listed as part of the reported results. Rather, they should be available if requested by the end-user and toxicologists should be

Our next example illustrates the uncertainty estimation for a breath alcohol measurement. We will assume the following measurement function which was presented earlier as

<sup>0</sup> *Sol Corr*

*GCSol* the mean of the simulator solution measurements by gas chromatography

*Y GC R <sup>Y</sup> X K GC*

*Cont*

(22)

*the true mean blood alcohol concentration is 0.0808 to 0.0846 g/dL.* 

**4.9 Assumptions of this approach** 

prepared to discuss them.

equation 12:

interpretation. Very generally, the assumptions are:

2. All standard uncertainties are valid estimates

**5. Breath alcohol measurement example** 

where: *Y*<sup>0</sup> the mean of the original n measurements

*R* the traceable reference value

the need for revision to ensure clarity in communication and interpretation.

1. The blood alcohol measurement results are normally distributed

4. The method of confidence interval estimation will be robust 5. With a fixed mean (), 95% of the intervals will bracket

7. This entire approach to estimating the uncertainty is uncertain. 8. We have assumed that all uncertainty components are independent

**4.8 Report the results** 

We now have an interval within which we would expect a large fraction (approximately 95%) of the expected values of the measurand to exist. If we were to assume k=2 to generate an approximate 95% uncertainty interval we would obtain: 0.0827 2 0.00067 0.0827 0.0013 0.0814 0.0840 / *to g dL* . We see that this interval is slightly narrower than that employing the effective degrees of freedom estimate. Choosing the appropriate coverage factor will be a decision made within each forensic laboratory. A 99% interval (k=3) will provide a higher degree of confidence that may be important in forensic applications. This is particularly true where results are near prohibited legal limits. Whatever decision is made, the value for k should be clearly identified in the program policy or SOP manuals and strictly adhered to in practice. In this example we have assumed our expanded interval to be an "uncertainty interval" rather than a "confidence interval". The *GUM* document prefers the term "uncertainty interval" or "level of confidence". (ISO/GUM, 2008) Others, however, interpret U as representing a confidence interval which has a specific definition in the classical statistical sense.

### **4.7 Produce the uncertainty budget**

Table 2 illustrates one form of an uncertainty budget for our example. The uncertainty budget lists the components contributing to the combined uncertainty along with the percent of their contribution to the total. The percent contributions were determined from the terms under the radical sign in equation 18. This is very useful for identifying which components are the major contributors and which may be reasonably ignored. The *GUM* document states that any contributions less than one-third of the largest contributor can be safely ignored. (ISO/GUM, 2008) Based on this we see that the analytical and dilutor components could be safely ignored in this example. However, from a forensic perspective it may be better to include all components considered, providing full disclosure. We see that the total method contributes the largest component at 59%. This is expected because of all of the contributing sub-components involved: analysts, calibrations, time, dilutions, etc. This analysis does not include, however, the venous blood sampling performed by the phlebotomist who typically performs only one venipuncture. Moreover, many laboratories do not even consider sampling as a component of their combined uncertainty. They simply consider their uncertainty estimates corresponding to the sample "as received in the laboratory". Jones, for example, has considered sampling as a source of uncertainty in some of his published work. (Jones, 1989)


1Percent of contribution to total combined uncertainty

Table 2. Uncertainty budget for the illustrated example
