**2.2.9 Performance of the algorithms with impact from extreme values of withinsubject biological variation (CVB)**

An important assumption for calculation of within-subject biological variation as the square root of the mean of the variances from the individual coefficients of variation of reference individuals in projects on biological variation is that these variations are distributed homogeneously. If there is variance homogeneity, this pooled coefficient of variance represents all individuals of the reference group and it is correct to use this pooled CVB in the simulations as a factor for the random Gaussian values. This assumption, however, is not fulfilled for TPA (Sölétormos et al. 2000a), where the range of coefficients of variation goes from 8.5% and 48.9% and represents individual CVB-values, from which the extreme values used for the challenging of algorithms in the simulations are selected.

The results in Table 1 are based on CVB = 24.5% (within-subject biological variation). This value is based on a 50th percentile from an investigation on 127 patients (Söletormos et al., 2000b). Due to the lack of variance homogeneity, we have also investigated the impact on the results from the algorithms based on a 95th percentile where CVB = 48.9%.

The results for CVB = 48.9% are listed in Table 2 where it can be seen that the detection times for tumour progression are practically the same as for the 50th percentile of biological variation. Only algorithm Söletormos et al. B {5} shows a 2 months later detection time for a slope of 0.0123. Nearly all algorithms show an increased percentage of false positive signals (the four first algorithms are already close to 100 % for CVB, = 24.5% for the highest start concentrations) with the higher biological variation CVB. Only the algorithm Molina et al. {6} maintains 0% FP results in situations with high biological variations. It should also be noted that the algorithm results from Söletormos et al. A {3} and Söletormos et al. B {5} both markedly increase the number of FP results, when the biological variation, CVB, is high and the start concentration is below cut-off. For Söletormos et al. B {5} this is partly due to the algorithm, where the significant change in the criterion is based on the 50th percentile of biological variation CVB = 24.5% whereas the simulation is based on the much higher extreme CVB = 48.9%. Consequently the use of significant change in the algorithm makes it sensitive to lack of variance homogeneity.

Computer Simulation Model System for Interpretation

Algorithm Slope 0.0907:

Barak et al. {1}

Tondini and Hayes {2}

Söletormos et al. A {3}

Söletormos et al. B {5}

Molina et al. {6}

Nicolini et al. {7}

Chan et al. {4} 100% at time

2 months (2 months)

2 months (2 months)

2 months (2 months)

[4 months]1 ([4months]1)

[4 months]1 ([4months]1)

[4 months]1 ([4months]1)

[4 months]1 ([4months]1)

concentrations have high percentages of FP after two years.

Slope 0.0346: 100% at time

6 months (6 months)

6 months (6 months)

6 months (6 months)

8 months (8 months)

8 months (8 months)

8 months (8 months)

[8 months]2 ([8 months]2)

100%. 3Only at start concentrations below 28 U/L obtain 100%.

**2.2.10 Biological variation of tumour growth** 

and Validation of Algorithms for Monitoring of Cancer Patients by Use of Serial Serum... 313

monitoring biomarker results will thus be the algorithm Söletormos et al. A {3}. In this situation very early detection times are combined with very low FP signals. It is notable that the algorithms {1} Barak et al., {2} Tondini and Hayes and {4} Chan et al. for the high start

> Slope 0.0123: 100% at time

12 months (12 months)

12 months (12 months)

12 months (14 months)

14 months (14 months)

14 months (16 months)

16 months (16 months)

[14 months]2 ([14 months]3)

1Three sample points are needed for the algorithm. 2 Only at start concentrations below 57 U/L obtain

Test of robustness of the algorithms when the biological variation is decreased from CVB = 24.5% to CVB = 8.5%. Times for detection of tumor progression using the different algorithms are listed for three slopes. Percentages false positive results after 1 and 2 years with TPA start concentrations below 95 U/L

simulations. Results from biological variation of CVB = 8.5% and CVB = 24.5 % just below in (brackets).

In conclusion, the most robust algorithm against biological variation is the algorithm Molina el al. {6}. And the most sensitive algorithms with influence from biological variations CVB

Just as we have investigated the impact of biological variation, CVB, on the performance in a *steady-state* situation, we have also challenged the variation in progression conditions. Thus,

and 57 U/L for each algorithm are also listed. All results are generated from 1000 computer

Table 3. Performance of seven algorithms with decreased biological variation.

are the algorithms Söletormos et al. A {3} and Söletormos et al. B {5}.

Falsepositive below 95 U/L at 1 year

> 98% (98%)

> 65% (91%)

> 1% (32%)

> 68% (64%)

0% (6%)

0% (0%)

0% (6%) False positive below 95 U/L at 2 years

> 100% (100%)

> 94% (100%)

1% (70%)

92% (91%)

0% (24%)

0% (0%)

0% (10%) False positive below 57 U/L at 1 year

> 0% (3%)

> 0% (3%)

> 0% (2%)

> 0% (0%)

> 0% (0%)

> 0% (0%)

> 0% (0%)

False positive below 57 U/L at 2 years

> 0% (4%)

> 0% (4%)

> 0% (3%)

> 0% (0%)

> 0% (0%)

> 0% (0%)

> 0% (0%)


1Three sample points are needed for the algorithm. 2 Only at start concentrations below 57 U/L obtain 100%. 3Only at start concentrations below 28 U/L obtain 100%. 4 Only at start concentration below 19 U/L. Test of robustness of the algorithms when the biological variation, CVB, is increased from 24.5% to 48.9%. Times for detection of tumor progression using the different algorithms are listed for three slopes. Percentages false positive results (FP) after 1 and 2 years with TPA start concentrations below 95 U/L and 57 U/L for each algorithm are also listed. All results are generated from 1000 computer simulations. Results from biological variation of CVB = 48.9% and CVB = 24.5% just below in (brackets).

Table 2. Performance of seven algorithms with increased biological variation.

On the other hand the results indicate that the algorithm from Molina et al. {6} is most robust against increased biological variation CVB values.

In the other extreme situation with a very low biological variation 5th percentile (CVB = 8.5%), the performances from the algorithms are listed in Table 3.

The most striking results in the table is the impact from low biological variation CVB on the false positive number (FP) from algorithm Söletormos et al. A {3} and algorithm Söletormos et al. B {5}. These algorithms show low percentage of FP results, when the biological variation CVB is low – and on the other hand - a high number of FP results when the biological variation CVB is high, as discussed above. It should also be noted that the detection time for the slowest slope is two months earlier for Söletormos et al. A {3}, who at the same time show very low percentages of FP signals. In a clinical situation with a patient, where the biological variation is known to be low, the best algorithm for interpreting

Slope 0.0123: 100% at time

12 months (12 months)

12 months (12 months)

14 months (14 months)

14 months (14 months)

18 months (16 months)

16 months (16 months)

[14 months]4 ([14 months]3)

On the other hand the results indicate that the algorithm from Molina et al. {6} is most

In the other extreme situation with a very low biological variation 5th percentile (CVB =

The most striking results in the table is the impact from low biological variation CVB on the false positive number (FP) from algorithm Söletormos et al. A {3} and algorithm Söletormos et al. B {5}. These algorithms show low percentage of FP results, when the biological variation CVB is low – and on the other hand - a high number of FP results when the biological variation CVB is high, as discussed above. It should also be noted that the detection time for the slowest slope is two months earlier for Söletormos et al. A {3}, who at the same time show very low percentages of FP signals. In a clinical situation with a patient, where the biological variation is known to be low, the best algorithm for interpreting

1Three sample points are needed for the algorithm. 2 Only at start concentrations below 57 U/L obtain 100%. 3Only at start concentrations below 28 U/L obtain 100%. 4 Only at start concentration below 19 U/L. Test of robustness of the algorithms when the biological variation, CVB, is increased from 24.5% to 48.9%. Times for detection of tumor progression using the different algorithms are listed for three slopes. Percentages false positive results (FP) after 1 and 2 years with TPA start concentrations below 95 U/L and 57 U/L for each algorithm are also listed. All results are generated from 1000 computer simulations. Results from biological variation of CVB = 48.9% and CVB = 24.5% just below in (brackets).

Table 2. Performance of seven algorithms with increased biological variation.

Falsepositive below 95 U/L at 1 year

> 98% (98%)

> 95% (91%)

> 76% (32%)

> 65% (64%)

19% (6%)

0% (0%)

16% (6%) False positive below 95 U/L at 2 years

> 100% (100%)

> 100% (100%)

98% (70%)

91% (91%)

47% (24%)

0% (0%)

29% (10%) False positive below 57 U/L at 1 year

> 45% (3%)

> 45% (3%)

> 35% (2%)

> 6% (0%)

> 3% (0%)

> 0% (0%)

> 0% (0%)

False positive below 57 U/L at 2 years

> 70% (4%)

> 70% (4%)

> 65% (3%)

> 13% (0%)

> 7% (0%)

> 0% (0%)

> 0% (0%)

Algorithm Slope 0.0907:

Barak et al. {1}

Tondini and Hayes {2}

Söletormos et al. A {3}

Söletormos et al. B {5}

Molina et al. {6}

Nicolini et al. {7}

Chan et al. {4} 100% at time

2 months (2 months)

2 months (2 months)

2 months (2 months)

[4 months]1 ([4months]1)

[4 months]1 ([4months]1)

[4 months]1 ([4months]1)

[4 months]1 ([4months]1) Slope 0.0346: 100% at time

> 6 months (6 months)

> 6 months (6 months)

> 6 months (6 months)

> 8 months (8 months)

> 8 months (8 months)

> 8 months (8 months)

[8 months]2 ([8 months]2)

robust against increased biological variation CVB values.

8.5%), the performances from the algorithms are listed in Table 3.

monitoring biomarker results will thus be the algorithm Söletormos et al. A {3}. In this situation very early detection times are combined with very low FP signals. It is notable that the algorithms {1} Barak et al., {2} Tondini and Hayes and {4} Chan et al. for the high start concentrations have high percentages of FP after two years.


1Three sample points are needed for the algorithm. 2 Only at start concentrations below 57 U/L obtain 100%. 3Only at start concentrations below 28 U/L obtain 100%.

Test of robustness of the algorithms when the biological variation is decreased from CVB = 24.5% to CVB = 8.5%. Times for detection of tumor progression using the different algorithms are listed for three slopes. Percentages false positive results after 1 and 2 years with TPA start concentrations below 95 U/L and 57 U/L for each algorithm are also listed. All results are generated from 1000 computer simulations. Results from biological variation of CVB = 8.5% and CVB = 24.5 % just below in (brackets).

Table 3. Performance of seven algorithms with decreased biological variation.

In conclusion, the most robust algorithm against biological variation is the algorithm Molina el al. {6}. And the most sensitive algorithms with influence from biological variations CVB are the algorithms Söletormos et al. A {3} and Söletormos et al. B {5}.

### **2.2.10 Biological variation of tumour growth**

Just as we have investigated the impact of biological variation, CVB, on the performance in a *steady-state* situation, we have also challenged the variation in progression conditions. Thus,

Computer Simulation Model System for Interpretation

months extended detection time at the slowest slope.

the start concentration was low, i.e. below 57 U/L.

simultaneously low number of FP results.

only few FP results.

**3.1 Computer simulations** 

performance of algorithms is independent of the biomarker.

**3. Conclusion** 

excluded from further computer simulation investigations.

and Validation of Algorithms for Monitoring of Cancer Patients by Use of Serial Serum... 315

Also the detection times for progression using the three slopes with biological variation were nearly the same as the results without variation. An example of the minor changes in results including variation of tumour growth is illustrated in Fig. 7. Results from the algorithm Barak et al.{1} include a variation of exponential function of 25%; only the slowest slope for tumour growth shows a slightly reduced number of percentages of positive results compared to the "normal" slope. The most marked change was found with algorithm Söletormos et al. A {3}. This algorithm showed two months extended detection time at the middle fast slope (λ = 0.0346) – and similarly the algorithm Molina et al. {6} showed two

Overall, the biological variation of tumour growth has only minimal effects on the results and does not change the conclusions based on results in Table 1; it may therefore be

The start concentration of the biomarker TPA is a very important parameter in the examination of the performance of the algorithm, i.e. time for detection of progression and percentage of false positive results (FP). Start concentrations near cut-off will give more FP in nearly every algorithm – but the algorithms with low FP results also have longer tumour detection time. All the investigated algorithms performed comparable in FP results, when

These overall conclusions are relatively identical to the conclusion on results from the same algorithms using biomarker CA 15-3 (Petersen et al., 2011) – this indicates that the relative

Differences in biological variation, CVB, have an influence on the performance of nearly all the algorithms. Only the algorithm Molina et al {6} has unchanged results with the different biological variations, CVB, – in other words this algorithm is the most robust against increasing biological variation CVB. Some algorithms show better performance when the biological variation CVB is low. When the biological variation CVB is low the algorithm Söletormos et al. A {3} has the best performance as regards early progression detection and

The biological variation of the tumour growth up to 25% has only a minor influence on the

In a clinical situation the start concentration should be the point for selecting the best algorithm. When the start concentration is near the cut-off, the algorithm Molina et al. {6} could be used to avoid too many FP results. When the start concentration is below 57 U/L, the algorithm Barak et al. {1} could be used to have a short progressive detection time with

performance of the algorithms and does not chance the overall conclusions.

A summary of new important conclusions from this investigation:

a. The relative performances of algorithms are independent of the biomarker.

we have included a variation of 25% with the selected three slopes in the simulation model and compared the results with the results in Table 1.

Nearly all the results were close to the same as in Table 1 when this variation of 25% was included in the exponential function. The false positive (FP) results chanced only a few per cent for the most algorithms and maximum increases in percentages were 4% found at the algorithms Barak et al. {1} and Tondini & Hayes {2} below 57 U/L after two years.

Percentage positive patients (POS) as a function of starting concentrations for algorithm Barak et al. {1} after one year. The upper figure shows results from the "normal" rate of increase. The figure below shows results from a modified rate of increase, including biological variation in the exponential function of tumour growth of 25%. For the slowest slope (λ =0.0132) (---o---), the modified slope shows a reduced number of percentage positives from start concentration 0 up to approx 57 U/L TPA with percentage positives increasing from 78% to 100%, respectively.

Fig. 7. Impact of biological variation on exponential tumour growth.

Also the detection times for progression using the three slopes with biological variation were nearly the same as the results without variation. An example of the minor changes in results including variation of tumour growth is illustrated in Fig. 7. Results from the algorithm Barak et al.{1} include a variation of exponential function of 25%; only the slowest slope for tumour growth shows a slightly reduced number of percentages of positive results compared to the "normal" slope. The most marked change was found with algorithm Söletormos et al. A {3}. This algorithm showed two months extended detection time at the middle fast slope (λ = 0.0346) – and similarly the algorithm Molina et al. {6} showed two months extended detection time at the slowest slope.

Overall, the biological variation of tumour growth has only minimal effects on the results and does not change the conclusions based on results in Table 1; it may therefore be excluded from further computer simulation investigations.
