3. Multi-scale method based on regression spline

For regression spline, the number of knots controls the smoothness of the estimator. The determination of knots is important and plays a large influence on the inference results. The GCV method is usually used to choose an optimal number of knots. While, but after the number of knots is given, the determination of the optimal positions of knots is difficult. Shi and Li [38] chose knots by placing an additional new knot to reduce the value of GCV, until it could not be reduced by placing any additional knots. Hence, once a knot was selected, it cannot be removed from the knot set. Mao and Zhao [39] determined the locations of knots conditioned on the number of knots m first and chose m later by GCV criterion. In fact, the locations of knots can be considered as parameters which can be estimated from data. This is the free-knot spline; see DiMatteo et al. [40] and Sonderegger and Hannig [41]. However, the estimation of the optimal locations is computationally intractable, and replicate knots might appear in the estimated knot vectors [42].

On the other hand, many statisticians think that the statistical inference based on one smoothing level is not reliable although it is the optimal one. Therefore, multi-scale method is developed to estimate and test nonparametric regression curves. Chaudhuri and Marron [25, 26] proposed a multi-scale method to explore the significant structures (local minima and maxima or global trend) in data, which is known as SiZer. Significant zero crossings of derivatives (SiZer) is a powerful visualization technique for exploratory data analysis. It applies a large range of smoothing parameter values to do statistical inference simultaneously and use a 2D colored map (SiZer map) to summarize all of the results inferred at different smoothing levels (scales) and locations.

different values of m are plotted in Figure 1 too. The simulated SiZerLS map and SiZerSS map

Model Testing Based on Regression Spline http://dx.doi.org/10.5772/intechopen.74858 91

In Figure 2, BYP SiZerLS is SiZerLS map based on multiple testing procedures, BYP, where BYP denotes the multiple testing procedure proposed in Benjamini and Yekutieli [43]. SiZerSS is the smoothing spline version of SiZer. The two SiZers are simulated under the same range of scales and nominal level 0.05. There are four colors in SiZer maps: red indicates that the

are shown in Figure 2, respectively.

Figure 2. BYP SiZerLS and SiZerSS for detecting peaks of data.

Figure 1. 200 observations and the estimated curves based on different knot sets.

In this section, a regression spline version of SiZer is proposed for exploring structures of curve and comparing multiple regression curves, respectively. The proposed SiZer employs the number of knots as smoothing parameter (scales). For the sake of simplicity, linear spline is employed first to construct SiZer, which is denoted as SiZerLS. In addition, another version of SiZer—SiZerSS—is introduced, which is proposed in Marron and Zhang [33]. In SiZerSS, smoothing spline is used to infer the monotonicity of f xð Þ, and the tuning parameter (penalty parameter) that controls the size of penalty is chosen to be as the smoothing parameter. Finally, SiZer-RS, a version of SiZer based on higher-order spline interpolation, is constructed to compare multiple regression curves at different scales and locations simultaneously.

In order to understand SiZerLS clearly, we first present an example in which SiZerLS are simulated. This example is modified from Hannig and Lee [28] with the same regression function:

$$f(\mathbf{x}) = \mathbf{5} + 4.2 \left( 1 + \frac{|\mathbf{x} - \mathbf{0.3}|}{0.03} \right) - \mathbf{4} + \mathbf{5.1} \left( 1 + \frac{|\mathbf{x} - \mathbf{0.7}|}{0.01} \right) - \mathbf{4.2}$$

The observations generated from model (1) with 200 equally spaced design points from (0, 1) and σ � Nð Þ 0; 0:5 are plotted in Figure 1. Estimator bf <sup>m</sup>ð Þx denotes the linear spline smoother obtained from (6) using m equally spaced knots chosen from (0, 1). The curves of bf <sup>m</sup>ð Þx with

Figure 1. 200 observations and the estimated curves based on different knot sets.

3. Multi-scale method based on regression spline

appear in the estimated knot vectors [42].

(scales) and locations.

90 Topics in Splines and Applications

function:

For regression spline, the number of knots controls the smoothness of the estimator. The determination of knots is important and plays a large influence on the inference results. The GCV method is usually used to choose an optimal number of knots. While, but after the number of knots is given, the determination of the optimal positions of knots is difficult. Shi and Li [38] chose knots by placing an additional new knot to reduce the value of GCV, until it could not be reduced by placing any additional knots. Hence, once a knot was selected, it cannot be removed from the knot set. Mao and Zhao [39] determined the locations of knots conditioned on the number of knots m first and chose m later by GCV criterion. In fact, the locations of knots can be considered as parameters which can be estimated from data. This is the free-knot spline; see DiMatteo et al. [40] and Sonderegger and Hannig [41]. However, the estimation of the optimal locations is computationally intractable, and replicate knots might

On the other hand, many statisticians think that the statistical inference based on one smoothing level is not reliable although it is the optimal one. Therefore, multi-scale method is developed to estimate and test nonparametric regression curves. Chaudhuri and Marron [25, 26] proposed a multi-scale method to explore the significant structures (local minima and maxima or global trend) in data, which is known as SiZer. Significant zero crossings of derivatives (SiZer) is a powerful visualization technique for exploratory data analysis. It applies a large range of smoothing parameter values to do statistical inference simultaneously and use a 2D colored map (SiZer map) to summarize all of the results inferred at different smoothing levels

In this section, a regression spline version of SiZer is proposed for exploring structures of curve and comparing multiple regression curves, respectively. The proposed SiZer employs the number of knots as smoothing parameter (scales). For the sake of simplicity, linear spline is employed first to construct SiZer, which is denoted as SiZerLS. In addition, another version of SiZer—SiZerSS—is introduced, which is proposed in Marron and Zhang [33]. In SiZerSS, smoothing spline is used to infer the monotonicity of f xð Þ, and the tuning parameter (penalty parameter) that controls the size of penalty is chosen to be as the smoothing parameter. Finally, SiZer-RS, a version of SiZer based on higher-order spline interpolation, is constructed to

In order to understand SiZerLS clearly, we first present an example in which SiZerLS are simulated. This example is modified from Hannig and Lee [28] with the same regression

The observations generated from model (1) with 200 equally spaced design points from (0, 1) and σ � Nð Þ 0; 0:5 are plotted in Figure 1. Estimator bf <sup>m</sup>ð Þx denotes the linear spline smoother obtained from (6) using m equally spaced knots chosen from (0, 1). The curves of bf <sup>m</sup>ð Þx with

� <sup>4</sup> <sup>þ</sup> <sup>5</sup>:1 1 <sup>þ</sup> j j <sup>x</sup>–0:<sup>7</sup>

0:01 � �

� 4:

compare multiple regression curves at different scales and locations simultaneously.

0:03 � �

f xð Þ¼ <sup>5</sup> <sup>þ</sup> <sup>4</sup>:2 1 <sup>þ</sup> j j <sup>x</sup> � <sup>0</sup>:<sup>3</sup>

different values of m are plotted in Figure 1 too. The simulated SiZerLS map and SiZerSS map are shown in Figure 2, respectively.

In Figure 2, BYP SiZerLS is SiZerLS map based on multiple testing procedures, BYP, where BYP denotes the multiple testing procedure proposed in Benjamini and Yekutieli [43]. SiZerSS is the smoothing spline version of SiZer. The two SiZers are simulated under the same range of scales and nominal level 0.05. There are four colors in SiZer maps: red indicates that the

Figure 2. BYP SiZerLS and SiZerSS for detecting peaks of data.

estimated regression curve is significantly decreasing; blue indicates that the estimated regression curve is significantly increasing; purple indicates that the curve is neither significantly increasing nor decreasing; gray shows that there are no sufficient data for conducting reasonable statistical inference. Figure 1 preliminarily shows that SiZer maps can locate peaks well. The theoretical foundation of SiZerLS and SiZerSS will be discussed in more detail at a later stage.

#### 3.1. Construction of SiZerLS map for exploring features of regression curve

The proposed SiZerLS map will be constructed on the basis of the p-values with multiple testing adjustment. The p-value for testing the monotonicity of the smoothed curve is defined first based on linear spline approximation and fiducial method in the same way as p-values in Section 2. Consequently, multiple testing adjustment is discussed detailedly to control the rowwise false discovery rate (FDR) of SiZerLS.

In the view of SiZer, all of the useful information is included in the smoothed curve, which is defined below. Suppose we have observations xi; yi � �<sup>n</sup> <sup>i</sup>¼<sup>1</sup> from regression model (1). By linear spline estimation, estimator bf <sup>m</sup>ð Þx can be obtained:

$$\widehat{f}\_m(\mathbf{x}) = \mathbf{g}(\mathbf{x})' \left(\mathbf{G}^T \mathbf{G}\right)^{-1} \mathbf{G}^T \mathbf{Y},\tag{22}$$

P∗ Ik bβ; S

where the subscript Ik of P<sup>∗</sup>

It is worth noting that p-value P<sup>∗</sup>

reasonable (see Theorem 1 in [44]):

for testing H<sup>∗</sup>

P∗ Ik bβ; S � � <sup>¼</sup> P LkR E; <sup>b</sup>β; <sup>S</sup>

8 ><

>:

¼ P

Dk satisfies equation P<sup>∗</sup>

� � <sup>≤</sup> <sup>0</sup> n o <sup>¼</sup> P Lk

Lk <sup>G</sup><sup>0</sup> ð Þ <sup>G</sup> �<sup>1</sup>

� �<sup>1</sup>

m represents the number of knots used in linear interpolation. In addition, p-value P<sup>∗</sup>

ffiffiffiffiffiffiffiffiffiffiffiffi <sup>n</sup> � <sup>m</sup> <sup>p</sup> Lk <sup>G</sup><sup>0</sup> ð Þ <sup>G</sup> �<sup>1</sup>

> Ik bβ; S � � <sup>þ</sup> <sup>P</sup><sup>∗</sup>

� �≜<sup>1</sup> � <sup>Φ</sup>

BYP was proved to control FDR under α for any dependent test statistics.

3.1.1. Benjamin-Yekutieli procedure to control FDR (BYP)

Suppose that we have obtained p-values PIk,m bβ; S

and reject all HI kð Þ,m for k ¼ 1, 2, ⋯, m � 1.

k ¼ 1, 2, ⋯, m � 1: 1. Order p-values P<sup>∗</sup> Ik bβ; S

PIk,m bβ; S

<sup>b</sup><sup>β</sup> � <sup>S</sup> E2

G0 E1

≥

ffiffiffiffiffiffiffiffiffiffiffiffi <sup>n</sup> � <sup>m</sup> <sup>p</sup> Lk

S Lk <sup>G</sup><sup>0</sup> ð Þ <sup>G</sup> �<sup>1</sup>

Ik,m and get the ordered p-values PIð Þ<sup>1</sup> ,m, PIð Þ<sup>2</sup> ,m, ⋯, PI mð Þ �<sup>1</sup> ,m.

2. For a given p-value <sup>α</sup>, find the largest <sup>i</sup> for <sup>k</sup> <sup>¼</sup> <sup>1</sup>, <sup>2</sup>, <sup>⋯</sup>, m � 1 for which PI ið Þ,m <sup>≤</sup> <sup>k</sup><sup>α</sup>

bβ

1

� � for testing hypotheses HIk in (23),

ð Þ <sup>m</sup>�<sup>1</sup> P<sup>m</sup>�<sup>1</sup> j¼1 1 j

CA: (26)

Lk � �<sup>1</sup>=<sup>2</sup>

2 E2

Dk bβ; S � � <sup>¼</sup> 1.

HIk, HDk, k ¼ 1, 2, ⋯, m � 1 are true (regression function is a constant). In applications, p-value

0 B@

The proposed SiZerLS map will be constructed on the basis of the above p-values with multiple testing adjustment. In fact, SiZer is a visual method for exploratory data analysis, and it focuses on exploring features that really exist in data instead of testing whether some assumed features are statistically significant in a strict way. FDR is the expected proportion of the false positives among all discoveries, and FDR can be either permissive or conservative according to the number of hypotheses. Considering that different numbers of hypotheses need to be tested for SiZerLS with respect to various smoothing parameters, the multiple testing adjustment to control FDR would be better if used to improve the exploratory property of SiZer. Hence, the well-known multiple testing procedure which was proposed in Benjamini and Yekutieli [43] (denoted as BYP) is applied to control the row-wise FDR of SiZerLS. The

� � for testing HIk can be approximated as below when <sup>n</sup> ! <sup>∞</sup>: This approximation is

L0 k <sup>G</sup><sup>0</sup> ð Þ <sup>G</sup> �<sup>1</sup>

� �

ffiffiffiffiffiffiffiffiffiffiffiffi <sup>n</sup> � <sup>m</sup> <sup>p</sup> <sup>b</sup><sup>β</sup>

� �<sup>1</sup>

L0 k

S Lk <sup>G</sup><sup>0</sup> ð Þ <sup>G</sup> �<sup>1</sup>

� � is uniformly distributed on (0,1) if all of the hypotheses

Ik represents the interval (tk, tkþ1) in which we test monotonicity and

<sup>2</sup>E<sup>1</sup> ≤ 0

Model Testing Based on Regression Spline http://dx.doi.org/10.5772/intechopen.74858

2

9 >=

>;

, (25)

Dk bβ; S � � 93

where <sup>g</sup>ð Þ¼<sup>x</sup> <sup>g</sup>1ð Þ<sup>x</sup> ; <sup>g</sup>2ð Þ<sup>x</sup> ; <sup>⋯</sup>; gmð Þ<sup>x</sup> � �<sup>0</sup> ; gj ð Þx , j ¼ 1, ⋯, m are the basis functions defined in (7) on the basis of m knots; and G is the matrix defined in Section 2. The smoothed curve at smoothing level m is denoted as.

$$f\_m(\mathbf{x}) = \operatorname{E} \left( \widehat{f}\_m(\mathbf{x}) \right) = \operatorname{g}(\mathbf{x})' \left( \mathbf{G}^T \mathbf{G} \right)^{-1} \mathbf{G}^T f\_m$$

where f¼f g f xð Þ<sup>1</sup> ; f xð Þ<sup>2</sup> ; ⋯; f xð Þ<sup>n</sup> <sup>0</sup> . SiZer focuses on f <sup>m</sup>ð Þx : Its monotonicity is determined totally by GTG � ��<sup>1</sup> GT <sup>f</sup>. Hence, it is enough to test the following <sup>m</sup> � 1 pairs of null hypotheses:

$$H\_{\rm lk} = f\_m(\text{t}\_k) = e\_k'(\text{G}'\text{G})^{-1}\text{G}'f \preceq e\_{k+1}'(\text{G}'\text{G})^{-1}\text{G}'f = f\_m(\text{t}\_{k+1})(\text{and})$$

$$H\_{\rm Dk} = f\_m(\text{t}\_k) = e\_k'(\text{G}'\text{G})^{-1}\text{G}'f \gtrsim e\_{k+1}'(\text{G}'\text{G})^{-1}\text{G}'f = f\_m(\text{t}\_{k+1}), k = 1, 2, \dots, m-1,\tag{23}$$

where ek is an m-dimensional column vector having 1 in the kth entry and zero elsewhere. Let b denote <sup>G</sup><sup>0</sup> ð Þ <sup>G</sup> �<sup>1</sup> G0 f. Then, HIk and HDk can be written as

$$H\_{\rm lk}^\* = L\_k b \le 0, k = 1, 2, \cdots, m - 1; \quad H\_{\rm Dk}^\* = L\_k b \ge 0, k = 1, 2, \cdots, m - 1,\tag{24}$$

where Lk ≜ e<sup>0</sup> <sup>k</sup> � e<sup>0</sup> kþ1 � ). The p-values to test hypotheses in (24) under linear model <sup>Y</sup> <sup>¼</sup> Gb <sup>þ</sup> <sup>ε</sup> can be defined using pivotal quantity about <sup>b</sup>. This pivotal quantity is R E; <sup>b</sup>β; <sup>S</sup><sup>2</sup> � �, which is defined in (11). The p-value for testing H<sup>∗</sup> Ik is the fiducial probability that null hypothesis holds:

Model Testing Based on Regression Spline http://dx.doi.org/10.5772/intechopen.74858 93

$$P\_{lk}^\*\left(\widehat{\boldsymbol{\beta}},\boldsymbol{S}\right) = P\left\{L\_k R\left(E; \widehat{\boldsymbol{\beta}},\boldsymbol{S}\right) \leq 0\right\} = P\left\{L\_k \widehat{\boldsymbol{\beta}} - \frac{\boldsymbol{S}}{E\_2} \left(G' \boldsymbol{G}\right)^{-\frac{1}{2}} \boldsymbol{E}\_1 \leq 0\right\}$$

$$= P\left\{\frac{\sqrt{n-m}L\_k \left(G' \boldsymbol{G}\right)^{-1} G' \boldsymbol{E}\_1}{\left(L\_k \left(G' \boldsymbol{G}\right)^{-1} L\_k'\right)^{\frac{1}{2}} \boldsymbol{E}\_2} \geq \frac{\sqrt{n-m}\widehat{\boldsymbol{\beta}}}{S\left(L\_k \left(G' \boldsymbol{G}\right)^{-1} L\_k'\right)^{\frac{1}{2}}}\right\},\tag{25}$$

where the subscript Ik of P<sup>∗</sup> Ik represents the interval (tk, tkþ1) in which we test monotonicity and m represents the number of knots used in linear interpolation. In addition, p-value P<sup>∗</sup> Dk bβ; S � � for testing H<sup>∗</sup> Dk satisfies equation P<sup>∗</sup> Ik bβ; S � � <sup>þ</sup> <sup>P</sup><sup>∗</sup> Dk bβ; S � � <sup>¼</sup> 1.

It is worth noting that p-value P<sup>∗</sup> Ik bβ; S � � is uniformly distributed on (0,1) if all of the hypotheses HIk, HDk, k ¼ 1, 2, ⋯, m � 1 are true (regression function is a constant). In applications, p-value P∗ Ik bβ; S � � for testing HIk can be approximated as below when <sup>n</sup> ! <sup>∞</sup>: This approximation is reasonable (see Theorem 1 in [44]):

$$P\_{\mathbb{R},m}\left(\widehat{\boldsymbol{\beta}},\boldsymbol{S}\right) \triangleq 1 - \Phi\left(\frac{\sqrt{n-m}L\_k\widehat{\boldsymbol{\beta}}}{S\left(L\_k\left(\boldsymbol{G}'\boldsymbol{G}\right)^{-1}L\_k\right)^{1/2}}\right).\tag{26}$$

The proposed SiZerLS map will be constructed on the basis of the above p-values with multiple testing adjustment. In fact, SiZer is a visual method for exploratory data analysis, and it focuses on exploring features that really exist in data instead of testing whether some assumed features are statistically significant in a strict way. FDR is the expected proportion of the false positives among all discoveries, and FDR can be either permissive or conservative according to the number of hypotheses. Considering that different numbers of hypotheses need to be tested for SiZerLS with respect to various smoothing parameters, the multiple testing adjustment to control FDR would be better if used to improve the exploratory property of SiZer. Hence, the well-known multiple testing procedure which was proposed in Benjamini and Yekutieli [43] (denoted as BYP) is applied to control the row-wise FDR of SiZerLS. The BYP was proved to control FDR under α for any dependent test statistics.

#### 3.1.1. Benjamin-Yekutieli procedure to control FDR (BYP)

estimated regression curve is significantly decreasing; blue indicates that the estimated regression curve is significantly increasing; purple indicates that the curve is neither significantly increasing nor decreasing; gray shows that there are no sufficient data for conducting reasonable statistical inference. Figure 1 preliminarily shows that SiZer maps can locate peaks well. The theoretical foundation of SiZerLS and SiZerSS will be discussed in more detail at a later

The proposed SiZerLS map will be constructed on the basis of the p-values with multiple testing adjustment. The p-value for testing the monotonicity of the smoothed curve is defined first based on linear spline approximation and fiducial method in the same way as p-values in Section 2. Consequently, multiple testing adjustment is discussed detailedly to control the row-

In the view of SiZer, all of the useful information is included in the smoothed curve, which is

the basis of m knots; and G is the matrix defined in Section 2. The smoothed curve at smooth-

<sup>¼</sup> <sup>g</sup>ð Þ<sup>x</sup> <sup>0</sup> <sup>G</sup>TG � ��<sup>1</sup>

GT <sup>f</sup>. Hence, it is enough to test the following <sup>m</sup> � 1 pairs of null hypotheses:

<sup>k</sup>þ<sup>1</sup> <sup>G</sup><sup>0</sup> ð Þ <sup>G</sup> �<sup>1</sup>

where ek is an m-dimensional column vector having 1 in the kth entry and zero elsewhere. Let b

<sup>k</sup>þ<sup>1</sup> <sup>G</sup><sup>0</sup> ð Þ <sup>G</sup> �<sup>1</sup>

G0

� ). The p-values to test hypotheses in (24) under linear model <sup>Y</sup> <sup>¼</sup> Gb <sup>þ</sup> <sup>ε</sup>

<sup>b</sup><sup>f</sup> <sup>m</sup>ð Þ¼ <sup>x</sup> <sup>g</sup>ð Þ<sup>x</sup> <sup>0</sup> <sup>G</sup>TG � ��<sup>1</sup>

; gj

f <sup>m</sup>ð Þ¼ x E bf <sup>m</sup>ð Þx

0 <sup>k</sup> <sup>G</sup><sup>0</sup> ð Þ <sup>G</sup> �<sup>1</sup>

� �

G0 f ≤ e 0

can be defined using pivotal quantity about <sup>b</sup>. This pivotal quantity is R E; <sup>b</sup>β; <sup>S</sup><sup>2</sup> � �

G0 f ≥ e 0

f. Then, HIk and HDk can be written as

Ik <sup>¼</sup> Lkb <sup>≤</sup> <sup>0</sup>, k <sup>¼</sup> <sup>1</sup>, <sup>2</sup>, <sup>⋯</sup>, m � <sup>1</sup>; H<sup>∗</sup>

� �<sup>n</sup>

<sup>i</sup>¼<sup>1</sup> from regression model (1). By linear

G<sup>T</sup> Y, (22)

ð Þx , j ¼ 1, ⋯, m are the basis functions defined in (7) on

G<sup>T</sup> f,

. SiZer focuses on f <sup>m</sup>ð Þx : Its monotonicity is determined totally

f ¼ f <sup>m</sup>ð Þð tkþ<sup>1</sup> andÞ

f ¼ f <sup>m</sup>ð Þ tkþ<sup>1</sup> , k ¼ 1, 2, ⋯, m � 1, (23)

Dk ¼ Lkb ≥ 0, k ¼ 1, 2, ⋯, m � 1, (24)

Ik is the fiducial probability that null hypothesis holds:

, which is

G0

3.1. Construction of SiZerLS map for exploring features of regression curve

wise false discovery rate (FDR) of SiZerLS.

where <sup>g</sup>ð Þ¼<sup>x</sup> <sup>g</sup>1ð Þ<sup>x</sup> ; <sup>g</sup>2ð Þ<sup>x</sup> ; <sup>⋯</sup>; gmð Þ<sup>x</sup> � �<sup>0</sup>

where f¼f g f xð Þ<sup>1</sup> ; f xð Þ<sup>2</sup> ; ⋯; f xð Þ<sup>n</sup> <sup>0</sup>

ing level m is denoted as.

by GTG � ��<sup>1</sup>

denote <sup>G</sup><sup>0</sup> ð Þ <sup>G</sup> �<sup>1</sup>

where Lk ≜ e<sup>0</sup>

defined below. Suppose we have observations xi; yi

spline estimation, estimator bf <sup>m</sup>ð Þx can be obtained:

HIk ¼ f <sup>m</sup>ð Þ¼ tk e

0 <sup>k</sup> <sup>G</sup><sup>0</sup> ð Þ <sup>G</sup> �<sup>1</sup>

HDk ¼ f <sup>m</sup>ð Þ¼ tk e

G0

H∗

<sup>k</sup> � e<sup>0</sup> kþ1

defined in (11). The p-value for testing H<sup>∗</sup>

stage.

92 Topics in Splines and Applications

Suppose that we have obtained p-values PIk,m bβ; S � � for testing hypotheses HIk in (23), k ¼ 1, 2, ⋯, m � 1:


The detailed steps to construct SiZerLS with BYP adjustment are given below:

Step 1. Construct 2D grid map. Without loss of generality, we assume that designed points xi, i ¼ 1, 2, ⋯, n are chosen from [0, 1]. Then the 2D map is a rectangular area [0, 1; log10ð Þ 1=mmax; , log10ð Þ� 1=mmin ; see BYP SiZerLS displayed in Figure 2. The value of m is determined by the following rule: <sup>m</sup> <sup>¼</sup> round 1=10<sup>l</sup> � �, where function round (∙) is the nearest integer function and <sup>l</sup> takes equally spaced values from interval log10ð Þ <sup>1</sup>=mmin ; � log10ð Þ� <sup>1</sup>=mmax . For a given m, abscissa x takes values at the corresponding knots T<sup>m</sup> ¼ f g t1; t2; ⋯; tm . On the basis of different values of m and Tm, the 2D map is divided into many pixels.

3.2. Construction of SiZerSS map for exploring features of regression curve

Xn i¼1

bfλ¼ bf <sup>λ</sup>ð Þ x<sup>1</sup> ;bf <sup>λ</sup>ð Þ x<sup>2</sup> ; ⋯;bf <sup>λ</sup>ð Þ xn

By simple calculation, we can get the estimator vector:

� �<sup>n</sup>

ω<sup>i</sup> yi � f xð Þ<sup>i</sup> � �<sup>2</sup> <sup>þ</sup> <sup>λ</sup>

� �¼ð Þ <sup>W</sup> <sup>þ</sup> <sup>λ</sup><sup>K</sup> �<sup>1</sup>

where weight matrix <sup>W</sup> <sup>¼</sup> diagð Þ <sup>ω</sup>1; <sup>ω</sup>2; <sup>⋯</sup>; <sup>ω</sup><sup>n</sup> and the hat matrix <sup>A</sup><sup>λ</sup> <sup>¼</sup> ð Þ <sup>W</sup> <sup>þ</sup> <sup>λ</sup><sup>K</sup> �<sup>1</sup>

<sup>j</sup>�<sup>1</sup>, qjj ¼ �s�<sup>1</sup>

00ð Þ xn

In order to construct SiZerSS, the derivative of f at any point x needs to be estimated along with its variance. Let si <sup>¼</sup> xiþ<sup>1</sup> � xi and <sup>n</sup> � ð Þ <sup>n</sup> � <sup>1</sup> matrix <sup>Q</sup> <sup>¼</sup> qij n o, i <sup>¼</sup> <sup>1</sup>, <sup>2</sup>, <sup>⋯</sup>, n,

<sup>j</sup>�<sup>1</sup> � <sup>s</sup>�<sup>1</sup>

where R is a (n � 2Þ � ð Þ n � 2 symmetric matrix with elements rij, i ¼ 2, ⋯, n � 1,

ð Þ<sup>x</sup> can be written as a linear combination of <sup>b</sup><sup>f</sup> and <sup>γ</sup>b. Let hið Þ¼ <sup>x</sup> <sup>x</sup> � xi, i <sup>¼</sup> <sup>1</sup>, <sup>2</sup>, <sup>⋯</sup>, n: When

� s1 6 γb 2

hið Þ<sup>x</sup> hiþ<sup>1</sup>ð Þ<sup>x</sup> ð Þ <sup>γ</sup>b<sup>i</sup>þ<sup>1</sup> � <sup>γ</sup>b<sup>i</sup> 6si

<sup>j</sup> , qjþ1,j <sup>¼</sup> <sup>s</sup>�<sup>1</sup>

. According to Theorem 2.1 of Green and Silverman [45], the

f ¼ Rγ,

ð Þ¼ <sup>x</sup> <sup>b</sup><sup>f</sup> <sup>λ</sup>ð Þ� <sup>x</sup><sup>2</sup> <sup>b</sup><sup>f</sup> <sup>λ</sup>ð Þ <sup>x</sup><sup>1</sup> s1

hið Þx hiþ<sup>1</sup>ð Þx δið Þx <sup>6</sup> ,

hið Þþ x hiþ<sup>1</sup>ð Þx

<sup>6</sup> <sup>δ</sup>ið Þ<sup>x</sup> :

� �. By the definition of natural cubic spline, <sup>f</sup>

<sup>3</sup> ð Þ si�<sup>1</sup> <sup>þ</sup> si , ri,iþ<sup>1</sup> <sup>¼</sup> riþ1,i <sup>¼</sup> <sup>1</sup>

,bf 0

þ

� �γb<sup>i</sup> for <sup>i</sup> <sup>¼</sup> <sup>1</sup>, <sup>2</sup>, <sup>⋯</sup>, n,

þ

hi

for nonparametric model (1). Given xi; yi

<sup>j</sup> <sup>¼</sup> <sup>2</sup>, <sup>⋯</sup>, n � 1, where qj�1,j <sup>¼</sup> <sup>s</sup>�<sup>1</sup>

ð Þ¼ xn 0. Let <sup>γ</sup> <sup>¼</sup> <sup>γ</sup>2; <sup>⋯</sup>; <sup>γ</sup><sup>n</sup>�<sup>1</sup>

00ð Þ <sup>x</sup><sup>1</sup> ; <sup>f</sup>

<sup>j</sup> <sup>¼</sup> <sup>2</sup>, <sup>⋯</sup>, n � <sup>1</sup>, which is given by rii <sup>¼</sup> <sup>1</sup>

When xi <sup>≤</sup> <sup>x</sup> <sup>≤</sup> xiþ1, let <sup>δ</sup>ið Þ¼ <sup>x</sup> <sup>1</sup> <sup>þ</sup> hið Þ<sup>x</sup>

bf 0

(When) x > xn

bf <sup>λ</sup>ð Þ¼ x

<sup>λ</sup>ð Þ¼ <sup>x</sup> <sup>b</sup><sup>f</sup> <sup>λ</sup>ð Þ� xiþ<sup>1</sup> <sup>b</sup><sup>f</sup> <sup>λ</sup>ð Þ xi si

� �<sup>0</sup>

<sup>b</sup><sup>f</sup> <sup>λ</sup>ð Þ¼ <sup>x</sup> <sup>b</sup><sup>f</sup> <sup>λ</sup>ð Þþ <sup>x</sup><sup>1</sup> <sup>h</sup>1ð Þ<sup>x</sup> <sup>b</sup><sup>f</sup> <sup>λ</sup>ð Þ� <sup>x</sup><sup>2</sup> <sup>b</sup><sup>f</sup> <sup>λ</sup>ð Þ <sup>x</sup><sup>1</sup>

00ð Þ <sup>x</sup><sup>2</sup> <sup>⋯</sup>; <sup>f</sup>

vectors f and γ specify a natural cubic spline f if and only if Q<sup>0</sup>

The estimator <sup>γ</sup><sup>b</sup> can be obtained from equation <sup>R</sup> <sup>þ</sup> <sup>λ</sup>Q<sup>0</sup> ð Þ <sup>Q</sup> <sup>γ</sup> <sup>¼</sup> <sup>Q</sup><sup>0</sup>

s1

si

þ

( )

� �γb<sup>i</sup>þ<sup>1</sup> <sup>þ</sup> <sup>1</sup> � hiþ1ð Þ<sup>x</sup>

hið Þx bf <sup>λ</sup>ð Þ� xiþ<sup>1</sup> hiþ<sup>1</sup>ð Þx bf <sup>λ</sup>ð Þ xi si

γ1; γ2; ⋯; γ<sup>n</sup> � � <sup>¼</sup> <sup>f</sup>

f }

bf 0

x < x<sup>1</sup>:

SiZerSS given in Marron and Zhang [33] employed smoothing spline to construct SiZer map

spline estimator is the function bf <sup>λ</sup> that minimizes the regularization criterion over function f :

ð f 00ð Þ<sup>x</sup> h i<sup>2</sup>

<sup>i</sup>¼<sup>1</sup> and a smoothing parameter <sup>λ</sup>, the smoothing

dx: (27)

Model Testing Based on Regression Spline http://dx.doi.org/10.5772/intechopen.74858 95

WY ¼ AλY, (28)

<sup>j</sup> , and qi,j ¼ 0 for j j i � j ≥ 2: Let

<sup>6</sup> si and rij ¼ 0 for j j i � j ≥ 2.

Y. Then estimator bf xð Þ and

� s1 <sup>6</sup> <sup>γ</sup>b2:

W.

} ð Þ¼ x<sup>1</sup>

Step 2. Calculate p-values for each pixel. Each pixel in the 2D map constructed in step 1 is determined by two adjacent knots and a determined m. For pixel tk ð Þ ; tkþ<sup>1</sup>; m ¼ m<sup>0</sup> , we calculate p-value PIk,m<sup>0</sup> and PDk,m<sup>0</sup> for testing hypotheses H<sup>∗</sup> Ik,m<sup>0</sup> and <sup>H</sup><sup>∗</sup> Dk,m<sup>0</sup> , respectively, with m<sup>0</sup> knots.

Step 3. Multiple testing adjustment. For a given value m ¼ m0, carry out multiple testing procedure BYP using p-values PIk,m<sup>0</sup> (PDk,m<sup>0</sup> ), k ¼ 1, 2, ⋯, m0, obtained from step 2 to test the fowling family of hypotheses simultaneously:

$$\left\{ H^\*\_{I1,m\_0}, H^\*\_{I2,m\_0}, \dots, H^\*\_{Im\_0 - 1, m\_0} \right\} \left( H^\*\_{D1, m\_0}, H^\*\_{D2, m\_0}, \dots, H^\*\_{Dm\_0 - 1, m\_0} \right).$$

Step 4. Color pixels. According to the multiple testing results at smoothing level m<sup>0</sup> if H<sup>∗</sup> Ik is rejected and H<sup>∗</sup> Dk is accepted, pixel tk ð Þ ; tkþ<sup>1</sup>; m ¼ m<sup>0</sup> is colored red to indicate significant decreasing. On the contrary, if H<sup>∗</sup> Ik,,m<sup>0</sup> is accepted and <sup>H</sup><sup>∗</sup> Dk,m<sup>0</sup> rejected, pixel tk ð Þ ; tkþ<sup>1</sup>; m ¼ m<sup>0</sup> is colored blue to show significant increasing; purple is used for no significant trend in other cases.

In SiZer map, gray indicates that no sufficient data can be used to test the monotonicity of regression function at point x with m knots. Such sufficiency is quantified as effective sample size (ESS). Noting that the number of nonzero elements in the kth column of G has a demonstrable effect on the inference in interval tk ð Þ ; tkþ<sup>1</sup> , and it is determined directly by how many observations are included in tk ð Þ ; tkþ<sup>1</sup> , we define ESS tk ð Þ ; m as.

$$(ESS(t\_1, m), ESS(t\_2, m), \dots, ESS(t\_m, m))' \triangleq G'G(1, 1, \dots, 1)'.$$

In SiZerLS map, pixel tk ð Þ ; tkþ<sup>1</sup>; m ¼ m<sup>0</sup> would be colored gray if.

$$\min(ESS(t\_k, m\_0), ESS(t\_{k+1}, m\_0)) < 5.1$$

In order to avoid selecting knots, m equally spaced knots or equal x-quantiles are used in interpolation. The smoothing level of regression spline estimate is controlled by m together with the positions of knots. The level of smoothness should be reduced to detect some local fine feature; however, the total number of knots should be limited to avoid excessive under-smoothing in a wide range. In applications of SiZerLS, the range of scales is recommended to include the coarsest smoothing level, m ¼ 2, and the finest smoothing level, avg<sup>x</sup><sup>∈</sup> <sup>T</sup>mmaxESS xð Þ ; mmax < 5.

#### 3.2. Construction of SiZerSS map for exploring features of regression curve

SiZerSS given in Marron and Zhang [33] employed smoothing spline to construct SiZer map for nonparametric model (1). Given xi; yi � �<sup>n</sup> <sup>i</sup>¼<sup>1</sup> and a smoothing parameter <sup>λ</sup>, the smoothing spline estimator is the function bf <sup>λ</sup> that minimizes the regularization criterion over function f :

$$\sum\_{i=1}^{n} w\_i \left[ y\_i - f(\mathbf{x}\_i) \right]^2 + \lambda \int \left[ f'(\mathbf{x}) \right]^2 \mathbf{d} \mathbf{x}.\tag{27}$$

By simple calculation, we can get the estimator vector:

The detailed steps to construct SiZerLS with BYP adjustment are given below:

basis of different values of m and Tm, the 2D map is divided into many pixels.

; ⋯; H<sup>∗</sup>

Ik,,m<sup>0</sup> is accepted and <sup>H</sup><sup>∗</sup>

ð Þ ESS tð Þ <sup>1</sup>; m ; ESS tð Þ <sup>2</sup>; m ; ⋯; ESS tð Þ <sup>m</sup>; m <sup>0</sup>

n o

observations are included in tk ð Þ ; tkþ<sup>1</sup> , we define ESS tk ð Þ ; m as.

In SiZerLS map, pixel tk ð Þ ; tkþ<sup>1</sup>; m ¼ m<sup>0</sup> would be colored gray if.

Im0�1,m<sup>0</sup>

p-value PIk,m<sup>0</sup> and PDk,m<sup>0</sup> for testing hypotheses H<sup>∗</sup>

fowling family of hypotheses simultaneously:

H∗ I1,m<sup>0</sup> ; H<sup>∗</sup> I2,m<sup>0</sup>

rejected and H<sup>∗</sup>

ing. On the contrary, if H<sup>∗</sup>

94 Topics in Splines and Applications

avg<sup>x</sup><sup>∈</sup> <sup>T</sup>mmaxESS xð Þ ; mmax < 5.

Step 1. Construct 2D grid map. Without loss of generality, we assume that designed points xi, i ¼ 1, 2, ⋯, n are chosen from [0, 1]. Then the 2D map is a rectangular area [0, 1; log10ð Þ 1=mmax; , log10ð Þ� 1=mmin ; see BYP SiZerLS displayed in Figure 2. The value of m is determined by the following rule: <sup>m</sup> <sup>¼</sup> round 1=10<sup>l</sup> � �, where function round (∙) is the nearest integer function and <sup>l</sup> takes equally spaced values from interval log10ð Þ <sup>1</sup>=mmin ; � log10ð Þ� <sup>1</sup>=mmax . For a given m, abscissa x takes values at the corresponding knots T<sup>m</sup> ¼ f g t1; t2; ⋯; tm . On the

Step 2. Calculate p-values for each pixel. Each pixel in the 2D map constructed in step 1 is determined by two adjacent knots and a determined m. For pixel tk ð Þ ; tkþ<sup>1</sup>; m ¼ m<sup>0</sup> , we calculate

Step 3. Multiple testing adjustment. For a given value m ¼ m0, carry out multiple testing procedure BYP using p-values PIk,m<sup>0</sup> (PDk,m<sup>0</sup> ), k ¼ 1, 2, ⋯, m0, obtained from step 2 to test the

> H∗ D1,m<sup>0</sup> ; H<sup>∗</sup> D2,m<sup>0</sup>

Step 4. Color pixels. According to the multiple testing results at smoothing level m<sup>0</sup> if H<sup>∗</sup>

In SiZer map, gray indicates that no sufficient data can be used to test the monotonicity of regression function at point x with m knots. Such sufficiency is quantified as effective sample size (ESS). Noting that the number of nonzero elements in the kth column of G has a demonstrable effect on the inference in interval tk ð Þ ; tkþ<sup>1</sup> , and it is determined directly by how many

minð Þ ESS tð Þ <sup>k</sup>; m<sup>0</sup> ; ESS tð Þ <sup>k</sup>þ<sup>1</sup>; m<sup>0</sup> < 5:

In order to avoid selecting knots, m equally spaced knots or equal x-quantiles are used in interpolation. The smoothing level of regression spline estimate is controlled by m together with the positions of knots. The level of smoothness should be reduced to detect some local fine feature; however, the total number of knots should be limited to avoid excessive under-smoothing in a wide range. In applications of SiZerLS, the range of scales is recommended to include the coarsest smoothing level, m ¼ 2, and the finest smoothing level,

blue to show significant increasing; purple is used for no significant trend in other cases.

Ik,m<sup>0</sup> and <sup>H</sup><sup>∗</sup>

Dk is accepted, pixel tk ð Þ ; tkþ<sup>1</sup>; m ¼ m<sup>0</sup> is colored red to indicate significant decreas-

Dk,m<sup>0</sup>

; ⋯; H<sup>∗</sup>

� �

≜ G<sup>0</sup>

Dm0�1,m<sup>0</sup>

Dk,m<sup>0</sup> rejected, pixel tk ð Þ ; tkþ<sup>1</sup>; m ¼ m<sup>0</sup> is colored

Gð Þ 1; 1; ⋯; 1 <sup>0</sup>

:

, respectively, with m<sup>0</sup> knots.

:

Ik is

$$\widehat{\mathbf{f}}\_{\lambda} = \left( \widehat{f}\_{\lambda}(\mathbf{x}\_1), \widehat{f}\_{\lambda}(\mathbf{x}\_2), \dots, \widehat{f}\_{\lambda}(\mathbf{x}\_n) \right) = \left( \mathbf{W} + \lambda \mathbf{K} \right)^{-1} \mathbf{W} \mathbf{Y} = A\_{\lambda} \mathbf{Y}, \tag{28}$$

where weight matrix <sup>W</sup> <sup>¼</sup> diagð Þ <sup>ω</sup>1; <sup>ω</sup>2; <sup>⋯</sup>; <sup>ω</sup><sup>n</sup> and the hat matrix <sup>A</sup><sup>λ</sup> <sup>¼</sup> ð Þ <sup>W</sup> <sup>þ</sup> <sup>λ</sup><sup>K</sup> �<sup>1</sup> W.

In order to construct SiZerSS, the derivative of f at any point x needs to be estimated along with its variance. Let si <sup>¼</sup> xiþ<sup>1</sup> � xi and <sup>n</sup> � ð Þ <sup>n</sup> � <sup>1</sup> matrix <sup>Q</sup> <sup>¼</sup> qij n o, i <sup>¼</sup> <sup>1</sup>, <sup>2</sup>, <sup>⋯</sup>, n, <sup>j</sup> <sup>¼</sup> <sup>2</sup>, <sup>⋯</sup>, n � 1, where qj�1,j <sup>¼</sup> <sup>s</sup>�<sup>1</sup> <sup>j</sup>�<sup>1</sup>, qjj ¼ �s�<sup>1</sup> <sup>j</sup>�<sup>1</sup> � <sup>s</sup>�<sup>1</sup> <sup>j</sup> , qjþ1,j <sup>¼</sup> <sup>s</sup>�<sup>1</sup> <sup>j</sup> , and qi,j ¼ 0 for j j i � j ≥ 2: Let γ1; γ2; ⋯; γ<sup>n</sup> � � <sup>¼</sup> <sup>f</sup> 00ð Þ <sup>x</sup><sup>1</sup> ; <sup>f</sup> 00ð Þ <sup>x</sup><sup>2</sup> <sup>⋯</sup>; <sup>f</sup> 00ð Þ xn � �. By the definition of natural cubic spline, <sup>f</sup> } ð Þ¼ x<sup>1</sup> f } ð Þ¼ xn 0. Let <sup>γ</sup> <sup>¼</sup> <sup>γ</sup>2; <sup>⋯</sup>; <sup>γ</sup><sup>n</sup>�<sup>1</sup> � �<sup>0</sup> . According to Theorem 2.1 of Green and Silverman [45], the vectors f and γ specify a natural cubic spline f if and only if Q<sup>0</sup> f ¼ Rγ,

where R is a (n � 2Þ � ð Þ n � 2 symmetric matrix with elements rij, i ¼ 2, ⋯, n � 1, <sup>j</sup> <sup>¼</sup> <sup>2</sup>, <sup>⋯</sup>, n � <sup>1</sup>, which is given by rii <sup>¼</sup> <sup>1</sup> <sup>3</sup> ð Þ si�<sup>1</sup> <sup>þ</sup> si , ri,iþ<sup>1</sup> <sup>¼</sup> riþ1,i <sup>¼</sup> <sup>1</sup> <sup>6</sup> si and rij ¼ 0 for j j i � j ≥ 2. The estimator <sup>γ</sup><sup>b</sup> can be obtained from equation <sup>R</sup> <sup>þ</sup> <sup>λ</sup>Q<sup>0</sup> ð Þ <sup>Q</sup> <sup>γ</sup> <sup>¼</sup> <sup>Q</sup><sup>0</sup> Y. Then estimator bf xð Þ and bf 0 ð Þ<sup>x</sup> can be written as a linear combination of <sup>b</sup><sup>f</sup> and <sup>γ</sup>b. Let hið Þ¼ <sup>x</sup> <sup>x</sup> � xi, i <sup>¼</sup> <sup>1</sup>, <sup>2</sup>, <sup>⋯</sup>, n: When x < x<sup>1</sup>:

$$
\widehat{f}\_{\lambda}(\mathbf{x}) = \widehat{f}\_{\lambda}(\mathbf{x}\_{1}) + h\_{1}(\mathbf{x}) \left\{ \frac{\widehat{f}\_{\lambda}(\mathbf{x}\_{2}) - \widehat{f}\_{\lambda}(\mathbf{x}\_{1})}{s\_{1}} - \frac{s\_{1}}{6} \widehat{\boldsymbol{\gamma}}\_{2} \right\} \\
\widehat{f}'(\mathbf{x}) = \frac{\widehat{f}\_{\lambda}(\mathbf{x}\_{2}) - \widehat{f}\_{\lambda}(\mathbf{x}\_{1})}{s\_{1}} - \frac{s\_{1}}{6} \widehat{\boldsymbol{\gamma}}\_{2} \dots
$$

When xi <sup>≤</sup> <sup>x</sup> <sup>≤</sup> xiþ1, let <sup>δ</sup>ið Þ¼ <sup>x</sup> <sup>1</sup> <sup>þ</sup> hið Þ<sup>x</sup> si � �γb<sup>i</sup>þ<sup>1</sup> <sup>þ</sup> <sup>1</sup> � hiþ1ð Þ<sup>x</sup> hi � �γb<sup>i</sup> for <sup>i</sup> <sup>¼</sup> <sup>1</sup>, <sup>2</sup>, <sup>⋯</sup>, n,

$$
\widehat{f}\_{\lambda}(\mathbf{x}) = \frac{h\_{i}(\mathbf{x})\widehat{f}\_{\lambda}(\mathbf{x}\_{i+1}) - h\_{i+1}(\mathbf{x})\widehat{f}\_{\lambda}(\mathbf{x}\_{i})}{\mathbf{s}\_{i}} + \frac{h\_{i}(\mathbf{x})h\_{i+1}(\mathbf{x})\delta\_{i}(\mathbf{x})}{6},
$$

$$
\widehat{f}'\_{\lambda}(\mathbf{x}) = \frac{\widehat{f}\_{\lambda}(\mathbf{x}\_{i+1}) - \widehat{f}\_{\lambda}(\mathbf{x}\_{i})}{s\_{i}} + \frac{h\_{i}(\mathbf{x})h\_{i+1}(\mathbf{x})(\widehat{\gamma}\_{i+1} - \widehat{\gamma}\_{i})}{6s\_{i}} + \frac{h\_{i}(\mathbf{x}) + h\_{i+1}(\mathbf{x})}{6}\delta\_{i}(\mathbf{x}) .
$$

(When) x > xn

$$
\widehat{f}\_{\boldsymbol{\lambda}}(\mathbf{x}) = \widehat{f}\_{\boldsymbol{\lambda}}(\mathbf{x}\_n) + \frac{h\_n(\mathbf{x})}{6} \left\{ \frac{\widehat{f}\_{\boldsymbol{\lambda}}(\mathbf{x}\_n) - \widehat{f}\_{\boldsymbol{\lambda}}(\mathbf{x}\_{n-1})}{s\_{n-1}} + s\_{n-1} \widehat{\boldsymbol{\mathcal{V}}}\_{n-1} \right\},
$$

$$
\widehat{f}'\_{\boldsymbol{\lambda}}(\mathbf{x}) = \frac{1}{6} \left\{ \frac{\widehat{f}\_{\boldsymbol{\lambda}}(\mathbf{x}\_n) - \widehat{f}\_{\boldsymbol{\lambda}}(\mathbf{x}\_{n-1})}{s\_{n-1}} + s\_{n-1} \widehat{\boldsymbol{\mathcal{V}}}\_{n-1} \right\}.
$$

The variance of bf 0 <sup>λ</sup>ð Þ<sup>x</sup> can be calculated easily if the estimator of <sup>σ</sup><sup>2</sup>, the variance of the error in model (1), is obtained. <sup>σ</sup>2can be estimated by the sum of squared residuals <sup>P</sup> yi � <sup>b</sup><sup>f</sup> <sup>λ</sup>ð Þ xi � �<sup>2</sup> . If <sup>σ</sup><sup>2</sup> is a function of <sup>x</sup>, <sup>σ</sup><sup>2</sup>ð Þ<sup>x</sup> can be estimated by yi � <sup>b</sup><sup>f</sup> <sup>λ</sup>ð Þ<sup>x</sup> � �<sup>2</sup> . The confidence interval of f <sup>λ</sup> 0 ð Þx are of the form:

$$
\widehat{f}'\_{\lambda}(\mathbf{x}) \pm q \, \widehat{\mathbf{SD}} \left( \widehat{f}\_{\lambda} \, \overset{\prime}{\!(\mathbf{x})} \right), \tag{29}
$$

based on local linear smoother. SiZer map for comparing regression curves is a 2D color map, which consists of a large number of pixels. Each pixel is indexed by a scale (smoothing parameter) and a location; the color of a pixel indicates the result for testing the equality of two or more multiple regression curves at the corresponding location and scale. SiZer provides us with more information about the locations of the differences among the regression curves if they do exist. Park et al. [46] developed an ANOVA-type test statistic and conducted it in scale

The works mentioned above are kernel-based method. Besides it, regression spline is an important smoothing device and is used widely in applications. For a given smoothing parameter m (the number of knots used in regression spline), the p-value for testing the equality of k regression curves at point x is established. Consequently, SiZer-RS is constructed in the same way as SiZerLS for comparing multiple retrogression curves based on higher-order spline

For a given smoothing parameter m (the number of knots used in regression spline), the smoothed curve is defined as f i,mð Þ¼ x Eðbf i,m (x)), where bf i,mð Þx is the regression spline estimator. SiZer-RS for comparing multiple regression curves is based on the testing results for

at point x with smoothing parameter m. Without loss of generality, we still suppose that the explanatory variable x takes value from [0, 1]. On the basis of a knot set

: The estimator of f <sup>i</sup>

βi,sgm,sð Þx ≜ Nmð Þx <sup>0</sup>

ð Þ <sup>t</sup> � <sup>x</sup> <sup>3</sup>

8 < :

For a function gð Þ� , tl�<sup>4</sup>; tl�<sup>3</sup>; tl�<sup>2</sup>; tl�<sup>1</sup>; tl ½ �gð Þ� denotes the fourth-order divided difference of

0, t ≤ x

<sup>i</sup> , in which, Nmð Þ¼ x gm,sð Þx ;s ¼ 1; 2; ⋯; m þ q � 1

, t > x

:

Hm, <sup>x</sup> : f <sup>1</sup>,mð Þ¼ x f <sup>2</sup>,mð Þ¼ x ⋯ ¼ f k,mð Þx , (31)

βm

<sup>i</sup> , (32)

Model Testing Based on Regression Spline http://dx.doi.org/10.5772/intechopen.74858 97

ð Þx at smoothing level m can be

n o: If <sup>q</sup> <sup>¼</sup> 3,

<sup>þ</sup>, l <sup>¼</sup> <sup>2</sup>, <sup>3</sup>, <sup>⋯</sup>, m <sup>þ</sup> <sup>3</sup>,

space for testing the equality of more than two regression curves.

T<sup>m</sup> ¼ f g 0 ¼ t<sup>1</sup> < t2; ⋯; < tm ¼ 1 , we have the approximation:

f i ð Þx ≈

β bm

where tl ¼ tmin max ð Þ ð Þ <sup>l</sup>;<sup>1</sup> ;<sup>m</sup> for l ¼ �2, � 1, ⋯, m þ 3:

<sup>i</sup> <sup>¼</sup> <sup>β</sup>i,1; <sup>β</sup>i, <sup>2</sup>; <sup>⋯</sup>; <sup>β</sup>i,mþp�<sup>1</sup> � �<sup>0</sup>

obtained bf i,mð Þ¼ x Nmð Þx <sup>0</sup>

Nl

Nmð Þx <sup>0</sup> is defined below:

<sup>m</sup><sup>X</sup> þq�1

s¼1

<sup>m</sup>ð Þ¼ <sup>x</sup> ð Þ tl � tl�<sup>4</sup> tl�<sup>4</sup>; tl�<sup>3</sup>; tl�<sup>2</sup>; tl�<sup>1</sup>; tl ½ �ð Þ <sup>t</sup> � <sup>x</sup> <sup>3</sup>

ð Þ <sup>t</sup> � <sup>x</sup> <sup>3</sup> <sup>þ</sup> ¼

interpolation.

where β<sup>m</sup>

gð Þ� , that is:

testing null hypothesis:

where q is based on the nominal level. For details, see Section 3 of Chaudhuri and Marron [25].

SiZerSS can be constructed as SiZerLS. For different values of x, if interval (29) contains zero, pixel ð Þ x; λ is colored purple; if confidence interval is on the right side of zero, blue is used to indicate increasing; otherwise, red is used to imply decreasing. Gray is used to indicate that there is no sufficient data to do reliable inference. The sufficiency can be found in Chaudhuri and Marron [25].

The simulated SiZerLS and SiZerSS maps are displayed in Figure 2, where the red and blue regions locate the bumps of regression curve accurately. This simulation illustrates the good behavior of SiZerLS and SiZerSS in exploring features in data.

#### 3.3. Construction of SiZer-RS map for comparing multiple regression curves

The comparison of two or more populations is a common problem and is of great practical interest in statistics. In this subsection, comparison of multiple regression curves in a general regression setting is developed based on regression spline. Suppose we have <sup>n</sup> <sup>¼</sup> <sup>P</sup> k i¼1 ni independent observations from the following k regression models:

$$y\_{i\dagger} = f\_i(\mathbf{x}\_{i\dagger}) + \sigma\_i(\mathbf{x}\_{i\dagger})\varepsilon\_{i\dagger} \qquad \mathbf{i} = 1, 2, \cdots, k, \quad \mathbf{j} = 1, 2, \cdots, n\_{\rm i} \tag{30}$$

where xij s are covariates, the errors εij � Nð Þ 0; 1 s are independent and identically distributed errors, f <sup>i</sup> ð Þ� is the regression function, and <sup>σ</sup><sup>2</sup> <sup>i</sup> ð Þ� is the conditional variance function of the ith population. We are concerned about whether the k populations in model (30) are equal; if not, what is the difference? To this end, a multi-scale method, SiZer-RS, based on regression spline is proposed to compare f <sup>i</sup> ð Þ� across multiple scales and locations.

As described in Park and Kang [32], the choice of smoothing parameter is also important for comparing regression curves. They developed SiZer for the comparison of regression curves based on local linear smoother. SiZer map for comparing regression curves is a 2D color map, which consists of a large number of pixels. Each pixel is indexed by a scale (smoothing parameter) and a location; the color of a pixel indicates the result for testing the equality of two or more multiple regression curves at the corresponding location and scale. SiZer provides us with more information about the locations of the differences among the regression curves if they do exist. Park et al. [46] developed an ANOVA-type test statistic and conducted it in scale space for testing the equality of more than two regression curves.

The works mentioned above are kernel-based method. Besides it, regression spline is an important smoothing device and is used widely in applications. For a given smoothing parameter m (the number of knots used in regression spline), the p-value for testing the equality of k regression curves at point x is established. Consequently, SiZer-RS is constructed in the same way as SiZerLS for comparing multiple retrogression curves based on higher-order spline interpolation.

For a given smoothing parameter m (the number of knots used in regression spline), the smoothed curve is defined as f i,mð Þ¼ x Eðbf i,m (x)), where bf i,mð Þx is the regression spline estimator. SiZer-RS for comparing multiple regression curves is based on the testing results for testing null hypothesis:

$$H\_{m, \mathbf{x}} : f\_{1, m}(\mathbf{x}) = f\_{2, m}(\mathbf{x}) = \dots = f\_{k, m}(\mathbf{x}), \tag{31}$$

at point x with smoothing parameter m. Without loss of generality, we still suppose that the explanatory variable x takes value from [0, 1]. On the basis of a knot set T<sup>m</sup> ¼ f g 0 ¼ t<sup>1</sup> < t2; ⋯; < tm ¼ 1 , we have the approximation:

$$f\_i(\mathbf{x}) \approx \sum\_{s=1}^{m+q-1} \beta\_{i,s} \mathbf{g}\_{m,s}(\mathbf{x}) \triangleq \mathcal{N}\_m(\mathbf{x})' \beta\_i^m \,\mathrm{.}\tag{32}$$

where β<sup>m</sup> <sup>i</sup> <sup>¼</sup> <sup>β</sup>i,1; <sup>β</sup>i, <sup>2</sup>; <sup>⋯</sup>; <sup>β</sup>i,mþp�<sup>1</sup> � �<sup>0</sup> : The estimator of f <sup>i</sup> ð Þx at smoothing level m can be obtained bf i,mð Þ¼ x Nmð Þx <sup>0</sup> β bm <sup>i</sup> , in which, Nmð Þ¼ x gm,sð Þx ;s ¼ 1; 2; ⋯; m þ q � 1 n o: If <sup>q</sup> <sup>¼</sup> 3, Nmð Þx <sup>0</sup> is defined below:

$$\mathbf{N}\_{m}^{l}(\mathbf{x}) = (t\_{l} - t\_{l-4})[t\_{l-4}, t\_{l-3}, t\_{l-2}, t\_{l-1}, t\_{l}](t - \mathbf{x})\_{+'}^{3} \quad l = 2, 3, \dots, m + 3, \dots$$

where tl ¼ tmin max ð Þ ð Þ <sup>l</sup>;<sup>1</sup> ;<sup>m</sup> for l ¼ �2, � 1, ⋯, m þ 3:

bf <sup>λ</sup>ð Þ¼ x bf <sup>λ</sup>ð Þþ xn

bf 0 <sup>λ</sup>ð Þ¼ x

<sup>σ</sup><sup>2</sup> is a function of <sup>x</sup>, <sup>σ</sup><sup>2</sup>ð Þ<sup>x</sup> can be estimated by yi � <sup>b</sup><sup>f</sup> <sup>λ</sup>ð Þ<sup>x</sup>

behavior of SiZerLS and SiZerSS in exploring features in data.

pendent observations from the following k regression models:

� � <sup>þ</sup> <sup>σ</sup><sup>i</sup> xij

yij ¼ f <sup>i</sup> xij

ð Þ� is the regression function, and <sup>σ</sup><sup>2</sup>

3.3. Construction of SiZer-RS map for comparing multiple regression curves

The variance of bf

96 Topics in Splines and Applications

are of the form:

and Marron [25].

errors, f <sup>i</sup>

is proposed to compare f <sup>i</sup>

0

hnð Þx 6

1 6

> bf 0

bf <sup>λ</sup>ð Þ� xn bf <sup>λ</sup>ð Þ xn�<sup>1</sup> sn�<sup>1</sup>

( )

<sup>λ</sup>ð Þ<sup>x</sup> can be calculated easily if the estimator of <sup>σ</sup><sup>2</sup>, the variance of the error in

� �<sup>2</sup>

bf <sup>λ</sup>ð Þ� xn bf <sup>λ</sup>ð Þ xn�<sup>1</sup> sn�<sup>1</sup>

model (1), is obtained. <sup>σ</sup>2can be estimated by the sum of squared residuals <sup>P</sup> yi � <sup>b</sup><sup>f</sup> <sup>λ</sup>ð Þ xi

<sup>λ</sup>ð Þ� x q:SD

c bf <sup>λ</sup> 0 ð Þx � �

where q is based on the nominal level. For details, see Section 3 of Chaudhuri and Marron [25]. SiZerSS can be constructed as SiZerLS. For different values of x, if interval (29) contains zero, pixel ð Þ x; λ is colored purple; if confidence interval is on the right side of zero, blue is used to indicate increasing; otherwise, red is used to imply decreasing. Gray is used to indicate that there is no sufficient data to do reliable inference. The sufficiency can be found in Chaudhuri

The simulated SiZerLS and SiZerSS maps are displayed in Figure 2, where the red and blue regions locate the bumps of regression curve accurately. This simulation illustrates the good

The comparison of two or more populations is a common problem and is of great practical interest in statistics. In this subsection, comparison of multiple regression curves in a general

where xij s are covariates, the errors εij � Nð Þ 0; 1 s are independent and identically distributed

population. We are concerned about whether the k populations in model (30) are equal; if not, what is the difference? To this end, a multi-scale method, SiZer-RS, based on regression spline

As described in Park and Kang [32], the choice of smoothing parameter is also important for comparing regression curves. They developed SiZer for the comparison of regression curves

ð Þ� across multiple scales and locations.

� �εij, i <sup>¼</sup> <sup>1</sup>, <sup>2</sup>, <sup>⋯</sup>, k, j <sup>¼</sup> <sup>1</sup>, <sup>2</sup>, <sup>⋯</sup>, ni, (30)

<sup>i</sup> ð Þ� is the conditional variance function of the ith

regression setting is developed based on regression spline. Suppose we have <sup>n</sup> <sup>¼</sup> <sup>P</sup>

( )

<sup>þ</sup> sn�<sup>1</sup>γb<sup>n</sup>�<sup>1</sup>

<sup>þ</sup> sn�<sup>1</sup>γb<sup>n</sup>�<sup>1</sup>

:

,

. The confidence interval of f <sup>λ</sup>

, (29)

� �<sup>2</sup>

k i¼1

ni inde-

. If

0 ð Þx

$$(t-x)\_+^3 = \begin{cases} (t-x)^3, t>x\\ \\ 0, \qquad t\le x \end{cases}.$$

For a function gð Þ� , tl�<sup>4</sup>; tl�<sup>3</sup>; tl�<sup>2</sup>; tl�<sup>1</sup>; tl ½ �gð Þ� denotes the fourth-order divided difference of gð Þ� , that is:

$$\begin{cases} [t\_1, t\_2] \mathbf{g} = \mathbf{g}'(t), \text{ if } \quad t\_1 = t\_2 = t \\\\ [t\_1, t\_2] \mathbf{g} = \frac{\mathbf{g}(t\_2) - \mathbf{g}(t\_1)}{t\_2 - t\_1} \text{ otherwise,} \\\\ [t\_1, t\_2, \dots, t\_k] \mathbf{g} = \mathbf{g}^{(k-1)}(t), \text{if } t\_1 = \dots = t\_k \\\\ [t\_1, t\_2, \dots, t\_k] \mathbf{g} = \frac{[t\_2, t\_3, \dots, t\_k] \mathbf{g} - [t\_1, t\_2, \dots, t\_{k-1}] \mathbf{g}}{t\_k - t\_1}, \text{otherwise.} \end{cases}$$

Then model (31) can be approximately written as the following linear regression model:

$$Y\_i = G\_i^{\text{ut}} \beta\_i^{\text{ut}} + \Sigma\_i E\_{i\nu} \tag{33}$$

The p-value for testing hypothesis Hm,x in (35) can be defined as

�

≥ β bm0

Gm<sup>0</sup> <sup>i</sup> Σb �1 2 i,m Ei

is an estimator of the variance matrix of the ith regression model and

<sup>Σ</sup><sup>b</sup> <sup>m</sup> <sup>¼</sup> diag <sup>G</sup><sup>m</sup><sup>0</sup>

is an estimator of the variance matrix of <sup>T</sup><sup>m</sup>ð Þ<sup>x</sup> given <sup>β</sup>

for SiZer-RS, and pixel ð Þ x; m is colored gray if ESS xð Þ ; m < 5:

the structure of regression curve and errors.

<sup>i</sup> <sup>Σ</sup>b�<sup>1</sup> i,mG<sup>m</sup> i � ��<sup>1</sup>

σ<sup>i</sup> xij � �can be found in Li and Xu [36], where the smoothing parameter, mp, can be used as a pilot smoothing parameter, which is different from m used in bf i,mð Þx . SiZer-RS map can be constructed based on different values of mp, which represents the different trade-offs between

The two SiZer maps given in Figure 4 are constructed using the data plotted in Figure 3 to compare three regression curves f <sup>1</sup>ð Þ¼ x f <sup>x</sup>ð Þ¼ x 0, f <sup>3</sup>ð Þ¼ x 0:5sin 2ð Þ πx . Since the variance of errors is a constant, it can be estimated by the sum of squares of residues. In this case, pilot smoothing parameter is avoided [47, 48]. The two blue regions in Figure 4 clearly show their difference across interval (0, 1). The gray color indicates that there is no sufficient data that can be used to get credible testing results at x and nearby. The sufficiency is quantized as ESS xð Þ ; m

� � <sup>¼</sup> P T<sup>m</sup>ð Þ<sup>x</sup> <sup>0</sup>

( )<sup>0</sup>

Lmð Þ<sup>x</sup> <sup>0</sup> Lmð Þ<sup>x</sup> <sup>Σ</sup><sup>b</sup> <sup>m</sup>Lmð Þ<sup>x</sup> <sup>0</sup> h i�<sup>1</sup>

<sup>i</sup> Lmð Þ<sup>x</sup> <sup>0</sup> Lmð Þ<sup>x</sup> <sup>Σ</sup><sup>b</sup> <sup>m</sup>Lmð Þ<sup>x</sup> <sup>0</sup> h i�<sup>1</sup>

� �

Lmð Þ<sup>x</sup> <sup>T</sup><sup>m</sup>ð Þ<sup>x</sup>

Model Testing Based on Regression Spline http://dx.doi.org/10.5772/intechopen.74858 99

<sup>i</sup> g, (36)

� �

<sup>i</sup> , i ¼ 1, 2, ⋯, k: The estimator of

Lmð Þx β bm

; i ¼ 1; 2; ⋯; k

bm <sup>i</sup> , <sup>σ</sup>b<sup>m</sup><sup>2</sup>

, i <sup>¼</sup> <sup>1</sup>, <sup>2</sup>, <sup>⋯</sup>, k;Σbi,m <sup>¼</sup> diag <sup>b</sup>σ<sup>i</sup> xij � �; <sup>j</sup> <sup>¼</sup> <sup>1</sup>; <sup>2</sup>; <sup>⋯</sup>; ni

pm, <sup>x</sup> β bm <sup>i</sup> ; Σb <sup>m</sup>

> <sup>i</sup> <sup>Σ</sup>b�<sup>1</sup> i,mG<sup>m</sup> i � ��<sup>1</sup>

where <sup>T</sup><sup>m</sup>ð Þ<sup>x</sup> <sup>≜</sup> <sup>G</sup><sup>m</sup><sup>0</sup>

Figure 3. 200 observations.

where

$$Y\_i = \left(y\_{i1}, y\_{i2}, \dots, y\_{in\_i}\right)',\\
\mathbf{G}\_i^m = \left(\mathbf{N}\_m^l(\mathbf{x}\_i)\right)\_{n \times (m+2)'},\\\boldsymbol{\Sigma}\_i = \text{diag}\left\{\sigma\_i(\mathbf{x}\_{i\bar{\eta}})\right\},\\\boldsymbol{E}\_i = \left(\varepsilon\_{i1}, \varepsilon\_{i2}, \dots, \varepsilon\_{in\_i}\right)'.$$

At first, we suppose Σ<sup>i</sup> is known and then replace it by its available estimator.

From regression model (33), we can get the estimator β bm <sup>i</sup> <sup>¼</sup> Gm<sup>0</sup> <sup>i</sup> Σ�<sup>1</sup> <sup>i</sup> G<sup>m</sup> i � ��<sup>1</sup> G<sup>m</sup> <sup>i</sup> <sup>0</sup>Σ�<sup>1</sup> <sup>i</sup> Yi: Let b<sup>m</sup> i denote the expectation of β bm i :

$$\mathbf{b}\_i^m = \mathbf{E}\left(\widehat{\boldsymbol{\beta}}\_i^m\right) = \left(\mathbf{G}\_i^m \boldsymbol{\prime} \boldsymbol{\Sigma}\_i^{-1} \mathbf{G}\_i^m\right)^{-1} \mathbf{G}\_i^m \boldsymbol{\prime} \boldsymbol{\Sigma}\_i^{-1} \mathbf{f}\_{i\prime}$$

where f<sup>i</sup> ¼ f <sup>i</sup> ð Þ xi<sup>1</sup> ; ⋯; f <sup>i</sup> xini � � � � <sup>0</sup> . Therefore, the smoothed curve

$$f\_{i,m}(\mathbf{x}) = \mathbb{E}\left(\hat{f}\_{i,m}(\mathbf{x})\right) = \mathbb{E}\left[\mathbf{N}\_m(\mathbf{x})^\prime \left(\mathbf{G}\_i^{\prime\prime}\boldsymbol{\Sigma}\_i^{-1}\mathbf{G}\_i^m\right)^{-1}\mathbf{G}\_i^{m\prime}\boldsymbol{\Sigma}\_i^{-1}\mathbf{Y}\_i\right] = \mathbf{N}\_m(\mathbf{x})^\prime \mathbf{b}\_i^m. \tag{34}$$

Denote <sup>b</sup><sup>m</sup> <sup>¼</sup> <sup>b</sup><sup>m</sup><sup>0</sup> <sup>1</sup> ; <sup>b</sup><sup>m</sup><sup>0</sup> <sup>2</sup> ; <sup>⋯</sup>; <sup>b</sup><sup>m</sup><sup>0</sup> k � �<sup>0</sup> , and correspondingly, denote its estimator as bβ m <sup>¼</sup> <sup>β</sup><sup>m</sup><sup>0</sup> <sup>1</sup> ; <sup>β</sup><sup>m</sup><sup>0</sup> <sup>2</sup> ; <sup>⋯</sup> � β<sup>m</sup><sup>0</sup> <sup>k</sup> Þ 0 . Hypothesis Hm, <sup>x</sup> can be presented as

$$H\_{m, \mathbf{x}} : \quad L\_m(\mathbf{x}) \mathbf{b}^m = \mathbf{0}\_{k-1} \tag{35}$$

where

$$L\_{\mathfrak{m}}(\mathbf{x}) = \begin{bmatrix} N\_{\mathfrak{m}}(\mathbf{x}) & N\_{\mathfrak{m}}(\mathbf{x}) & N\_{\mathfrak{m}}(\mathbf{x}) & N\_{\mathfrak{m}}(\mathbf{x}) \\ -N\_{\mathfrak{m}}(\mathbf{x}) & 0 & 0 & \cdots & 0 \\ 0 & -N\_{\mathfrak{m}}(\mathbf{x}) & 0 & & 0 \\ \cdots & \cdots & \cdots & \cdots & \cdots \\ 0 & 0 & 0 & \cdots & 0 \\ 0 & 0 & 0 & & -N\_{\mathfrak{m}}(\mathbf{x}) \end{bmatrix}$$

is a ð Þ� k � 1 k mð Þ þ q � 1 matrix.

The p-value for testing hypothesis Hm,x in (35) can be defined as

½ � t1; t<sup>2</sup> g ¼ g<sup>0</sup>

, G<sup>m</sup> <sup>i</sup> <sup>¼</sup> <sup>N</sup><sup>l</sup>

From regression model (33), we can get the estimator β

bm <sup>i</sup> ¼ E β bm i � �

� �

bm i :

ð Þ xi<sup>1</sup> ; ⋯; f <sup>i</sup> xini � � � � <sup>0</sup>

f i,mð Þ¼ x E bf i,mð Þx

� �<sup>0</sup>

<sup>2</sup> ; <sup>⋯</sup>; <sup>b</sup><sup>m</sup><sup>0</sup> k

Lmð Þ¼ x

is a ð Þ� k � 1 k mð Þ þ q � 1 matrix.

. Hypothesis Hm, <sup>x</sup> can be presented as

<sup>1</sup> ; <sup>b</sup><sup>m</sup><sup>0</sup>

8

98 Topics in Splines and Applications

>>>>>>>>>>><

>>>>>>>>>>>:

Yi ¼ yi1; yi2; ⋯; yini � �<sup>0</sup>

denote the expectation of β

where f<sup>i</sup> ¼ f <sup>i</sup>

Denote <sup>b</sup><sup>m</sup> <sup>¼</sup> <sup>b</sup><sup>m</sup><sup>0</sup>

β<sup>m</sup><sup>0</sup> <sup>k</sup> Þ 0

where

where

½ � <sup>t</sup>1; <sup>t</sup><sup>2</sup> <sup>g</sup> <sup>¼</sup> g tð Þ� <sup>2</sup> g tð Þ<sup>1</sup>

t<sup>2</sup> � t<sup>1</sup>

<sup>t</sup>1; <sup>t</sup>2; <sup>⋯</sup>; tk ½ �<sup>g</sup> <sup>¼</sup> <sup>g</sup>ð Þ <sup>k</sup>�<sup>1</sup> ð Þ<sup>t</sup> , if t<sup>1</sup> <sup>¼</sup> <sup>⋯</sup> <sup>¼</sup> tk

<sup>t</sup>1; <sup>t</sup>2; <sup>⋯</sup>; tk ½ �<sup>g</sup> <sup>¼</sup> <sup>t</sup>2; <sup>t</sup>3; <sup>⋯</sup>; tk ½ �<sup>g</sup> � ½ � <sup>t</sup>1; <sup>t</sup>2; <sup>⋯</sup>; tk�<sup>1</sup> <sup>g</sup>

Yi <sup>¼</sup> <sup>G</sup><sup>m</sup> <sup>i</sup> β<sup>m</sup>

<sup>m</sup>ð Þ xi � �

At first, we suppose Σ<sup>i</sup> is known and then replace it by its available estimator.

<sup>¼</sup> <sup>G</sup><sup>m</sup> <sup>i</sup> <sup>0</sup>Σ�<sup>1</sup> <sup>i</sup> Gm i � ��<sup>1</sup>

<sup>¼</sup> <sup>E</sup> Nmð Þ<sup>x</sup> <sup>0</sup> <sup>G</sup><sup>m</sup><sup>0</sup>

. Therefore, the smoothed curve

<sup>i</sup> Σ�<sup>1</sup> <sup>i</sup> Gm i � ��<sup>1</sup>

Nmð Þx Nmð Þx Nmð Þx Nmð Þx �Nmð Þx 0 0 ⋯ 0 0 �Nmð Þx 0 0 ⋯⋯⋯ ⋯ 000 ⋯ 0

000 �Nmð Þx

h i

, and correspondingly, denote its estimator as bβ

Then model (31) can be approximately written as the following linear regression model:

ð Þt , if t<sup>1</sup> ¼ t<sup>2</sup> ¼ t

otherwise,

tk � t<sup>1</sup>

ni�ð Þ <sup>m</sup>þ<sup>2</sup> , <sup>Σ</sup><sup>i</sup> <sup>¼</sup> diag <sup>σ</sup><sup>i</sup> xij

bm <sup>i</sup> <sup>¼</sup> Gm<sup>0</sup>

Gm <sup>i</sup> <sup>0</sup>Σ�<sup>1</sup> <sup>i</sup> fi,

> Gm<sup>0</sup> <sup>i</sup> Σ�<sup>1</sup> <sup>i</sup> Yi

Hm, <sup>x</sup> : Lmð Þ<sup>x</sup> <sup>b</sup><sup>m</sup> <sup>¼</sup> <sup>0</sup><sup>k</sup>�<sup>1</sup>, (35)

, otherwise:

<sup>i</sup> þ ΣiEi, (33)

<sup>i</sup> Σ�<sup>1</sup> <sup>i</sup> G<sup>m</sup> i � ��<sup>1</sup>

� � � � , Ei <sup>¼</sup> <sup>ε</sup>i1; <sup>ε</sup>i2; <sup>⋯</sup>; <sup>ε</sup>ini

¼ Nmð Þx <sup>0</sup>

bm

m <sup>¼</sup> <sup>β</sup><sup>m</sup><sup>0</sup> <sup>1</sup> ; <sup>β</sup><sup>m</sup><sup>0</sup> <sup>2</sup> ; <sup>⋯</sup> �

G<sup>m</sup> <sup>i</sup> <sup>0</sup>Σ�<sup>1</sup>

� �<sup>0</sup>

:

<sup>i</sup> Yi: Let b<sup>m</sup>

<sup>i</sup> : (34)

i

$$p\_{m,\mathbf{x}}\left(\widehat{\boldsymbol{\beta}}\_{i}^{m},\widehat{\boldsymbol{\Sigma}}\_{m}\right) = P\left\{T^{m}(\mathbf{x})^{\prime}L\_{m}(\mathbf{x})^{\prime}\left[L\_{m}(\mathbf{x})\widehat{\boldsymbol{\Sigma}}\_{m}L\_{m}(\mathbf{x})^{\prime}\right]^{-1}L\_{m}(\mathbf{x})T^{m}(\mathbf{x})\right\}$$

$$\geq \widehat{\boldsymbol{\beta}}\_{i}^{m\prime}L\_{m}(\mathbf{x})^{\prime}\left[L\_{m}(\mathbf{x})\widehat{\boldsymbol{\Sigma}}\_{m}L\_{m}(\mathbf{x})^{\prime}\right]^{-1}L\_{m}(\mathbf{x})\widehat{\boldsymbol{\beta}}\_{i}^{m}\right\},\tag{36}$$

where <sup>T</sup><sup>m</sup>ð Þ<sup>x</sup> <sup>≜</sup> <sup>G</sup><sup>m</sup><sup>0</sup> <sup>i</sup> <sup>Σ</sup>b�<sup>1</sup> i,mG<sup>m</sup> i � ��<sup>1</sup> Gm<sup>0</sup> <sup>i</sup> Σb �1 2 i,m Ei ( )<sup>0</sup> , i <sup>¼</sup> <sup>1</sup>, <sup>2</sup>, <sup>⋯</sup>, k;Σbi,m <sup>¼</sup> diag <sup>b</sup>σ<sup>i</sup> xij � �; <sup>j</sup> <sup>¼</sup> <sup>1</sup>; <sup>2</sup>; <sup>⋯</sup>; ni � �

is an estimator of the variance matrix of the ith regression model and

$$\widehat{\Sigma}\_{m} = \text{diag}\left\{ \left( G\_{i}^{m} \widehat{\Sigma}\_{i,m}^{-1} G\_{i}^{m} \right)^{-1}, i = 1, 2, \dots, k \right\}.$$

is an estimator of the variance matrix of <sup>T</sup><sup>m</sup>ð Þ<sup>x</sup> given <sup>β</sup> bm <sup>i</sup> , <sup>σ</sup>b<sup>m</sup><sup>2</sup> <sup>i</sup> , i ¼ 1, 2, ⋯, k: The estimator of σ<sup>i</sup> xij � �can be found in Li and Xu [36], where the smoothing parameter, mp, can be used as a pilot smoothing parameter, which is different from m used in bf i,mð Þx . SiZer-RS map can be constructed based on different values of mp, which represents the different trade-offs between the structure of regression curve and errors.

The two SiZer maps given in Figure 4 are constructed using the data plotted in Figure 3 to compare three regression curves f <sup>1</sup>ð Þ¼ x f <sup>x</sup>ð Þ¼ x 0, f <sup>3</sup>ð Þ¼ x 0:5sin 2ð Þ πx . Since the variance of errors is a constant, it can be estimated by the sum of squares of residues. In this case, pilot smoothing parameter is avoided [47, 48]. The two blue regions in Figure 4 clearly show their difference across interval (0, 1). The gray color indicates that there is no sufficient data that can be used to get credible testing results at x and nearby. The sufficiency is quantized as ESS xð Þ ; m for SiZer-RS, and pixel ð Þ x; m is colored gray if ESS xð Þ ; m < 5:

Figure 3. 200 observations.

cases and are consistent under some mild conditions, which means that the p-value tends to be zero when null hypothesis is false as sample size and the number of knots used in spline interpolation tend to be infinity. Hence, the proposed test procedures are performed well

Model Testing Based on Regression Spline http://dx.doi.org/10.5772/intechopen.74858 101

The spline-based method frequently used smoothing method, which can be used easily with other statistical methods. When using the spline-based method, the smoothing level is controlled by the number of knots and their positions. In order to sidestep the determination of knots and obtain more reliable results, multi-scale smoothing methods are proposed based on spline regression to infer structures of regression function. The multi-scale method is a visual method to do inference at different locations and smoothing levels. In addition, the smoothing spline version of multi-scale method is also introduced. The proposed multi-scale method can also be used for comparing multiple regression curves. Some real data examples illustrate the

The MATLAB code of SiZerLL and other versions of SiZer based on kernel smoother is available from the homepage of Professor Marron JS; the MATLAB code of SiZerLS can be

http://www.tandfonline.com/doi/suppl/10.1080/10618600.2014.1001069?scroll=top.

School of Econometrics and Management, University of the Chinese Academy of Sciences,

[1] Härdle W, Mammen E. Comparing nonparametric regression fits. Annals of Statistics.

[3] Cox D, Koh E, Wahba G, Yandell BS. Testing the (parametric) null model hypothesis in (semiparametric) partial and generalized spline models. Annals of Statistics. 1988;16:113-119

[4] Cox D, Koh EA. Smoothing spline based test of model adequacy in polynomial regression.

[5] Fan J, Zhang C, Zhang J. Generalized likelihood ratio statistics and Wilks phenomenon.

[2] Hart JD. Nonparametric Smoothing and Lack-of-Fit Test. New York: Springer; 1997

Annals of the Institute of Statistical Mathematics. 1989;41:383-400

especially in small sample size case.

practicability of the proposed multi-scale method.

downloaded from the following website:

Address all correspondence to: nali@amss.ac.cn

Annals of Statistics. 2001;29:153-193

Author details

Beijing, China

References

1993;21:1926-1947

Na Li

Figure 4. SiZer-RS map.

$$ESS(\mathbf{x}, m) \triangleq \min\_{i=1,2,\cdots,k} \left\{ N\_m(\mathbf{x}) G\_i^{m\prime} G\_i^m(1, 1, \cdots, 1)^{\prime} \right\}.$$

Figure 4 shows that SiZer-RS map can explore the differences between regression curves accurately.

It is worth noting that, for SiZer-RS map, the coarsest smoothing level should be m ¼ q þ 1 to ensure the effectiveness of the qth regression spline and the finest smoothing level is recommend to be the one such that avg<sup>x</sup><sup>∈</sup>½ � <sup>x</sup>1;x2;⋯;xg ESS xð Þ ; <sup>m</sup> <sup>&</sup>lt; 5, where <sup>x</sup>1, x2, <sup>⋯</sup>, xg are points at which hypothesis Hm, <sup>x</sup> is tested and pixels are produced by combing different values of m. In applications, a wide range of values of mp can be used to generate a family of SiZer-RS maps. Particularly, mp and m can both be used as smoothing parameters simultaneously to construct a 3D SiZer-RS map [47, 48].

#### 4. Conclusion

This chapter introduces regression spline method for testing the parametric form of nonparametric regression function in nonparametric, partial linear, and varying-coefficient models, respectively. The corresponded p-values are established based on fiducial method and spline interpolation. The test procedures on the basis of the proposed p-value are accurate in some cases and are consistent under some mild conditions, which means that the p-value tends to be zero when null hypothesis is false as sample size and the number of knots used in spline interpolation tend to be infinity. Hence, the proposed test procedures are performed well especially in small sample size case.

The spline-based method frequently used smoothing method, which can be used easily with other statistical methods. When using the spline-based method, the smoothing level is controlled by the number of knots and their positions. In order to sidestep the determination of knots and obtain more reliable results, multi-scale smoothing methods are proposed based on spline regression to infer structures of regression function. The multi-scale method is a visual method to do inference at different locations and smoothing levels. In addition, the smoothing spline version of multi-scale method is also introduced. The proposed multi-scale method can also be used for comparing multiple regression curves. Some real data examples illustrate the practicability of the proposed multi-scale method.

The MATLAB code of SiZerLL and other versions of SiZer based on kernel smoother is available from the homepage of Professor Marron JS; the MATLAB code of SiZerLS can be downloaded from the following website:

http://www.tandfonline.com/doi/suppl/10.1080/10618600.2014.1001069?scroll=top.
