5.1. Black and white photography

eigenvalues can indicate linear dependencies in the data and cause deformations in the inter-

In general, principal components are not invariant against changes of scale in the original variables, as has been mentioned when referring to the normalized population principal components. Normalizing, or standardizing, the variables consists of performing the following

> xpj�xp ffiffiffiffi spp p h i<sup>t</sup>

matrix whose columns are zj, it can be shown that its sample mean vector is the null vector and that its correlation matrix is the sample correlation matrix, R, of the original variables.

Remark 4.1: Applying that the principal components of the normalized variables are those obtained for the sample observations but substituting the matrix S for R, we can establish that if z1, …, z<sup>n</sup> are the normalized observations, with covariance matrix R ¼ ð Þ rik , where rik is the sample correlation coefficient between observations x<sup>i</sup> and xk, and if the pairs of eigenvalues and eigenvectors of R are

i

6. The proportion of the total sample variance explained by the ith principal component is <sup>v</sup>^<sup>i</sup>

The eigenvalues and eigenvectors of the covariance matrix, or correlation matrix, are the essence of the analysis of principal components, since the eigenvalues indicate the directions of maximum variability and the eigenvectors determine the variances. If a few eigenvalues are much larger than the rest, most of the variance can be explained with less than p variables.

In practice, decisions about the number of components to be considered must be made in terms of the pairs of eigenvalues and eigenvectors of the covariance matrix, or correlation

values there is a decrease with a linear tendency of quite steep slope and that from a certain eigenvalue this decrease is stabilized. That is, there is a point from which the eigenvalues are very similar. The criterion consists of staying with the components that

� �

exclude the small eigenvalues and that are approximately equal.

, j ¼ 1, …, p. If the matrix Z is the p by n

z ¼ u^1iz<sup>1</sup> þ ⋯ þ u^pizp, i ¼ 1, …, p.

ffiffiffiffi v^i

, it has been empirically verified that with the first

<sup>p</sup> , i, k <sup>¼</sup> <sup>1</sup>, …, p.

p .

s<sup>11</sup> <sup>p</sup> ;…;

pretations, calculations, and consequent analysis.

122 Statistics - Growing Data Sets and Growing Demand for Statistics

4.2. Standardized sample principal components

� �, with v^<sup>1</sup> ≥ ⋯ ≥ v^<sup>p</sup> ≥ 0, then

3. The sample covariance of ω^ <sup>i</sup> ð Þ ; ω^ <sup>k</sup> , i 6¼ k, is equal to 0. 4. The total sample variance is trð Þ¼ R p ¼ v^<sup>1</sup> þ ⋯ þ v^p.

5. The sample correlation coefficients between zkand ω^ <sup>i</sup> are rzk,ω^ <sup>i</sup> ¼ u^ki

1. The ith principal component is given by <sup>ω</sup>^ <sup>i</sup> <sup>¼</sup> <sup>u</sup>^<sup>t</sup>

2. The sample variance of ω^ <sup>k</sup> is v^k, k ¼ 1, …, p.

4.3. Criteria for reducing the dimension

matrix, and different rules have been suggested:

a. When performing the graph i; λ^<sup>i</sup>

transformation <sup>z</sup><sup>j</sup> <sup>¼</sup> D x<sup>j</sup> � <sup>x</sup> � � <sup>¼</sup> <sup>x</sup>1j�x<sup>1</sup> ffiffiffiffi

ð Þ v^1; u^<sup>1</sup> ,…, v^p; u^<sup>p</sup>

The black and white photograph shown in Figure 6 was considered. First, the image in .jpg format was converted into the numerical matrix Image of dimension 512 by 512 (i.e., 29 x2<sup>9</sup> ). Second, to obtain the observation vectors, the matrix was divided into blocks of dimension 23x23, Aij, with which 4096 blocks were obtained, and each of them was a vector of observations.

Figure 6. Black and white photograph of Lena.

$$\mathbf{Image} = \begin{bmatrix} \mathbf{A}\_{1,1} & \dots & \mathbf{A}\_{1,64} \\ \vdots & \ddots & \vdots \\ \mathbf{A}\_{64,1} & \dots & \mathbf{A}\_{64,64} \end{bmatrix} \tag{17}$$

<sup>E</sup>^<sup>j</sup> <sup>¼</sup>

canonical basis of the vectors of <sup>B</sup><sup>0</sup> were the columns of the matrix PC <sup>¼</sup> <sup>e</sup>^<sup>t</sup>

components are shown in Figure 8.

½ � <sup>x</sup>1;…; <sup>x</sup><sup>64</sup> <sup>t</sup> <sup>¼</sup> PC <sup>y</sup>1;…; <sup>y</sup><sup>64</sup> � �<sup>t</sup>

to explain 97% of the variability, because P<sup>5</sup>

Therefore, the dimension of y<sup>M</sup> ¼ y � T<sup>M</sup> was 4096 � 64.

want to explain 98% of the total variability.

vectors of B<sup>0</sup>

Third component.

<sup>¼</sup> <sup>y</sup> � PC<sup>t</sup>

2 6 4

^e1,j ⋯ ^e8,j ⋮ ⋱⋮ ^e57,j ⋯ ^e64,j

Each of the 64 matrices E^<sup>j</sup> was converted into an image. The images of the first three principal

At this point, it is important to mention that the data matrix x has been assumed to be formed by 4096 vectors of ℜ<sup>64</sup> expressed in the canonical base, B. Also, the base whose vectors were the eigenvectors of S, B<sup>0</sup> ¼ f g e^1;…; e^<sup>64</sup> , was considered. The coordinates with respect to the

given a vector v that with respect to the canonical base had coordinates ð Þ x1;…; x<sup>64</sup> and with respect to the base <sup>B</sup><sup>0</sup> had coordinates <sup>y</sup>1;…; <sup>y</sup><sup>64</sup> � �, the relation between them was

Thus, the coordinates of the 4096 vectors that formed the observations matrix had as coordinates, with respect to the new base, the rows of the matrix of dimension 4096x64 given by y ¼ x � PC. Eight, in order to reduce the dimension, it was taken into consideration that if we keep all the

change rule, we can consider the first two principal components; five components if we want

In order to compress the image, the first vectors of the base B<sup>0</sup> were used. Moreover, supposing

Ninth, to reconstruct the compressed image, each row of y<sup>M</sup> was regrouped in an 8x8matrix. The ith row of yM, denoted by yMi ¼ ½ � bi,1; …; bi, <sup>8</sup>; bi,9;…; bi, <sup>16</sup>;…; bi, <sup>64</sup> , was transformed into

Figure 8. Images of the matrices of the first three principal components. (a) First component. (b) Second component. (c)

0ð Þ <sup>64</sup>�<sup>M</sup> <sup>x</sup><sup>M</sup> 0ð Þ <sup>64</sup>�<sup>M</sup> x 64 ð Þ �<sup>M</sup> " #

<sup>T</sup><sup>M</sup> <sup>¼</sup> <sup>I</sup>Mx<sup>M</sup> <sup>0</sup>Mx 64 ð Þ �<sup>M</sup>

that we were left with M, M < 64, the matrix T<sup>M</sup> given by Eq. (19) was defined:

<sup>i</sup>¼<sup>1</sup> <sup>λ</sup>^i<sup>=</sup>

, we can perfectly reconstruct our data matrix, because <sup>y</sup> <sup>¼</sup> <sup>x</sup> � PC ) <sup>x</sup> <sup>¼</sup> <sup>y</sup> � PC�<sup>1</sup>

P<sup>64</sup>

. Additionally, for the case under study, to reduce the dimension, if we use the slope

3 7

Application of Principal Component Analysis to Image Compression

. Also, as PC is an orthogonal matrix, <sup>y</sup>1;…; <sup>y</sup><sup>64</sup> � � <sup>¼</sup> ½ � <sup>x</sup>1;…; <sup>x</sup><sup>64</sup> PC.

<sup>5</sup> (18)

http://dx.doi.org/10.5772/intechopen.75007

125

<sup>j</sup>¼<sup>1</sup> <sup>λ</sup>^<sup>j</sup> <sup>¼</sup> <sup>97</sup>%; or eight components if we

<sup>1</sup>;…; e^<sup>t</sup> <sup>64</sup> � �. Then,

(19)

Third, each matrix Aij was stored in a vector of dimension 64, x, which contained the elements of the matrix by rows, that is, x ¼ ½ � ai,1; …; ai, <sup>8</sup>; aiþ1,1; …; aiþ1,8;…; aiþ8,<sup>8</sup> . This way, we had the observations x<sup>k</sup> ∈ ℜ<sup>64</sup>� �<sup>k</sup> <sup>¼</sup> <sup>1</sup>; …; <sup>4096</sup> � �, which were grouped in the observation matrix <sup>x</sup> <sup>¼</sup> xij � � <sup>∈</sup> <sup>Μ</sup>4096,64ð Þ <sup>ℜ</sup> .

Fourth, the average of each column, x ¼ ½ � x1; …; x<sup>64</sup> , was calculated obtaining the vector of means, and from each observation xij, its corresponding mean xj was subtracted. Thus, the matrix of centered observations <sup>U</sup> was obtained. The covariance matrix of <sup>x</sup> was <sup>S</sup> <sup>¼</sup> <sup>U</sup><sup>t</sup> U ∈ Μ64,64ð Þ ℜ .

Fifth, the 64 pairs of eigenvalues and eigenvectors of <sup>S</sup>, <sup>λ</sup>^i; <sup>e</sup>^<sup>i</sup> � �, were found, and they were ordered according to the eigenvalues from highest to lowest. The 8 largest eigenvalues are drawn in Figure 7. As can be seen, the first eigenvalue is much larger than the rest. Thus, the first principal component completely dominates the total variability.

Sixth, with the theoretical results and the calculations previously made, the 64 principal components <sup>y</sup>^<sup>j</sup> <sup>¼</sup> <sup>e</sup>^<sup>t</sup> j x ¼ ^e1,jx<sup>1</sup> þ ⋯ þ ^e64,jxp, j ¼ 1, …, p, were built. The first principal component was <sup>y</sup>^<sup>1</sup> ¼ �0:1167x<sup>1</sup> <sup>þ</sup> <sup>⋯</sup> � <sup>0</sup>:1166x64. Therefore, an orthonormal basis of <sup>ℜ</sup><sup>64</sup> was built.

Seventh, each vector e^<sup>j</sup> ¼ ^e1,j;…; ^e64,j � �<sup>t</sup> was grouped by rows in a matrix Μ8,8:

Figure 7. Graph i; λb<sup>i</sup> � �, <sup>i</sup> <sup>¼</sup> <sup>1</sup>,…, 8, with <sup>λ</sup>b<sup>i</sup> being the eigenvalues ordered from highest to lowest.

$$
\hat{\mathbf{E}}\_{j} = \begin{bmatrix}
\hat{e}\_{1,j} & \cdots & \hat{e}\_{8,j} \\
\vdots & \ddots & \vdots \\
\hat{e}\_{57,j} & \cdots & \hat{e}\_{64,j}
\end{bmatrix} \tag{18}
$$

Each of the 64 matrices E^<sup>j</sup> was converted into an image. The images of the first three principal components are shown in Figure 8.

At this point, it is important to mention that the data matrix x has been assumed to be formed by 4096 vectors of ℜ<sup>64</sup> expressed in the canonical base, B. Also, the base whose vectors were the eigenvectors of S, B<sup>0</sup> ¼ f g e^1;…; e^<sup>64</sup> , was considered. The coordinates with respect to the canonical basis of the vectors of <sup>B</sup><sup>0</sup> were the columns of the matrix PC <sup>¼</sup> <sup>e</sup>^<sup>t</sup> <sup>1</sup>;…; e^<sup>t</sup> <sup>64</sup> � �. Then, given a vector v that with respect to the canonical base had coordinates ð Þ x1;…; x<sup>64</sup> and with respect to the base <sup>B</sup><sup>0</sup> had coordinates <sup>y</sup>1;…; <sup>y</sup><sup>64</sup> � �, the relation between them was ½ � <sup>x</sup>1;…; <sup>x</sup><sup>64</sup> <sup>t</sup> <sup>¼</sup> PC <sup>y</sup>1;…; <sup>y</sup><sup>64</sup> � �<sup>t</sup> . Also, as PC is an orthogonal matrix, <sup>y</sup>1;…; <sup>y</sup><sup>64</sup> � � <sup>¼</sup> ½ � <sup>x</sup>1;…; <sup>x</sup><sup>64</sup> PC. Thus, the coordinates of the 4096 vectors that formed the observations matrix had as coordinates, with respect to the new base, the rows of the matrix of dimension 4096x64 given by y ¼ x � PC.

Eight, in order to reduce the dimension, it was taken into consideration that if we keep all the vectors of B<sup>0</sup> , we can perfectly reconstruct our data matrix, because <sup>y</sup> <sup>¼</sup> <sup>x</sup> � PC ) <sup>x</sup> <sup>¼</sup> <sup>y</sup> � PC�<sup>1</sup> <sup>¼</sup> <sup>y</sup> � PC<sup>t</sup> . Additionally, for the case under study, to reduce the dimension, if we use the slope change rule, we can consider the first two principal components; five components if we want to explain 97% of the variability, because P<sup>5</sup> <sup>i</sup>¼<sup>1</sup> <sup>λ</sup>^i<sup>=</sup> P<sup>64</sup> <sup>j</sup>¼<sup>1</sup> <sup>λ</sup>^<sup>j</sup> <sup>¼</sup> <sup>97</sup>%; or eight components if we want to explain 98% of the total variability.

In order to compress the image, the first vectors of the base B<sup>0</sup> were used. Moreover, supposing that we were left with M, M < 64, the matrix T<sup>M</sup> given by Eq. (19) was defined:

$$\mathbf{T}\_M = \begin{bmatrix} \mathbf{I}\_{M \ge M} & \mathbf{0}\_{M \ge (64-M)} \\ \mathbf{0}\_{(64-M) \ge M} & \mathbf{0}\_{(64-M) \ge (64-M)} \end{bmatrix} \tag{19}$$

Therefore, the dimension of y<sup>M</sup> ¼ y � T<sup>M</sup> was 4096 � 64.

Image ¼

124 Statistics - Growing Data Sets and Growing Demand for Statistics

observations x<sup>k</sup> ∈ ℜ<sup>64</sup>�

� � <sup>∈</sup> <sup>Μ</sup>4096,64ð Þ <sup>ℜ</sup> .

x ¼ xij

nents <sup>y</sup>^<sup>j</sup> <sup>¼</sup> <sup>e</sup>^<sup>t</sup>

Figure 7. Graph i; λb<sup>i</sup>

� �

j

Seventh, each vector e^<sup>j</sup> ¼ ^e1,j;…; ^e64,j

2 6 4

A1, <sup>1</sup> … A1,<sup>64</sup> ⋮ ⋱⋮ A64,<sup>1</sup> … A64, <sup>64</sup>

�<sup>k</sup> <sup>¼</sup> <sup>1</sup>; …; <sup>4096</sup> � �, which were grouped in the observation matrix

Third, each matrix Aij was stored in a vector of dimension 64, x, which contained the elements of the matrix by rows, that is, x ¼ ½ � ai,1; …; ai, <sup>8</sup>; aiþ1,1; …; aiþ1,8;…; aiþ8,<sup>8</sup> . This way, we had the

Fourth, the average of each column, x ¼ ½ � x1; …; x<sup>64</sup> , was calculated obtaining the vector of means, and from each observation xij, its corresponding mean xj was subtracted. Thus, the matrix

ordered according to the eigenvalues from highest to lowest. The 8 largest eigenvalues are drawn in Figure 7. As can be seen, the first eigenvalue is much larger than the rest. Thus, the

Sixth, with the theoretical results and the calculations previously made, the 64 principal compo-

, i ¼ 1,…, 8, with λb<sup>i</sup> being the eigenvalues ordered from highest to lowest.

x ¼ ^e1,jx<sup>1</sup> þ ⋯ þ ^e64,jxp, j ¼ 1, …, p, were built. The first principal component was

of centered observations <sup>U</sup> was obtained. The covariance matrix of <sup>x</sup> was <sup>S</sup> <sup>¼</sup> <sup>U</sup><sup>t</sup>

<sup>y</sup>^<sup>1</sup> ¼ �0:1167x<sup>1</sup> <sup>þ</sup> <sup>⋯</sup> � <sup>0</sup>:1166x64. Therefore, an orthonormal basis of <sup>ℜ</sup><sup>64</sup> was built.

Fifth, the 64 pairs of eigenvalues and eigenvectors of <sup>S</sup>, <sup>λ</sup>^i; <sup>e</sup>^<sup>i</sup>

first principal component completely dominates the total variability.

� �<sup>t</sup>

3 7

� �

was grouped by rows in a matrix Μ8,8:

<sup>5</sup> (17)

U ∈ Μ64,64ð Þ ℜ .

, were found, and they were

Ninth, to reconstruct the compressed image, each row of y<sup>M</sup> was regrouped in an 8x8matrix. The ith row of yM, denoted by yMi ¼ ½ � bi,1; …; bi, <sup>8</sup>; bi,9;…; bi, <sup>16</sup>;…; bi, <sup>64</sup> , was transformed into

Figure 8. Images of the matrices of the first three principal components. (a) First component. (b) Second component. (c) Third component.

the matrix B<sup>i</sup> given by Eq. (20), and the matrix Compressed\_image given by Eq. (21) was built:

$$\mathbf{B}\_{i} = \begin{bmatrix} b\_{i,1} & \cdots & b\_{i,8} \\ b\_{i,9} & \cdots & b\_{i,16} \\ \vdots & \ddots & \vdots \\ b\_{i,57} & \cdots & b\_{i,64} \end{bmatrix} \quad i = 1, \ldots, 4096 \tag{20}$$

Tenth and finally, Eq. (21) was converted into a .jpg file. Figure 9 shows the original image and

By increasing the number of principal components, the percentage of the variability explained is increased by very small percentages, but, nevertheless, nuances are added to the photo sufficiently remarkable, since they make it sharper, smooth out the contours, and mark the

The two methods that we will use are the peak signal-to-noise ratio (PSNR) and the entropy of the error image. The PSNR measure evaluates the quality in terms of deviations between the processed and the original image, and the entropy of an image is a measure of the information

Definition 5.1: Let N be the number of rows by the number of columns in the image. Let

Definition 5.2: Let the images under study be the 8 bit images. The peak signal-to-noise ratio of the

Figure 10 (a) shows PSNR of the reconstructions of the image versus the number of principal components used for the reconstruction, together with the regression line that adjusts the said cloud of points. Figure 10 (b) shows the values of the PSNR when we use three quarters (black), half (red), quarter (blue), eighth (green), sixteenth (brown), and the thirty-second part (yellow) of the components, which means a corresponding reduction in compression. A behavior close to linearity with a slope of approximately 0:2 can be seen. With the reductions

If the entropy is high, the variability of the pixels is very high, and there is little redundancy. Thus, if we exceed a certain threshold in compression, the original image cannot be recovered exactly. If the entropy is small, then the variability will be smaller. Therefore, the information of a pixel with respect to the pixels of its surroundings is high and, therefore, randomness is lost.

Definition 5.3: Let I be an 8 bit image that can take the values f g 0;…; 255 . Let pi be the frequency

n¼1 r 2

<sup>28</sup> � <sup>1</sup> � �<sup>2</sup>

MSE <sup>¼</sup> <sup>1</sup> N X N

PSNR ¼ 10log10

�

Application of Principal Component Analysis to Image Compression

http://dx.doi.org/10.5772/intechopen.75007

127

�<sup>n</sup> <sup>¼</sup> <sup>1</sup>;…; <sup>N</sup> � � be the error. The mean square error (MSE) is

�<sup>n</sup> <sup>¼</sup> <sup>1</sup>;…; <sup>N</sup> � � be the set of

<sup>n</sup> (22)

MSE ! (23)

compressed images with two, five, and eight principal components.

f g xnjn ¼ 1;…; N be the set of pixels of the original image. Let yn

�

5.1.1. Objective measures of the quality of reconstructions

tones more precisely.

reconstruction is

content contained in that image.

reconstruction pixels. Let rn ¼ xn � yn

considered, the PSNR varies between 27 and 63.

with which the value i∈f g 0; …; 255 appears. Then, the entropy is

$$\mathbf{\color{red}{\textbf{Compressed\\_image}}} = \begin{bmatrix} \mathbf{B}\_1 & & \cdots & \mathbf{B}\_{64} \\ \mathbf{B}\_{65} & \cdots & \mathbf{B}\_{128} \\ \vdots & & \ddots & \vdots \\ \mathbf{B}\_{4033} & & \mathbf{B}\_{4096} \end{bmatrix} \tag{21}$$

Figure 9. Original and compressed image with two, five, and eight principal components. (a) Original image. (b) Compression with two components. (c) Compression with five components. (d) Compression with eight components.

Tenth and finally, Eq. (21) was converted into a .jpg file. Figure 9 shows the original image and compressed images with two, five, and eight principal components.

By increasing the number of principal components, the percentage of the variability explained is increased by very small percentages, but, nevertheless, nuances are added to the photo sufficiently remarkable, since they make it sharper, smooth out the contours, and mark the tones more precisely.

#### 5.1.1. Objective measures of the quality of reconstructions

the matrix B<sup>i</sup> given by Eq. (20), and the matrix Compressed\_image given by Eq. (21) was

Figure 9. Original and compressed image with two, five, and eight principal components. (a) Original image. (b) Compression with two components. (c) Compression with five components. (d) Compression with eight components.

B<sup>1</sup> ⋯ B<sup>64</sup> B<sup>65</sup> ⋯ B<sup>128</sup> ⋮ ⋱⋮ B<sup>4033</sup> B<sup>4096</sup>

i ¼ 1, …, 4096 (20)

(21)

bi, <sup>1</sup> ⋯ bi, <sup>8</sup> bi, <sup>9</sup> ⋯ bi, <sup>16</sup> ⋮ ⋱⋮ bi, <sup>57</sup> ⋯ bi, <sup>64</sup>

B<sup>i</sup> ¼

126 Statistics - Growing Data Sets and Growing Demand for Statistics

Compressed\_image ¼

built:

The two methods that we will use are the peak signal-to-noise ratio (PSNR) and the entropy of the error image. The PSNR measure evaluates the quality in terms of deviations between the processed and the original image, and the entropy of an image is a measure of the information content contained in that image.

Definition 5.1: Let N be the number of rows by the number of columns in the image. Let f g xnjn ¼ 1;…; N be the set of pixels of the original image. Let yn � �<sup>n</sup> <sup>¼</sup> <sup>1</sup>;…; <sup>N</sup> � � be the set of reconstruction pixels. Let rn ¼ xn � yn � �<sup>n</sup> <sup>¼</sup> <sup>1</sup>;…; <sup>N</sup> � � be the error. The mean square error (MSE) is

$$MSE = \frac{1}{N} \sum\_{n=1}^{N} r\_n^2 \tag{22}$$

Definition 5.2: Let the images under study be the 8 bit images. The peak signal-to-noise ratio of the reconstruction is

$$PSNR = 10 \log\_{10} \left( \frac{\left(2^8 - 1\right)^2}{MSE} \right) \tag{23}$$

Figure 10 (a) shows PSNR of the reconstructions of the image versus the number of principal components used for the reconstruction, together with the regression line that adjusts the said cloud of points. Figure 10 (b) shows the values of the PSNR when we use three quarters (black), half (red), quarter (blue), eighth (green), sixteenth (brown), and the thirty-second part (yellow) of the components, which means a corresponding reduction in compression. A behavior close to linearity with a slope of approximately 0:2 can be seen. With the reductions considered, the PSNR varies between 27 and 63.

If the entropy is high, the variability of the pixels is very high, and there is little redundancy. Thus, if we exceed a certain threshold in compression, the original image cannot be recovered exactly. If the entropy is small, then the variability will be smaller. Therefore, the information of a pixel with respect to the pixels of its surroundings is high and, therefore, randomness is lost.

Definition 5.3: Let I be an 8 bit image that can take the values f g 0;…; 255 . Let pi be the frequency with which the value i∈f g 0; …; 255 appears. Then, the entropy is

Figure 10. PSNR of the reconstructions according to the used principal components. (a) PSNRof 256 reconstructions. (b) PSNR of some reconstructions.

$$H(I) = -\sum\_{i=0}^{255} p\_i \log\_2(p\_i) \tag{24}$$

the reconstructions using 8 components (black), 16 components (brown), 32 components

Application of Principal Component Analysis to Image Compression

http://dx.doi.org/10.5772/intechopen.75007

129

Finally, we consider the entropy of the images of the errors. Given an image, I, the value of each of its pixels is an element of the set 0f g ;…; <sup>255</sup> , and if we have a reconstruction, ^I, and consider the error, <sup>E</sup> <sup>¼</sup> <sup>I</sup> � ^I, then the value of its pixels will be an element of the set f g �255;…; 255 . Therefore, E cannot be considered as an image. Since a pixel of value eij in E is an error of the same size as �eij, to consider images we denominate image of the error to

Figure 12 (a) shows the entropy of the error image versus the number of principal components used for the reconstruction, together with an adjusted line of slope � 0.02. Figure 12(b) shows the entropy when we use 8 components (black), 16 components (brown), 32 components (green), 64 components (blue), and 128 components (red), respectively. With more than 200 principal components, the entropy of the errors is zero, which means that the errors have very little variability, and with fewer components, the decrease seems linear with slope �0:02.

In this section, we will consider the coordinates of the first vectors that form the principal

vectors will have 64 coordinates. Figure 13 shows the coordinates of the first six principal

As can be seen from Figure 13, all coordinates seem to have some component with period 8. This suggests that there may be some relationship with the shape of the blocks chosen and that most vectors are close to being periodic with period 8, because when we consider each of the

Figure 12. Entropy of the errors of the reconstructions converted into images according to the used principal components.

x2<sup>3</sup> dimension blocks,

components. If we consider that the vectors have been obtained as 23

(green), 64 components (blue), and 128 components (red), respectively.

Imð Þ¼ E eij 

, being <sup>E</sup> <sup>¼</sup> eij .

5.2. Coordinates of the first principal component

components with respect to the canonical base.

(a) Entropy of differences (b) Entropy of some differences.

Figure 11 (a) shows the entropy of the reconstructions from 1 to 256 components. As can be seen, the entropy is increasing until the first 10 components, and then it becomes damped tending asymptotically to the value of the entropy of the image (7:4452). It can be seen that the difference with more than 170 components is insignificant. Figure 11 (b) shows the entropy of

Figure 11. Entropy of reconstructions according to the used principal components. (a) Entropy of reconstructions. (b) Entropy of some reconstructions.

the reconstructions using 8 components (black), 16 components (brown), 32 components (green), 64 components (blue), and 128 components (red), respectively.

Finally, we consider the entropy of the images of the errors. Given an image, I, the value of each of its pixels is an element of the set 0f g ;…; <sup>255</sup> , and if we have a reconstruction, ^I, and consider the error, <sup>E</sup> <sup>¼</sup> <sup>I</sup> � ^I, then the value of its pixels will be an element of the set f g �255;…; 255 . Therefore, E cannot be considered as an image. Since a pixel of value eij in E is an error of the same size as �eij, to consider images we denominate image of the error to Imð Þ¼ E eij , being <sup>E</sup> <sup>¼</sup> eij .

Figure 12 (a) shows the entropy of the error image versus the number of principal components used for the reconstruction, together with an adjusted line of slope � 0.02. Figure 12(b) shows the entropy when we use 8 components (black), 16 components (brown), 32 components (green), 64 components (blue), and 128 components (red), respectively. With more than 200 principal components, the entropy of the errors is zero, which means that the errors have very little variability, and with fewer components, the decrease seems linear with slope �0:02.

#### 5.2. Coordinates of the first principal component

H IðÞ¼�<sup>X</sup>

(b) PSNR of some reconstructions.

128 Statistics - Growing Data Sets and Growing Demand for Statistics

(b) Entropy of some reconstructions.

255

Figure 10. PSNR of the reconstructions according to the used principal components. (a) PSNRof 256 reconstructions.

� � (24)

i¼0 pi log2 pi

Figure 11 (a) shows the entropy of the reconstructions from 1 to 256 components. As can be seen, the entropy is increasing until the first 10 components, and then it becomes damped tending asymptotically to the value of the entropy of the image (7:4452). It can be seen that the difference with more than 170 components is insignificant. Figure 11 (b) shows the entropy of

Figure 11. Entropy of reconstructions according to the used principal components. (a) Entropy of reconstructions.

In this section, we will consider the coordinates of the first vectors that form the principal components. If we consider that the vectors have been obtained as 23 x2<sup>3</sup> dimension blocks, vectors will have 64 coordinates. Figure 13 shows the coordinates of the first six principal components with respect to the canonical base.

As can be seen from Figure 13, all coordinates seem to have some component with period 8. This suggests that there may be some relationship with the shape of the blocks chosen and that most vectors are close to being periodic with period 8, because when we consider each of the

Figure 12. Entropy of the errors of the reconstructions converted into images according to the used principal components. (a) Entropy of differences (b) Entropy of some differences.

Figure 13. Coordinates of the first six principal components with respect to the canonical base. (a) First component. (b) Second component. (c) Third component. (d) Fourth component. (e) Fifth component. (f) Sixth component.

Figure 14. Coordinates of the first three principal components when vectors are constructed from blocks of 22x22 and 24x24. (a) First component 22x22 (b) Second component 22x22 (c) Third component 22x22 (d) First component 24x24 (e)

Application of Principal Component Analysis to Image Compression

http://dx.doi.org/10.5772/intechopen.75007

131

Second component 24x24 (f) Third component 24x24.

Application of Principal Component Analysis to Image Compression http://dx.doi.org/10.5772/intechopen.75007 131

Figure 14. Coordinates of the first three principal components when vectors are constructed from blocks of 22x22 and 24x24. (a) First component 22x22 (b) Second component 22x22 (c) Third component 22x22 (d) First component 24x24 (e) Second component 24x24 (f) Third component 24x24.

Figure 13. Coordinates of the first six principal components with respect to the canonical base. (a) First component.

(b) Second component. (c) Third component. (d) Fourth component. (e) Fifth component. (f) Sixth component.

130 Statistics - Growing Data Sets and Growing Demand for Statistics

4096 vectors of 64 components, the first 8 pixels are adjacent to the next 16 pixels, and these are adjacent to the next 8 pixels, and so on, up to 8 times.

Since the first principal components collect a large part of the characteristics of the vectors, it is plausible that they also reflect the periodicity of the vectors. Recall that principal components are linear combinations of vectors and that if all of them had all their periodic coordinates with the same period, then all components would be periodic as well.

In Figure 14, the coordinates of the first three principal components are shown when the vectors are constructed from blocks of 22x22(see Figure 14 (a-c)) and from blocks of 24x24 (see Figure 14 (d-f)). As can be seen, the periodicity in the first components is again appreciated.

Figure 15. Compression with 2 and 8 original and periodic principal components. (a) Compression with two components (b) Compression with two componentsper. (c) Compression with eight components. (d) Compression with eight componentsper.

Figure 16. Differences between the image and the reconstruction according to the number of chosen components.

Application of Principal Component Analysis to Image Compression

http://dx.doi.org/10.5772/intechopen.75007

133

(a) 1-norm (b) 2-norm (c) ∞-norm.

4096 vectors of 64 components, the first 8 pixels are adjacent to the next 16 pixels, and these are

Since the first principal components collect a large part of the characteristics of the vectors, it is plausible that they also reflect the periodicity of the vectors. Recall that principal components are linear combinations of vectors and that if all of them had all their periodic coordinates with

In Figure 14, the coordinates of the first three principal components are shown when the vectors are constructed from blocks of 22x22(see Figure 14 (a-c)) and from blocks of 24x24 (see Figure 14 (d-f)). As can be seen, the periodicity in the first components is again appreciated.

Figure 15. Compression with 2 and 8 original and periodic principal components. (a) Compression with two components (b) Compression with two componentsper. (c) Compression with eight components. (d) Compression with eight

componentsper.

adjacent to the next 8 pixels, and so on, up to 8 times.

132 Statistics - Growing Data Sets and Growing Demand for Statistics

the same period, then all components would be periodic as well.

Figure 16. Differences between the image and the reconstruction according to the number of chosen components. (a) 1-norm (b) 2-norm (c) ∞-norm.
