1. Introduction

The quality control and inspection of the high-speed printing process is a critical topic in various industries ranging from electronics to pharmaceutical products. The advent of higher quality requirements from end users and higher costs of raw materials has limited the profit margin of labeling producers. The label producers need to improve the quality of their products and improve the production efficiency by keeping the cost and management thresholds to a minimum level. Typical industrial label printing mechanisms include offset-printing, letterpress-printing, combination-printing, and flexography printing. Various challenges faced by the

labels are the limited sized of text-containing region, relatively small size of labels, clear text, ink-flow control for same shades, and text alignment. Usually, the common defects encountered in the printing process are misprinting, text fading, and various shades in printed text. According to the report [1], in 2010 the Chinese pharmaceutical label market demand was more than 165 million m<sup>2</sup> , and the increase is predicted to be 10–12% in the next 3–5 years. Europe and America had a market demand of 300 million m<sup>2</sup> and 250 million m<sup>2</sup> , respectively, and both are expected to increase by about 3% annually. It is evident that the market demand of pharmaceutical labels in western countries is higher than in China.

a statistical distribution estimation-based algorithm, while the second is a function approximation-based algorithm. However, the preliminary experiment based only on face detection and face-based quality modeling yielded encouraging results. However, more work needs to be done in the areas such as object detection, feature

SAML-QC: A Stochastic Assessment and Machine Learning-Based QC Technique for Industrial…

As an efficient alternative, the machine vision systems can filter out the physical

The process consists of image acquisition, filtering, segmentation, and image matching. In this chapter, JSEG [9] algorithm is used to segment the textile printing images with obvious texture and then make a match for the segmented edge information, to obtain deviation area of the image and calculate the deviation position. However, in their experiments, several limitations are found for the system. One case is when the similar colors between the two neighbor regions cannot be segmented. The work in [10] introduced the algorithm based on wavelet packet and the regional analysis. It analyzes the significance of the threshold of binarization in defect detection combined with morphology area analysis method to extract specific features of defects and determine the type of defect. In [11] a study was conducted to evaluate how accurately a machine learning method can predict human judgment about printed pattern quality. Similarly [12] also proposed

The block diagram of the proposed scheme is shown in Figure 1. It shows that the algorithm receives the input image and performs pre-filtering. The pre-filtering process starts by RGB to gray scale conversion and resizing the input image to a width of 500 pixels without losing the aspect ratio. A Gaussian blur (of kernel size

9 11) filters the salt and pepper noise in the image. Then the histogram

limitations and subjective judgmental decisions of humans. In [7] an image processing technique for the development of a low-cost machine vision system is explored for the inspection of the pharmaceutical capsule. This work discusses the two-part gelatin capsule inspection system by using image processing techniques for border tracing and approximation of the capsule to a circle. A quality control feedback performs pass/reject decision and puts capsules to the appropriate bin. In [8] a new approach for detecting the printing accuracy based on the technology of

selection, and machine learning to better establish this method.

machine learning-based quality assessment scheme.

3. Proposed SAML-QC scheme

Figure 1.

105

Block diagram of SAML-QC algorithm.

machine vision is presented.

DOI: http://dx.doi.org/10.5772/intechopen.85680

The Chinese market is expected to witness an increase in demand in the coming years. It is also reported that in the past 3 years, the labor costs have increased dramatically and QC management has become more difficult. Manual QC inspection can be easily affected by factors such as lack of experience, emotions, human eye perception, and environmental conditions. These factors lead to an inconsistent and poor standard of inspection. Although a review process is usually added in the inspection process in order to recheck the misprinted labels and quality defects, it cannot guarantee 100% quality inspection and also leads to extra labor costs and time consumption. In order to win the customer satisfaction, the inspection QC should be improved through automation. Even tiny mistakes in the label printing can put a bad impression in the overall product quality and can reduce a good customer relationship. Therefore, automatic inspection system is an obvious choice for the industries, and it will have room to grow in the near future.

The proposed scheme performs quality inspection in three steps. The first step is related to the inspection of printed text alignment with respect to the industrial object. Although the requirement for printing alignment is company specific, the proposed mechanism is adaptable and adjustable according to the specific requirement. The second step performs quality inspection based on detection of varying printing shades in the detected text. The third step is aided by supervised machine learning, and it performs the detection of misprinted text. The rest of the paper is composed as follows. Section 2 addresses the related work for computer-aided quality inspection of printing. Section 3 explains the SAML-QC algorithm. Section 4 shows the results of performing the proposed inspection on a given industrial object. Finally, Section 5 presents conclusion and future work.

#### 2. Related works

Recently, a few researches have investigated computer-aided detection and image quality assessment. In [2] a technique based on the comparison of an inspected document with its referential version is discussed. In [3] an image quality assessment algorithm is proposed that does not rely on reference images, and its general framework emulates human quality assessment by first detecting visual components and then assessing quality against an empirical model for face detection. In the algorithm of Rowley et al. [4], a neural network is trained to detect face patterns in a region of 20-by-20 pixels. The determination of an arbitrary image such that if the square is a face region, the square is down-sampled to the size of 20 by 20 and equalized, resulting in a normalized signal.

A similar proposition holds for the face detector of Viola and Jones [5]. In their algorithm a feature-based classifier is trained over a square box of 24-by-24 pixels, with each feature defined by a feature template composed of a group of rectangular sub-windows. In [6], two machine learning algorithms are used for feature selection: mixture of Gaussian and radial basis function (RBF). The first one is

#### SAML-QC: A Stochastic Assessment and Machine Learning-Based QC Technique for Industrial… DOI: http://dx.doi.org/10.5772/intechopen.85680

a statistical distribution estimation-based algorithm, while the second is a function approximation-based algorithm. However, the preliminary experiment based only on face detection and face-based quality modeling yielded encouraging results. However, more work needs to be done in the areas such as object detection, feature selection, and machine learning to better establish this method.

As an efficient alternative, the machine vision systems can filter out the physical limitations and subjective judgmental decisions of humans. In [7] an image processing technique for the development of a low-cost machine vision system is explored for the inspection of the pharmaceutical capsule. This work discusses the two-part gelatin capsule inspection system by using image processing techniques for border tracing and approximation of the capsule to a circle. A quality control feedback performs pass/reject decision and puts capsules to the appropriate bin. In [8] a new approach for detecting the printing accuracy based on the technology of machine vision is presented.

The process consists of image acquisition, filtering, segmentation, and image matching. In this chapter, JSEG [9] algorithm is used to segment the textile printing images with obvious texture and then make a match for the segmented edge information, to obtain deviation area of the image and calculate the deviation position. However, in their experiments, several limitations are found for the system. One case is when the similar colors between the two neighbor regions cannot be segmented. The work in [10] introduced the algorithm based on wavelet packet and the regional analysis. It analyzes the significance of the threshold of binarization in defect detection combined with morphology area analysis method to extract specific features of defects and determine the type of defect. In [11] a study was conducted to evaluate how accurately a machine learning method can predict human judgment about printed pattern quality. Similarly [12] also proposed machine learning-based quality assessment scheme.

#### 3. Proposed SAML-QC scheme

The block diagram of the proposed scheme is shown in Figure 1. It shows that the algorithm receives the input image and performs pre-filtering. The pre-filtering process starts by RGB to gray scale conversion and resizing the input image to a width of 500 pixels without losing the aspect ratio. A Gaussian blur (of kernel size 9 11) filters the salt and pepper noise in the image. Then the histogram

Figure 1. Block diagram of SAML-QC algorithm.

labels are the limited sized of text-containing region, relatively small size of labels, clear text, ink-flow control for same shades, and text alignment. Usually, the common defects encountered in the printing process are misprinting, text fading, and various shades in printed text. According to the report [1], in 2010 the Chinese

increase is predicted to be 10–12% in the next 3–5 years. Europe and America had a

expected to increase by about 3% annually. It is evident that the market demand of

years. It is also reported that in the past 3 years, the labor costs have increased dramatically and QC management has become more difficult. Manual QC inspection can be easily affected by factors such as lack of experience, emotions, human eye perception, and environmental conditions. These factors lead to an inconsistent and poor standard of inspection. Although a review process is usually added in the inspection process in order to recheck the misprinted labels and quality defects, it cannot guarantee 100% quality inspection and also leads to extra labor costs and time consumption. In order to win the customer satisfaction, the inspection QC should be improved through automation. Even tiny mistakes in the label printing can put a bad impression in the overall product quality and can reduce a good customer relationship. Therefore, automatic inspection system is an obvious choice

The Chinese market is expected to witness an increase in demand in the coming

The proposed scheme performs quality inspection in three steps. The first step is related to the inspection of printed text alignment with respect to the industrial object. Although the requirement for printing alignment is company specific, the proposed mechanism is adaptable and adjustable according to the specific requirement. The second step performs quality inspection based on detection of varying printing shades in the detected text. The third step is aided by supervised machine learning, and it performs the detection of misprinted text. The rest of the paper is composed as follows. Section 2 addresses the related work for computer-aided quality inspection of printing. Section 3 explains the SAML-QC algorithm. Section 4 shows the results of performing the proposed inspection on a given industrial object. Finally, Section 5 presents conclusion and future work.

Recently, a few researches have investigated computer-aided detection and image quality assessment. In [2] a technique based on the comparison of an

inspected document with its referential version is discussed. In [3] an image quality assessment algorithm is proposed that does not rely on reference images, and its general framework emulates human quality assessment by first detecting visual components and then assessing quality against an empirical model for face detection. In the algorithm of Rowley et al. [4], a neural network is trained to detect face patterns in a region of 20-by-20 pixels. The determination of an arbitrary image such that if the square is a face region, the square is down-sampled to the size of

A similar proposition holds for the face detector of Viola and Jones [5]. In their algorithm a feature-based classifier is trained over a square box of 24-by-24 pixels, with each feature defined by a feature template composed of a group of rectangular

sub-windows. In [6], two machine learning algorithms are used for feature selection: mixture of Gaussian and radial basis function (RBF). The first one is

20 by 20 and equalized, resulting in a normalized signal.

, and the

, respectively, and both are

pharmaceutical label market demand was more than 165 million m<sup>2</sup>

Computer Architecture in Industrial, Biomechanical and Biomedical Engineering

pharmaceutical labels in western countries is higher than in China.

for the industries, and it will have room to grow in the near future.

2. Related works

104

market demand of 300 million m<sup>2</sup> and 250 million m<sup>2</sup>

equalization process enhances the contrast. The next step is the initialization of structuring kernels. These kernels play a major role in the morphological closing operation. This operation combines the text-containing region and supports the detection of text-containing region for the alignment testing. The next step is the initialization of TopHat morphology [13] to find the white regions against the darker ones.

difference in terms of pixels between the top edge of industrial object and top of the bounding box of the text region is given by Eq. (1), and similarly, the absolute

SAML-QC: A Stochastic Assessment and Machine Learning-Based QC Technique for Industrial…

The proposed criterion for text alignment is based on threshold value for vertical

The value of LThresh is set as Wo/4. It means that it contains the width of one-

and horizontal alignments. Let UDThresh represent threshold for up-down or vertical alignment testing. The value of UDThresh is set as h. Let LThresh represent

fourth of the full width of object. The decision boundaries for horizontal and vertical alignment passed or failed are given as follows. If abs{(U-D)} ≤ UDThresh, then the vertical alignment is considered as passed else it is failed. If L ≤ LThresh, then the horizontal alignment is considered as passed, else it is considered as failed.

After the assessment of alignment, the next step is to perform the optical

text-containing region (xt,yt, x + h,w + h) of the input image. The median filter is applied to remove any salt paper noise in the image. The next step is to perform the Gaussian blur with emphasis in the vertical direction by selecting the kernel size of 1 � 5. The reason behind this step is to exploit the nature of text printing. For example, in Figure 2 the vertical difference between the TEXT CONTAINING and REGION IS HERE is more than the horizontal difference between consecutive individual letters. The next step in the process of obtaining optimum image ready to perform optical character recognition (OCR) is to perform the image contrast

The proposed scheme used bilateral filtering [16] to improve the text regions while suppressing the background. The bilateral filtering alters the intensity value of each pixel with the weighted average of its neighbors that is Gaussian distributed. The next step is to perform the gray scale conversion and perform Otsu's autothresholding and bit-wise logical NOT operation to obtain binary image. This image is saved as a PNG format. This PNG image is provided as input image to the state-

performing OCR is to get bounding boxes and location of anything that looks like a character. As a result an html file containing the position and sizes of each detected letter is generated. Figure 3(a) and (b) shows an image that is cropped

of-the-art Tesseract open-source OCR engine v3.02 [17]. The purpose of

The detection of bounding boxes of letters: (a) text area selected; and (b) detection of letters.

character recognition. For this purpose the proposed scheme crops the

U ¼ abs yo f g ð Þ � yt (1)

L ¼ abs xo f g ð Þ � xt (3)

D ¼ abs yo f g ð Þ� þ Ho ð Þ yt þ h (2)

differences for down and left are given by Eqs. (2) and (3):

the left-right or horizontal alignment.

DOI: http://dx.doi.org/10.5772/intechopen.85680

3.2 Optical character recognition

enhancement.

Figure 3.

107

It follows the computation of Scharr gradient [14] of the TopHat image for edge detection. The proposed scheme used the Sobel operator in horizontal axis to calculate the absolute value element-wise. The minimum and maximum values of Scharr gradient are obtained followed by scaling to the range 0–255 per pixel value. The next step performs the morphological closing operation to fill the gaps. It is followed by Otsu's auto-thresholding [15] to binaries the image. The morphological closing and dilation process is performed to convert the text-containing region as a combined blob. It is intuitive that this combined block contains the actual area of the text and makes the image ready for the detection of contours. The purpose of finding contours in the image is to detect the text-containing region inside the given object. After that, the text-containing region is cropped and stored in a buffer for later use in stochastic assessment. The text-containing region is identified by selecting the bounding box with optimum aspect ratio in the process of iterating over the found contours. The red rectangle is displayed on the image of the selected text. The coordinates of the text-containing region are used for the testing of text alignment.

#### 3.1 Text alignment assessment

In printing industry the common text alignment errors are related to the horizontal and vertical alignments. In the current paper, it is assumed that the best position of the text is right at the center of the object. Therefore, any text region that is printed too much horizontally or vertically should be identified. However, the height and width of the industrial objects and their respective printed texts are different due to the varying nature of performance of the printing labelers as discussed in [10]. Therefore, the proposed algorithm presents a mathematical formulation to detect alignment error of the text inside an industrial object. Figure 2 shows an abstract image of an industrial object (red portion) with markers showing the dimensions. The width and height of the industrial object are represented by Wo and Ho, respectively. The width and height of text-containing region are represented by w and h. Let (xo,yo) and (xt,yt) represent the top left corners for the object and text boxes, respectively, as shown in Figure 2. The absolute

Figure 2. Parameters for the alignment test of text-containing region.

SAML-QC: A Stochastic Assessment and Machine Learning-Based QC Technique for Industrial… DOI: http://dx.doi.org/10.5772/intechopen.85680

difference in terms of pixels between the top edge of industrial object and top of the bounding box of the text region is given by Eq. (1), and similarly, the absolute differences for down and left are given by Eqs. (2) and (3):

$$U = abs\{ (yo - \jmath t) \}\tag{1}$$

$$D = abs\{ (yo + Ho) - (yt + h) \}\tag{2}$$

$$L = abs\{ (\mathbf{x}o - \mathbf{x}t) \}\tag{3}$$

The proposed criterion for text alignment is based on threshold value for vertical and horizontal alignments. Let UDThresh represent threshold for up-down or vertical alignment testing. The value of UDThresh is set as h. Let LThresh represent the left-right or horizontal alignment.

The value of LThresh is set as Wo/4. It means that it contains the width of onefourth of the full width of object. The decision boundaries for horizontal and vertical alignment passed or failed are given as follows. If abs{(U-D)} ≤ UDThresh, then the vertical alignment is considered as passed else it is failed. If L ≤ LThresh, then the horizontal alignment is considered as passed, else it is considered as failed.

#### 3.2 Optical character recognition

equalization process enhances the contrast. The next step is the initialization of structuring kernels. These kernels play a major role in the morphological closing operation. This operation combines the text-containing region and supports the detection of text-containing region for the alignment testing. The next step is the initialization of TopHat morphology [13] to find the white regions against the

Computer Architecture in Industrial, Biomechanical and Biomedical Engineering

It follows the computation of Scharr gradient [14] of the TopHat image for edge detection. The proposed scheme used the Sobel operator in horizontal axis to calculate the absolute value element-wise. The minimum and maximum values of Scharr gradient are obtained followed by scaling to the range 0–255 per pixel value. The next step performs the morphological closing operation to fill the gaps. It is followed by Otsu's auto-thresholding [15] to binaries the image. The morphological closing and dilation process is performed to convert the text-containing region as a combined blob. It is intuitive that this combined block contains the actual area of the text and makes the image ready for the detection of contours. The purpose of finding contours in the image is to detect the text-containing region inside the given object. After that, the text-containing region is cropped and stored in a buffer for later use in stochastic assessment. The text-containing region is identified by selecting the bounding box with optimum aspect ratio in the process of iterating over the found contours. The red rectangle is displayed on the image of the selected text. The coordinates of the text-containing region are used for the testing of

In printing industry the common text alignment errors are related to the horizontal and vertical alignments. In the current paper, it is assumed that the best position of the text is right at the center of the object. Therefore, any text region that is printed too much horizontally or vertically should be identified. However, the height and width of the industrial objects and their respective printed texts are different due to the varying nature of performance of the printing labelers as discussed in [10]. Therefore, the proposed algorithm presents a mathematical formulation to detect alignment error of the text inside an industrial object. Figure 2 shows an abstract image of an industrial object (red portion) with markers showing the dimensions. The width and height of the industrial object are represented by Wo and Ho, respectively. The width and height of text-containing region are represented by w and h. Let (xo,yo) and (xt,yt) represent the top left corners for the object and text boxes, respectively, as shown in Figure 2. The absolute

darker ones.

text alignment.

Figure 2.

106

Parameters for the alignment test of text-containing region.

3.1 Text alignment assessment

After the assessment of alignment, the next step is to perform the optical character recognition. For this purpose the proposed scheme crops the text-containing region (xt,yt, x + h,w + h) of the input image. The median filter is applied to remove any salt paper noise in the image. The next step is to perform the Gaussian blur with emphasis in the vertical direction by selecting the kernel size of 1 � 5. The reason behind this step is to exploit the nature of text printing. For example, in Figure 2 the vertical difference between the TEXT CONTAINING and REGION IS HERE is more than the horizontal difference between consecutive individual letters. The next step in the process of obtaining optimum image ready to perform optical character recognition (OCR) is to perform the image contrast enhancement.

The proposed scheme used bilateral filtering [16] to improve the text regions while suppressing the background. The bilateral filtering alters the intensity value of each pixel with the weighted average of its neighbors that is Gaussian distributed. The next step is to perform the gray scale conversion and perform Otsu's autothresholding and bit-wise logical NOT operation to obtain binary image. This image is saved as a PNG format. This PNG image is provided as input image to the stateof-the-art Tesseract open-source OCR engine v3.02 [17]. The purpose of performing OCR is to get bounding boxes and location of anything that looks like a character. As a result an html file containing the position and sizes of each detected letter is generated. Figure 3(a) and (b) shows an image that is cropped

Figure 3. The detection of bounding boxes of letters: (a) text area selected; and (b) detection of letters.

and processed by the stated procedure to get the detected boxes of letters in the text and marked with green rectangles.

The expression in Eq. (8) can be defined in terms of q(y) as shown in Eq. (9):

SAML-QC: A Stochastic Assessment and Machine Learning-Based QC Technique for Industrial…

� <sup>1</sup>=2σ<sup>2</sup> ð Þ <sup>H</sup>�σ2=σ<sup>n</sup>

<sup>2</sup> ∑ K i¼1 yi

(9)

<sup>2</sup> (10)

yi (11)

yi (12)

yi (13)

� �<sup>2</sup>

2 σn

f Hð Þ¼ j y q y ð Þe

<sup>σ</sup><sup>2</sup> <sup>¼</sup> <sup>1</sup>

HMAP <sup>¼</sup> <sup>σ</sup><sup>2</sup>

<sup>σ</sup><sup>2</sup> <sup>¼</sup> <sup>σ</sup><sup>H</sup>

KσH<sup>2</sup> þ σ<sup>n</sup> <sup>2</sup> ð Þ <sup>σ</sup><sup>H</sup>

It should be noted that q(y) is only the function of y; therefore, the best estimate HMAP of H is the value where f(H|y) gets the highest peak, which is obtained from

> =σ<sup>n</sup> <sup>2</sup> ∑ K i¼1

2

1 <sup>K</sup> <sup>∑</sup> K i¼1

HMAP ¼ f g hMAPð Þ1 ; hMAPð Þ2 ; hMAPð Þ3 ; :……; hMAPð Þ B (14)

hMAPð Þ u P Hf g MAP ¼ hMAPð Þ u (15)

<sup>2</sup> h i (16)

σH<sup>2</sup> þ σ<sup>n</sup> <sup>2</sup> ð Þ =K

<sup>2</sup>=K, then the best estimate of H is given as Eq. (13):

HMAP <sup>≈</sup> <sup>1</sup>

<sup>K</sup> <sup>∑</sup> K i¼1

HMAP is the best estimate of intensity levels of the character-containing region of a detected box, yet its value varies for every other detected box, and its probability distribution is also unknown, so it can also be assumed as a random variable that follows Gaussian distribution. If B represents the number of detected character boxes in a given image, then the set of real values hMAP(u) assigned to HMAP for all detected boxes is shown in Eq. (14), where u represents the index of each box.

Let E [HMAP] be the expectation of HMAP, and it is expressed as Eq. (15). The

HMAP ¼ E Hð Þ MAP � E H½ � MAP

Definition 1: (Set of bad boxes) The set of all those boxes such that members do

Here n represents the quality index, and smaller value of n corresponds to higher

The selection of bad box is performed by Eq. (17), as one of the main concerns in the quality inspection of label printing is to find the overly printed or faded printed

ð Þ E H½ �� MAP nσHMAP ≤ hMAPð Þ u ≤ð Þ E H½ �þ MAP nσHMAP (17)

The notation σ is given by Eq. (10):

DOI: http://dx.doi.org/10.5772/intechopen.85680

Eq. (10) can be solved to get Eq. (12):

variance σ 2HMAP can be found in Eq. (16):

demand of quality of label printing,

109

E H½ �¼ MAP ∑

σ2

B u¼1

not satisfy the conditions in Eq. (17) is called set of bad boxes.

Eq. (11) when H = HMAP:

If σ<sup>S</sup>

<sup>2</sup> ≫ σ<sup>n</sup>

#### 3.3 Stochastic assessment of printing quality

The next block performs the stochastic assessment of the printing quality in terms of detection of overly faded printed text due to the imperfections in printing process as discussed in [1].

#### 3.3.1 Detection of printing in varying shades

The intensity value of each printed pixel can be considered as a random variable H sampled at time t, and since its distribution is also unknown, therefore, it follows Gaussian distribution. From [18] it is clear that a sequence of random variables is independent and identically distributed (IID), if every random variable has the identical probability distribution and all are mutually independent. Thus it is also assumed that H is IID Gaussian random variable. The noise related to the intensity variation is Gaussian in nature; hence the mean and the acquired image also have additive Gaussian noise ni with H. In order to estimate the parameter H, from parametric space to the estimation space H<sup>0</sup> via observation space y(H,t) at any time t, the maximum a posteriori probability (MAP) estimation [19] is used. MAP maximizes the a posteriori probability, which means most likely the value of H is given by Eq. (4):

$$\max\_{\{H\}} f(H|\mathcal{y}) = \max\_{\{H\}} \left\{ \frac{f(\mathcal{y}|H)f(H)}{f(\mathcal{y})} \right\} \tag{4}$$

It is quite clear that the probability density function (p.d.f) of parameter H needs to be determined in order to maximize the expression Eq. (4). It is assumed that ni is IID with N (0, σn2) and H is independent of ni with N (0, σH2). Let K be numbers of pixels which are available for a given box to estimate H. The conditional p.d.f of y given H is given by Eq. (5):

$$f(\boldsymbol{y}|\boldsymbol{H}) = \prod\_{i=1}^{K} \frac{\mathbf{1}}{\sqrt{2\pi\sigma\_n^2}} e^{-\frac{(\boldsymbol{y}\_i - \boldsymbol{H})^2}{2\sigma\_n^2}} \tag{5}$$

And the p.d.f of H is given by Eq. (6):

$$f(H) = \frac{1}{\sqrt{2\pi\sigma\_H^2}} e^{-\frac{(H)^2}{2\sigma\_H^2}}\tag{6}$$

It is known that the conditional p.d.f of H given y is in Eq. (7):

$$f(H|\boldsymbol{y}) = \left\{ \frac{f(\boldsymbol{y}|H)f(\boldsymbol{H})}{f(\boldsymbol{y})} \right\} \tag{7}$$

Inserting the values from Eqs. (5) and (6) in Eq. (7) provides Eq. (8):

$$f(H|y) = \left(\prod\_{i=1}^{K} \frac{\mathbf{1}}{\sqrt{2\pi\sigma\_n^2}}\right) e^{-\sum\_{i=1}^{K} \left(y\_i - H\right)^2} e^{-\left(H^2/2\sigma\_H^2\right)} \frac{\mathbf{1}}{f(y)\sqrt{2\pi}\sigma\_H} \tag{8}$$

SAML-QC: A Stochastic Assessment and Machine Learning-Based QC Technique for Industrial… DOI: http://dx.doi.org/10.5772/intechopen.85680

The expression in Eq. (8) can be defined in terms of q(y) as shown in Eq. (9):

$$f(H|\mathcal{y}) = q(\mathcal{y})e^{-\left(\mathbf{1}/2\sigma^2\right)\left(H-\sigma^2/\sigma\_n^2 \sum\_{i=1}^{K} y\_i\right)^2} \tag{9}$$

The notation σ is given by Eq. (10):

and processed by the stated procedure to get the detected boxes of letters in the

Computer Architecture in Industrial, Biomechanical and Biomedical Engineering

The next block performs the stochastic assessment of the printing quality in terms of detection of overly faded printed text due to the imperfections in printing

The intensity value of each printed pixel can be considered as a random variable

H sampled at time t, and since its distribution is also unknown, therefore, it follows Gaussian distribution. From [18] it is clear that a sequence of random variables is independent and identically distributed (IID), if every random variable has the identical probability distribution and all are mutually independent. Thus it is also assumed that H is IID Gaussian random variable. The noise related to the intensity variation is Gaussian in nature; hence the mean and the acquired image also have additive Gaussian noise ni with H. In order to estimate the parameter H, from parametric space to the estimation space H<sup>0</sup> via observation space y(H,t) at any time t, the maximum a posteriori probability (MAP) estimation [19] is used. MAP maximizes the a posteriori probability, which means most likely the value of

text and marked with green rectangles.

3.3.1 Detection of printing in varying shades

max

p.d.f of y given H is given by Eq. (5):

And the p.d.f of H is given by Eq. (6):

f Hð Þ¼ <sup>j</sup> <sup>y</sup> <sup>Y</sup>

108

K

1 ffiffiffiffiffiffiffiffiffiffiffiffi 2πσ<sup>n</sup> <sup>2</sup> p !

i¼1

f g <sup>H</sup> f Hð Þ¼ <sup>j</sup> <sup>y</sup> max

f y ð Þ¼ <sup>j</sup><sup>H</sup> <sup>Y</sup>

f g H

It is quite clear that the probability density function (p.d.f) of parameter H needs to be determined in order to maximize the expression Eq. (4). It is assumed that ni is IID with N (0, σn2) and H is independent of ni with N (0, σH2). Let K be numbers of pixels which are available for a given box to estimate H. The conditional

K

1 ffiffiffiffiffiffiffiffiffiffi 2πσ<sup>2</sup> n <sup>p</sup> <sup>e</sup>

ffiffiffiffiffiffiffiffiffiffiffi 2πσ<sup>2</sup> H <sup>p</sup> <sup>e</sup>

f y ð Þ � �

f Hð Þ¼ <sup>j</sup> <sup>y</sup> f y ð Þ <sup>j</sup><sup>H</sup> f Hð Þ

Inserting the values from Eqs. (5) and (6) in Eq. (7) provides Eq. (8):

e � ∑ K i¼1 1 <sup>2</sup>σn<sup>2</sup> <sup>y</sup>ð Þ <sup>i</sup> �<sup>H</sup> <sup>2</sup> e

i¼1

f Hð Þ¼ <sup>1</sup>

It is known that the conditional p.d.f of H given y is in Eq. (7):

f y ð Þ jH f Hð Þ f y ð Þ � �

> � <sup>y</sup>ð Þ <sup>i</sup>�<sup>H</sup> <sup>2</sup> 2σ2

�ð Þ <sup>H</sup> <sup>2</sup> 2σ2

<sup>n</sup> (5)

<sup>H</sup> (6)

� <sup>H</sup>2=2σH<sup>2</sup> ð Þ <sup>1</sup>

f y ð Þ ffiffiffiffiffi <sup>2</sup><sup>π</sup> <sup>p</sup> <sup>σ</sup><sup>H</sup> (4)

(7)

(8)

process as discussed in [1].

H is given by Eq. (4):

3.3 Stochastic assessment of printing quality

$$
\sigma^2 = \frac{1}{\left(K\sigma\_H^2 + \sigma\_n^2\right)} \sigma\_H^{-2} \sigma\_n^{-2} \tag{10}
$$

It should be noted that q(y) is only the function of y; therefore, the best estimate HMAP of H is the value where f(H|y) gets the highest peak, which is obtained from Eq. (11) when H = HMAP:

$$H\_{\rm MAP} = \sigma^2 / \sigma\_n^{-2} \sum\_{i=1}^{K} \mathcal{y}\_i \tag{11}$$

Eq. (10) can be solved to get Eq. (12):

$$
\sigma^2 = \frac{\sigma\_H^2}{\left(\sigma\_H^2 + \sigma\_n^2/K\right)K} \frac{1}{K} \sum\_{i=1}^K y\_i \tag{12}
$$

If σ<sup>S</sup> <sup>2</sup> ≫ σ<sup>n</sup> <sup>2</sup>=K, then the best estimate of H is given as Eq. (13):

$$H\_{\rm MAP} \approx \frac{1}{K} \sum\_{i=1}^{K} \mathcal{y}\_i \tag{13}$$

HMAP is the best estimate of intensity levels of the character-containing region of a detected box, yet its value varies for every other detected box, and its probability distribution is also unknown, so it can also be assumed as a random variable that follows Gaussian distribution. If B represents the number of detected character boxes in a given image, then the set of real values hMAP(u) assigned to HMAP for all detected boxes is shown in Eq. (14), where u represents the index of each box.

$$H\_{\rm MAP} = \{h\_{\rm MAP}(\mathbf{1}), h\_{\rm MAP}(\mathbf{2}), h\_{\rm MAP}(\mathbf{3}), \dots, h\_{\rm MAP}(\mathbf{B})\} \tag{14}$$

Let E [HMAP] be the expectation of HMAP, and it is expressed as Eq. (15). The variance σ 2HMAP can be found in Eq. (16):

$$E\left[H\_{MAP}\right] = \sum\_{u=1}^{B} h\_{MAP}(u)P\{H\_{MAP} = h\_{MAP}(u)\}\tag{15}$$

$$\sigma^2\_{\text{HMAP}} = E\left[ \left( H\_{\text{MAP}} - E\left[ H\_{\text{MAP}} \right] \right)^2 \right] \tag{16}$$

Definition 1: (Set of bad boxes) The set of all those boxes such that members do not satisfy the conditions in Eq. (17) is called set of bad boxes.

Here n represents the quality index, and smaller value of n corresponds to higher demand of quality of label printing,

$$(E[H\_{\rm MAP}] - n\sigma\_{\rm HMAP}) \le h\_{\rm MAP}(u) \le (E[H\_{\rm MAP}] + n\sigma\_{\rm HMAP}) \tag{17}$$

The selection of bad box is performed by Eq. (17), as one of the main concerns in the quality inspection of label printing is to find the overly printed or faded printed characters. All the character boxes that fall under the stated condition Eq. (17) are considered as members of the set of good boxes. It is quite intuitive that highquality printing demands larger set of good boxes. Therefore, the proposed scheme used two separated counters called GBc and BBc representing the numbers of good boxes and bad boxes, respectively. The % quality success for similar intensity levels QSI is given by Eq. (18):

$$\text{QS}\_I = \frac{\text{GB}\_c}{(\text{BB}\_c + \text{GB}\_c)} \times 100 \tag{18}$$

The supervised machine learning procedure of the proposed scheme takes input sample image of industrial object which is printed with no defects. All the characters in this sample image are alphabets from A to Z, a to z, -, and numeric digits from 0 to 9. Let us call the set of these input images as no-misprinted-images (NMI). The training process starts by iterating the NMI for all detected boxes, and for each detected box, a human response is provided as a label and aggregated to a human responses database (HRD) file, because the human needs to press the related key from the keyboard. As for each detected letter box, its ratio width/height = 2/3; therefore each box is aggregated as a matrix of 20 � 30 pixels in separate matrices

SAML-QC: A Stochastic Assessment and Machine Learning-Based QC Technique for Industrial…

Therefore, at the end of training process, two (HRD and MRD) files are ready for the k-NN algorithm. The k-NN algorithm takes these two files as input and finds the nearest neighbor in terms of its output as hamming distance D as stated above. It is quite intuitive that D will have higher value for the misprinted or unknown letters, because the training process did not consider those letters. The value of D for each printed letter can be considered as a random variable sampled from a random process at time t, and since its distribution is also unknown, therefore, it is assumed that it follows Gaussian distribution. Based on the fact stated by [22], the proposed scheme also assumes that D is IID Gaussian random variable. The next step is finding the expectation of D for each letter box. It should be noted that D is normalized by

of D. We can compute these values using the similar way as described in Eqs. (15) and (16), which provide the second order statistics for the distance. Let the set of

Definition 2: (Misprinted box detection) The set of all those boxes such that members do not satisfy the conditions Eq. (20) is called set of misprinted bad boxes.

Again the quality index m represents the strict boundary for the selection of misprinted boxes. All the character boxes that fall under the stated condition Eq. (20) are considered as members of the set of good printed boxes. It is quite intuitive that high-quality printing demands larger set of good boxes. Therefore, two separated counters called GPBcount and MPBcount represent the numbers of good printed boxes and misprinted boxes, respectively. The % quality success for

ð Þ MPBcount þ GPBcount

The final step is the printing of the acquired results related to the text alignment, QSI and QSGPB. These results can not only draw the outcome of printing inspection process but can also provide feedback to the printing labelers to automatically

Table 1 summarizes the values of horizontal and vertical alignments for sample

image in Figure 5. The results support the human observation of central text

D ¼ f g dð Þ1 ; dð Þ2 ; dð Þ3 ; :……; dð Þ B (19)

ð Þ E D½ �� mσ<sup>D</sup> ≤d uð Þ≤ ð Þ E D½ �þ mσ<sup>D</sup> (20)

real values d(u) assigned to the random variable D be given by Eq. (19):

QSGPB <sup>¼</sup> GPBcount

adjust the process according to the type of printing errors.

<sup>2</sup> as the mean and variance

� 100 (21)

received database (MRD) file.

DOI: http://dx.doi.org/10.5772/intechopen.85680

the size of box, i. e., 20 � 30. Let us represent E[D] and σ<sup>D</sup>

good printed boxes QSGPB is given by Eq. (21):

4. Results

111

It should also be noted that the value of QSI has a direct relationship with quality index n. The smaller value of quality index means strict requirement of quality and for the same image QSI decreases with the decrease in value of n and vice versa.

In order to plot the probability density estimate of HMAP, the procedure for kernel density estimation [20] is used. It returns a probability density estimate f for the sampled data in the vector or two-column matrix. It estimates the density at 100 points for univariate data. Figure 4 shows the plot of probability density estimate for HMAP of the image in Figure 3(a). It is clear that p.d.f approximately follows Gaussian curve with a certain value of mean and variances.

#### 3.3.2 Detection of misprinted boxes

The detection of misprinted box is challenging in terms of its probability of occurrences and detection. The OCR engine can detect it as a text although being probably wrong detected and classified. The proposed scheme achieves the detection of misprinted box by deploying supervised machine learning using k-NN. The k-NN algorithm is a nonparametric method used for the classification and regression [21] in the pattern recognition. The supervised training aggregates the human responses to the appearance of a letter inside the detected box. The purpose of this step is to provide supervised learning samples and responses for the k-NN algorithm.

Figure 4. Kernel density estimate plot of HMAP for the image in Figure 3(a).

SAML-QC: A Stochastic Assessment and Machine Learning-Based QC Technique for Industrial… DOI: http://dx.doi.org/10.5772/intechopen.85680

The supervised machine learning procedure of the proposed scheme takes input sample image of industrial object which is printed with no defects. All the characters in this sample image are alphabets from A to Z, a to z, -, and numeric digits from 0 to 9. Let us call the set of these input images as no-misprinted-images (NMI). The training process starts by iterating the NMI for all detected boxes, and for each detected box, a human response is provided as a label and aggregated to a human responses database (HRD) file, because the human needs to press the related key from the keyboard. As for each detected letter box, its ratio width/height = 2/3; therefore each box is aggregated as a matrix of 20 � 30 pixels in separate matrices received database (MRD) file.

Therefore, at the end of training process, two (HRD and MRD) files are ready for the k-NN algorithm. The k-NN algorithm takes these two files as input and finds the nearest neighbor in terms of its output as hamming distance D as stated above. It is quite intuitive that D will have higher value for the misprinted or unknown letters, because the training process did not consider those letters. The value of D for each printed letter can be considered as a random variable sampled from a random process at time t, and since its distribution is also unknown, therefore, it is assumed that it follows Gaussian distribution. Based on the fact stated by [22], the proposed scheme also assumes that D is IID Gaussian random variable. The next step is finding the expectation of D for each letter box. It should be noted that D is normalized by the size of box, i. e., 20 � 30. Let us represent E[D] and σ<sup>D</sup> <sup>2</sup> as the mean and variance of D. We can compute these values using the similar way as described in Eqs. (15) and (16), which provide the second order statistics for the distance. Let the set of real values d(u) assigned to the random variable D be given by Eq. (19):

$$D = \{d(\mathbf{1}), d(\mathbf{2}), d(\mathbf{3}), \dots, d(\mathbf{B})\} \tag{19}$$

Definition 2: (Misprinted box detection) The set of all those boxes such that members do not satisfy the conditions Eq. (20) is called set of misprinted bad boxes.

$$d(E[D] - m\sigma\_D) \le d(u) \le (E[D] + m\sigma\_D) \tag{20}$$

Again the quality index m represents the strict boundary for the selection of misprinted boxes. All the character boxes that fall under the stated condition Eq. (20) are considered as members of the set of good printed boxes. It is quite intuitive that high-quality printing demands larger set of good boxes. Therefore, two separated counters called GPBcount and MPBcount represent the numbers of good printed boxes and misprinted boxes, respectively. The % quality success for good printed boxes QSGPB is given by Eq. (21):

$$\text{QS}\_{GPB} = \frac{GPB\_{\text{count}}}{(MPB\_{\text{count}} + GPB\_{\text{count}})} \times 100 \tag{21}$$

The final step is the printing of the acquired results related to the text alignment, QSI and QSGPB. These results can not only draw the outcome of printing inspection process but can also provide feedback to the printing labelers to automatically adjust the process according to the type of printing errors.

#### 4. Results

Table 1 summarizes the values of horizontal and vertical alignments for sample image in Figure 5. The results support the human observation of central text

characters. All the character boxes that fall under the stated condition Eq. (17) are considered as members of the set of good boxes. It is quite intuitive that highquality printing demands larger set of good boxes. Therefore, the proposed scheme used two separated counters called GBc and BBc representing the numbers of good boxes and bad boxes, respectively. The % quality success for similar intensity levels

ð Þ BBc þ GBc

The detection of misprinted box is challenging in terms of its probability of occurrences and detection. The OCR engine can detect it as a text although being probably wrong detected and classified. The proposed scheme achieves the detection of misprinted box by deploying supervised machine learning using k-NN. The k-NN algorithm is a nonparametric method used for the classification and regression [21] in the pattern recognition. The supervised training aggregates the human responses to the appearance of a letter inside the detected box. The purpose of this step is to provide supervised learning samples and responses for

It should also be noted that the value of QSI has a direct relationship with quality index n. The smaller value of quality index means strict requirement of quality and for the same image QSI decreases with the decrease in value of n and vice versa. In order to plot the probability density estimate of HMAP, the procedure for kernel density estimation [20] is used. It returns a probability density estimate f for the sampled data in the vector or two-column matrix. It estimates the density at 100 points for univariate data. Figure 4 shows the plot of probability density estimate for HMAP of the image in Figure 3(a). It is clear that p.d.f approximately

� 100 (18)

QSI <sup>¼</sup> GBc

Computer Architecture in Industrial, Biomechanical and Biomedical Engineering

follows Gaussian curve with a certain value of mean and variances.

QSI is given by Eq. (18):

3.3.2 Detection of misprinted boxes

the k-NN algorithm.

Figure 4.

110

Kernel density estimate plot of HMAP for the image in Figure 3(a).

alignment for this kind of particular object. It should also be noted that text alignment specification and parameters are user specific.

Figure 6 shows the bar graph of the estimated values for the random variable HMAP for each detected text in Figure 3(b).

Table 2 summarizes the results of printing quality assessment in order to test the intensity variations in the printed labels. It shows mean, variance, quality (Q) factor nσHMAP, the sum of good and bad boxes, and finally the QSI. It is obvious from the

SAML-QC: A Stochastic Assessment and Machine Learning-Based QC Technique for Industrial…

The value of quality index is varied as n = 1, 2, and 3 that increases the value of GBc as 13, 22, and 23, respectively. Additionally, the values for QSI for n = 1, 2, and

Figure 8 shows the output of SAML-QC scheme for the detection of bad printed

Figure 8(a) shows that for the restriction n = 1, all good boxes are detected and marked with green color, whereas, all the boxes in red color are the ones with value of hMAP(u) ≥ E[HMAP]+nσ2HMAP, whereas, boxes with hMAP(u) ≤ E[HMAP] nσ2HMAP are represented by yellow boxes because they belong to the faded printing category. Similarly, Figure 8(b) and (c) shows the output images for n = 2 and 3,

The next assessment is related to the detection of misprinted characters, and

It is clear from the results in Figure 9 that the two higher spikes for the detected

Figure 9 shows result of k-NN classification performed through supervised

machine learning and the distance D for each detected character.

character S and 3 are located at the position of misprinted characters.

results that Q factor decides the value of QSI.

DOI: http://dx.doi.org/10.5772/intechopen.85680

3 are 56.52, 95.65, and 100%, respectively.

respectively.

Figure 7.

Table 2.

113

Q Index n Mean E

[HMAP]

Printing quality assessment results for HMAP.

Plot shows fitting a normal distribution to the values of HMAP.

Variance σ 2HMAP

Q factor nσHMAP

1 66 6 2.44 13 + 10 56.52 2 66 6 4.89 22 + 1 95.65 3 66 6 7.35 23 + 0 100

Box counts GBc + BBc

QSI (%)

labels. The higher quality control requires lower value of n.

Figure 7 shows the histogram of HMAP in terms of numbers of boxes and the value of random variable HMAP. It also shows the corresponding normal distribution curve fitted based on the values of HMAP.


Table 1.

Text alignment assessment results for the given image.

Figure 5. Text alignment detection.

Figure 6. Estimated intensity values of HMAP.

SAML-QC: A Stochastic Assessment and Machine Learning-Based QC Technique for Industrial… DOI: http://dx.doi.org/10.5772/intechopen.85680

Table 2 summarizes the results of printing quality assessment in order to test the intensity variations in the printed labels. It shows mean, variance, quality (Q) factor nσHMAP, the sum of good and bad boxes, and finally the QSI. It is obvious from the results that Q factor decides the value of QSI.

The value of quality index is varied as n = 1, 2, and 3 that increases the value of GBc as 13, 22, and 23, respectively. Additionally, the values for QSI for n = 1, 2, and 3 are 56.52, 95.65, and 100%, respectively.

Figure 8 shows the output of SAML-QC scheme for the detection of bad printed labels. The higher quality control requires lower value of n.

Figure 8(a) shows that for the restriction n = 1, all good boxes are detected and marked with green color, whereas, all the boxes in red color are the ones with value of hMAP(u) ≥ E[HMAP]+nσ2HMAP, whereas, boxes with hMAP(u) ≤ E[HMAP] nσ2HMAP are represented by yellow boxes because they belong to the faded printing category. Similarly, Figure 8(b) and (c) shows the output images for n = 2 and 3, respectively.

The next assessment is related to the detection of misprinted characters, and Figure 9 shows result of k-NN classification performed through supervised machine learning and the distance D for each detected character.

It is clear from the results in Figure 9 that the two higher spikes for the detected character S and 3 are located at the position of misprinted characters.

Figure 7. Plot shows fitting a normal distribution to the values of HMAP.


Table 2. Printing quality assessment results for HMAP.

alignment for this kind of particular object. It should also be noted that text

Computer Architecture in Industrial, Biomechanical and Biomedical Engineering

Figure 6 shows the bar graph of the estimated values for the random variable

Figure 7 shows the histogram of HMAP in terms of numbers of boxes and the value of random variable HMAP. It also shows the corresponding normal distribution

Images U L D UDThresh LThresh Alignment testing

1 42 59 50 170 119 Passed Passed

Horizontal Vertical

alignment specification and parameters are user specific.

HMAP for each detected text in Figure 3(b).

curve fitted based on the values of HMAP.

Text alignment assessment results for the given image.

Table 1.

Figure 5.

Figure 6.

112

Estimated intensity values of HMAP.

Text alignment detection.

#### Figure 8.

Detection of bad printed boxes for different n: (a) n = 1, (b) n = 2, and (c) n = 3.

5. Conclusions

Figure 10.

tronics parts and medical parts.

Acknowledgements

kind support, and wisdom.

\* and Kamal Bhattarai<sup>2</sup>

Detection of misprinted boxes: (a) m = 1, (b) m = 2, and (c) m = 3.

DOI: http://dx.doi.org/10.5772/intechopen.85680

\*Address all correspondence to: engrazr@gmail.com

provided the original work is properly cited.

2 University of Engineering and Technology, Taxila, Pakistan

1 Harbin Engineering University, Harbin, China

Author details

Azhar Hussain<sup>1</sup>

115

This chapter proposes a stochastic assessment and machine vision-based technique to assess the printing quality of text on industrial objects. The assessment is based on three quality defects such as text misalignment, varying printing shades, and misprinted text. It is concluded from the results that second-order statistics related to the intensity values of pixels of text located under the threshold region of the original image can provide enough information to perform quality inspection. On the other hand, the hamming distance acquired by the k-NN supervised machine learning can also be taken as a random variable, and again the secondorder statistics are helpful in order to detect the misprinted letter. In the future SAML-QC technique needs to be evaluated on various other images such as elec-

SAML-QC: A Stochastic Assessment and Machine Learning-Based QC Technique for Industrial…

My deepest acknowledgment is to Prof. Tao JIANG of the College of Information and Communication Engineering, Harbin Engineering University, for guidance,

© 2019 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/ by/3.0), which permits unrestricted use, distribution, and reproduction in any medium,

Figure 9. Bar graph of k-NN classification distance D.


#### Table 3.

Misprinted labels' assessment results.

Table 3 shows results of the process for the detection of misprinted boxes as discussed in Section 3. It is observed that the proposed scheme put a stable restriction on the misprinted labels, and it is shown in Table 3 that the value of QSGBP remains 91.30% for all three values of m.

Figure 10 shows the detection of misprinted labels, and they are marked with the red boxes for m = 1, 2, and 3.

SAML-QC: A Stochastic Assessment and Machine Learning-Based QC Technique for Industrial… DOI: http://dx.doi.org/10.5772/intechopen.85680

Figure 10. Detection of misprinted boxes: (a) m = 1, (b) m = 2, and (c) m = 3.
