**3. Proposed method**

*Artificial Intelligence - Latest Advances, New Paradigms and Novel Applications*

each map but keeps the important information.

fully connected layer like neural network.

ƒ(x) = max(0,x).

**Step-2: Classification**

**2.2 Association rule**

b.Activation layer: this layer is used to increase non-linearity of the network without affecting receptive fields of convolution layer. The output is

c.Pooling layer: this layer reduces the number of parameters when the images are too large. This operation means that the layer reduces the dimensionality of

This process executes image classification. There are two main layers as follows:

a.Fully connected layer (FC layer): to connect every neuron in one layer to every neuron in another layer. We flattened our matrix into vector and feed it into a

b.Softmax layer: A special kind of activation layer, usually at the end of FC layer

At this stage, the pretrained models such as *AlexNet, VGGNet, GoogLeNet*, and *ResNet* will be applied in order to integrate the CNNs. The procedure is to merge the multiple CNNs with fixed number of CNNs and trained them after merging.

Association rule is utilized in data mining phase as it is a machine learning method which finds the relationship between the items based on the frequency of item sets [18, 19]. To do this, a mathematical algorithm is needed to arrange the

There are three measures in association rule task as shown in **Figure 2**:

c.Lift: It indicates how the *antecedent* and the *consequent* are related to one another. If the Lift value is 1, it means that the *antecedent* and the *consequent* are independent. If the Lift value is less than 1, it means that the *antecedent* has negative effect on occurrence on the *consequent*. In addition, if the Lift value is more than 1, it means that the *antecedent* and the *consequent* are dependent.

a.Support: It indicates how often the items appear in the data-set.

b.Confidence: It indicates how often a rule is found to be true.

outputs. It produces a discrete probability distribution vector.

frequent item sets, for extracting association rule of food sets.

**76**

**Figure 2.**

*Basic terminologies of association rule measure.*

As the uncertainty of convolutional architecture due to the fix number of CNNs and the interpretability after merging of CNNs [20]. Simpler and more highly scalable method to integrate the multiple networks is necessary and therefore there are three features in consideration - liability score for the system, no further training after integration, and high scalability. Deep neural network (DNN) may be an alternative choice to solve this problem [21]. It provides the neural basis for efficient visual object recognition in humans. Its architecture comprises more than three layers. Each of layers contains a combination of convolution, max-pooling and normalization stages, whereas these layers are fully connected. It inherently fuses the process of feature extraction with classification into learning using Fuzzy Support Vector Machine (FSVM) and enables the decision making [22, 23]. Many efforts were made to speed up both the training time as well as inference time of DNNs. Since the flexibility and the performance scalability to deal with various types of networks are crucial requirement of the DNN accelerator design [24], we therefore proposed a scalable architecture for integrating different deep neural networks with a reliability score to increase the probability to return correct class as the result of food recognition. The reliability score allowed the integrated DNN to select a suitable recognition result obtained from the different DNNs that were independently constructed. In this study, we evaluated the feasibility of our proposed method.

The frequent set of foods extracted from food images was applied to a data mining algorithm such as the *Apriori* algorithm for the food recommendation process.

**Figure 3** shows the schematic diagram of food recommender. This system would recommend a set of foods suitable for users based on association rule, which were extracted by food set mining from food images.

The process of food recognition and extraction of association rule is given in **Figure 4**. Firstly, the food set images from database were collected. Secondly, the reliability score was calculated by Eq. (1). Then the food recognition and

**Figure 3.** *Schematic diagram of food recommender.*

**Figure 4.**

*Process of recognition of food set images and extraction of association rule.*

segmentation from the images were operated using DNNs. Finally, the extraction of association rule was obtained as the frequent food sets.

$$\text{Reliability score} = \mathbf{A} \times \mathbf{C} \times \mathbf{I} \times \mathbf{D} \tag{1}$$

where A, C, I, and D represented the accuracy of the learning process, number of classes, number of leaning images, and difference between learning data and test data, respectively.

We combined several DNNs and calculated new probability with reliability scores as shown in **Figure 5**. Each DNN has different reliability score depending on following factors:


**79**

necessary.

**Figure 6.**

predicted result was provided.

*Workflow of proposed food recommender.*

the phase of data mining.

DNNs in this experiment.

False Negative respectively.

**4. Experimental and evaluation**

*A Food Recommender Based on Frequent Sets of Food Mining Using Image Recognition*

If the learning data and the test data are similar, the accuracy will be high but it may be not reliable. As we test with the unseen data, the DNN may predict wrong answer. Therefore, the difference between the learning data and the test data is

We calculated the reliability score and the weight output (W) of fully connected layer as the new probability using Eq. (2). The label of the new probability as the

Accordingly, the food set could be recognized from food images using different DNNs with the reliability scores as shown in **Figure 6**. Based on the recognition results, transactions of food were formed and applied to conventional association in

We implemented our prototype of merging the results from different DNNs using MATLAB. The performance of recognition accuracy and the relationship between the recognition accuracy and the number of classes were evaluated. We used the dataset of food images for creating the DNNs and evaluated them as shown in **Table 1**. Three DNNs with different number of recognition classes for each dataset were created as given in **Table 2**. In addition, we made each DNN from *Scratch* by applying the transfer learning using *GoogLeNet*. Accordingly, we totally had 18

Recognition accuracy was calculated using the following equation:

New probability W Reliability score = ´ (2)

Accuracy TP TN / TP TN FP FN = + + ++ ( ) ( ) (3)

where TP, TN, FP, and FN were True Positive, True Negative, False Positive, and

*DOI: http://dx.doi.org/10.5772/intechopen.97186*

```
Figure 5.
The feature of proposed method.
```
*A Food Recommender Based on Frequent Sets of Food Mining Using Image Recognition DOI: http://dx.doi.org/10.5772/intechopen.97186*

#### **Figure 6.** *Workflow of proposed food recommender.*

*Artificial Intelligence - Latest Advances, New Paradigms and Novel Applications*

segmentation from the images were operated using DNNs. Finally, the extraction of

where A, C, I, and D represented the accuracy of the learning process, number of classes, number of leaning images, and difference between learning data and test

We combined several DNNs and calculated new probability with reliability scores as shown in **Figure 5**. Each DNN has different reliability score depending on

Reliability score A C I D = ´ ´´ (1)

association rule was obtained as the frequent food sets.

*Process of recognition of food set images and extraction of association rule.*

data, respectively.

**Figure 4.**

following factors:

• number of classes

• accuracy of the learning process

• difference between learning data and test data

• number of learning images

**78**

**Figure 5.**

*The feature of proposed method.*

If the learning data and the test data are similar, the accuracy will be high but it may be not reliable. As we test with the unseen data, the DNN may predict wrong answer. Therefore, the difference between the learning data and the test data is necessary.

We calculated the reliability score and the weight output (W) of fully connected layer as the new probability using Eq. (2). The label of the new probability as the predicted result was provided.

$$\text{New probability} = \mathbf{W} \times \text{Reliability score} \tag{2}$$

Accordingly, the food set could be recognized from food images using different DNNs with the reliability scores as shown in **Figure 6**. Based on the recognition results, transactions of food were formed and applied to conventional association in the phase of data mining.
