**3. Ensemble methods and design analysis**

The ensemble system of machine learning application is a type of method that allows the combination of multiple models to produce an output that is improved and enhanced than applying the single model. The ensemble predictive model generated is to have a decrease in variance (bagging), bias (boosting), or to improve the predictions (stacking). The ensemble machine learning methods consist of sequential, parallel, homogenous and heterogenous ensemble, while the technical classifications of ensemble method are, bagging, boosting, stacking, and random forest [38].

### **3.1 Ensemble methods**

In the continuous method, the base learners (where the data dependency stays) are being initiated successively. Moreover, there is a dependency on the former data of all other data in the base level and to get the performance analysis of the system, the wrongfully-labeled are adjusted based on the weight. An example of this type of analysis is the boosting method. The parallel methodology ensures that the base learner is initiated in a parallel manner and the data dependency is not available, and all the data are generated separately. A very good example of this model is the stacking method [38].

The homogenous ensemble method can be applied to a large number of the dataset as it employed the combination of the very same types of classifiers. The dataset is always different for each classifier and the model works well following the results collection for each of the classifier models. The feature selection method is the same for different training of data. The major disadvantage of this type of model is that the computational cost is very expensive. The popular type of this model is the bagging and boosting method. Conversely, the heterogeneous ensemble method combines different classifiers and each of them is generated on the same data, these types of methods are used for small datasets and the feature selection method is different for the same training dataset. An example of this type of classifier is stacking [38].

### **3.2 Technical classification of ensemble approach**

### *3.2.1 Bagging*

The bagging method ensemble learning represents the bootstrapping and aggregation of combining two models into one ensemble model. Bagging uses random sampling to reduce the variance of the model by creating more data at the training stage of the model and every element in bagging has a way to appear in the new dataset. This method decreases the variance and narrowly changes the prediction to an expected output [38].

### *3.2.2 Boosting*

The boosting method of ensemble classification makes use of a continuous method of classifying based on the features that the next model will utilize. The boosting methods make a stronger learner out of a weak learner model using weight average. The much stronger trained model relies on the multiple weak trained models. The weak learner is the one with the less correlated true classification and the next one is improved over the last one which will be a bit more correlated, the aggregation of the weak learners makes up the strong learner that is well correlated with accurate classification [38].

### *3.2.3 Stacking*

In the stacking method, a different classifier is also used using the regression techniques or multiple combinations of the models. The operation involves training the lower-level with the entire dataset in which the outcomes are used to train the combined model. This is different from boosting in the sense that the lower level is in line with parallel trained. The prediction from the outcome of the lower level forms the next model as a stack. The stack operation is such that the top of the stack is the most trained and has goo prediction while the down of the stack is the least trained. The predictions continue until the best surfaced with fewer errors [38].

### *3.2.4 Random forest*

The random forest models employ tree operations to carry out the sampling of the dataset. The tree is fitted on bootstrap samples and the output is aggregated together to reduce variance. It uses the features to sample the dataset in a random subset for the creation of the tree to reduce the correlation of the outputs. This model of random forest is best for determining the missing data as it selects a random subset of the sample to reduce the possibility of having common prediction values, with each tree having a different structure. The resultant variance result from the average of the lesser prediction from different trees gives better output [38].

### **3.3 Ensemble framework**

In machine learning, ensemble methods and/or models employ the blend of different learning algorithms to give a better predictive performance that displays an outcome that surpasses what could be gathered from the single application of any of the learning models. Experimentally, ensembles models produce a more improved

*Ensemble Machine Learning Algorithms for Prediction and Classification of Medical Images DOI: http://dx.doi.org/10.5772/intechopen.100602*

output as there is major distinctiveness among the model. The ensemble framework is depicted in **Figures 2** and **3** below.
