**4. Ensemble strategies**

• Using ensemble methods as a preprocessing step before performing the essential environ-

• Clustering environmental documents according to their topics and main contents.

• Usage of process mining to improve work management in the environmental science.

Ensemble learning is a machine learning technique where multiple learners are trained to solve the same problem and their predictions are combined with a single output that probably has better performance on average than any individual ensemble member. The fundamental idea behind ensemble learning is to combine weak learners into one, a strong learner, who has a better generalization error and is less sensitive to overfitting in the presence of noise or small sample size. This is because different classifiers can sometimes misclassify different patterns and accuracy can be improved by combining the decisions of complementary classifiers.

A typical ensemble framework for classification tasks contains four fundamental components

• *Training set*: a training set is a special set of labeled examples providing known information

• *Base inducer*(s) or *base classifier*(s): an inducer is a learning algorithm that is used to learn from a training set. A base inducer obtains a training set and constructs a classifier that

• *Diversity generator*: it is clear that nothing is gained from an ensemble model if all ensemble members are identical. The diversity generator is responsible for generating the diverse classifiers and decides the type of every base classifier that differs from each other. Diversity can be realized in different ways depending on the accuracy of individual classifiers for the improved classification performance. Common diversity creation approaches are (i) using different training sets, (ii) combining different inducers, and (iii) using different

• *Combiner*: the task of the combiner is to produce the final decision by combining all classification results of the various base inducers. There are two main methods of combining: weighting methods and meta-learning methods. *Weighting methods* give each classifier a weight proportional to its strength and combine their votes based on these weights. The weights can be fixed or dynamically determined when classifying an instance. Common weighting methods are majority voting, performance weighting, Bayesian combination, and vogging. *Meta-learning methods* learn from new training data created from the predictions of a set of base classifiers. The most well-known meta-learning methods are stacking

generalizes relationship between the input features and the target outcome.

mental study.

6 Data Mining

**3. Background information**

**3.2. Elements of an ensemble classifier**

**3.1. Ensemble learning**

descripted as follows:

that are used for training.

parameters for a single inducer.

In order to construct an ensemble model, any of the following strategies can be performed:

### **4.1. Strategy 1: different training sets using random sampling with replacement**

One ensemble strategy is to train different base learners by different subsets of the training set. This can be done by random resampling of a dataset (i.e., *bagging*; **Figure 2a**). When we train multiple base learners with different training sets, it is possible to reduce variance and therefore error.

### **4.2. Strategy 2: different training sets obtained by random instance and feature subset selection**

The combination of bagged decision trees is constructed similar to Strategy 1 using one significant adjustment that random feature subsets are used (i.e., *random forest*; **Figure 2b**). When we have enough trees in the forest, random forest classifier is less likely overfit the model. It is also useful to reduce the variance of low-bias models, besides handling missing values easily.
