**4. Leveraging ML techniques for hedging against uncertainty in data-driven optimization**

In the big data and ML era, a large amount of interactive data are routinely generated and collected in different industries. Intelligence and data-driven analysis and decision-making have a critical role in process operations, design, and control. The success of the DSS depends primarily on the ability to process and analyze large amounts of data and extract relevant and useful knowledge and information from them. In this context, the data-driven approach has gained prominence due to its provision of insights for decision-making and easy implementation. The data-driven optimization framework is a hybrid system that integrates AI and optimization methods for devising a data-driven and intelligent DSS. The data-driven system applied ML techniques for uncertainty modeling. The data-driven approach can discover various database patterns without relying on prior knowledge while also can handle multiple scenarios and flexible objectives. It can also extract information and knowledge from data without speed [29, 30].

The framework of data-driven optimization under uncertainty could be considered a hybrid system that integrates the data-driven system based on ML to extract useful and relevant information from data. The model-based system is based on mathematical programming to derive the optimal decisions from the information [28]. The inability of traditional optimization methods to analyze big data, as well as recent advances in ML techniques, made data-driven optimization a promising way to hedge against uncertainty in the era of big data and ML. Therefore, these promises create the need for organic integration and effective interaction between ML and mathematical programming. In existing data-driven optimization frameworks, data serve as input to a data-driven system. After that, useful, accurate, and relevant uncertainty information is extracted through the data-driven system and further passed along to the model-based system based on mathematical programming for rigorous and systematic optimization under uncertainty, using paradigms such as robust optimization and stochastic programming.

The various ML techniques and their potentials applications in data-driven optimization under uncertainty are presented in the following.

#### **4.1 Distributionally robust optimization**

The stochastic programs are used where the distribution of the uncertain parameters is only observable through a finite training dataset [31]. As the primary assumption in the stochastic programming approach, the probability distribution of uncertain parameters should be clear. However, such complete knowledge of parameters probability distribution is rarely available in practice. In practice, instead of knowing the actual distribution of an uncertainty parameter, what the decisionmaker has is a set of historical/ or real-time uncertainty data and possibly some prior structure knowledge of the probability. Also, the assumed possibility distribution of uncertain parameters may deviate from their actual distribution. Moreover, relying on a single probability distribution could lead to sub-optimal solutions or even lead to the deterioration in out-of-sample performance [32]. Motivated by these stochastic programming weaknesses, DRO emerges as a new data-driven optimization paradigm that hedges against the worst-case distribution in an ambiguity set [28]. DRO paradigm integrates data-driven systems and model-based systems. A data-driven approach is applied in the DRO model to construct an uncertainty set of probability

#### *Artificial Intelligence and Its Application in Optimization under Uncertainty DOI: http://dx.doi.org/10.5772/intechopen.98628*

distributions from uncertainty data through statistical inference and big data analytics [28]. In data-driven stochastic modeling, the uncertainty is modeled via a family of probability distributions that well capture uncertainty data on hand [28]. This set of probability distributions is referred to as an ambiguity set. With this ambiguity set, a model is then proposed for problem design. Finally, a solution strategy is applied for solving the optimization problem. For example in the literature, the Wasserstein metric has been used, to construct a ball in the space of (multivariate and non-discrete) probability distributions centered at the uniform distribution on the training samples, to seek decisions that perform best in view of the worst-case distribution within this Wasserstein ball [31]. Different practical approaches, such as the moment-based, and the adopted distance metric, were employed for uncertainty constructing [33, 34], and [31]. DRO is an effective method to address the inexactness of probability distributions of uncertain parameters in decision-making under uncertainty that can be applied for optimizing supply chain activities, for planning and scheduling under uncertainty. This way reduces the modeling difficulty for uncertain parameters. Wang & Chen [35] proposed a two-stage DRO model considering scarce data of disasters. A moment-based fuzzy set describes uncertain distributions of blood demand to optimize blood inventory prepositioning and relief activities together. Chiou [36], to regulate the risk associated with hazardous material transportation and minimize total travel cost on the interested area under stochasticity, presented a multi-objective data-driven stochastic optimization model to determine generalized travel cost for hazmat carriers. Gao et al. [37] proposed a two-stage DRO model for better decision making in optimal design and shale gas supply chains under uncertainty. They applied a data-driven approach to construct the ambiguity set based on principal component analysis and first-order deviation functions. In the other study, Ning & You [28] proposed a novel data-driven Wasserstein DRO model for biomass with agricultural waste-to-energy network design under uncertainty. They proposed a data-driven approach to construct the Wasserstein ambiguity set for the feedstock price uncertainty, which is utilized to quantify their distances from the data-based empirical distribution.

## **4.2 Data-driven robust optimization**

A robust optimization is a popular approach for optimization under uncertainty. It defines an uncertainty set of possible realizations of the uncertain parameters and then optimizes against worst-case realizations within this set [5, 6]. In real-world applications, the underlying distribution of uncertainties may be intrinsically complicated and vary under different circumstances [38]. Choosing the accurate underlying distribution of uncertainties and the uncertainty sets by prior knowledge is somewhat challenging in practice. In robust optimization, the uncertainty is formed as an uncertainty set in which any point is a possible scenario [39]. Since the uncertainty set includes the worst case, robust optimization may be over-conservative. It is essential to apply the appropriate approach to construct the uncertainty set and adjust the conservatism level simultaneously [39]. As an essential ingredient in robust optimization, uncertainty sets endogenously determine robust optimal solutions and, therefore, should be devised with special care [28]. However, uncertainty sets in the conventional robust optimization methodology are typically set a priori using a fixed shape and model without providing sufficient flexibility to capture the structure and complexity of uncertainty data [28]. For instance, the geometric shapes of uncertainty set in the conventional robust optimization methodology do not change with

the intrinsic structure and complexity of uncertainty data. Furthermore, these uncertainty sets are specified by a finite number of parameters, thereby limiting modeling flexibility. Motivated by this knowledge gap, data-driven robust optimization emerges as a powerful paradigm for addressing uncertainty in decision making.

Choosing a good uncertainty set enables robust optimization models to provide better solutions than other approaches solutions [5, 6]. Poor choice of the uncertainty set makes robust optimization model overly conservative or computationally intractable. In the era of big data, many data are routinely generated and collected containing abundant information about the distribution of uncertainties; thereby, ML tools can construct the uncertainty sets based upon these data. Data-driven robust optimization is a new paradigm for hedging against uncertainty in the era of big data. The ML tools can be applied to estimate data densities with sufficient accuracy and construct an appropriate uncertainty set based upon intelligent analysis and the use of uncertainty data for modeling robust optimization problems. A desirable uncertainty set shall have enough flexibility to adapt to the intrinsic structure behind data, thereby characterizing the underlying distribution and facilitating the solutions.

Data-driven robust optimization could be considered a "hybrid" system that integrates the data-driven system based on ML to construct the uncertainty set from historical uncertainty data. The model-based system is based on the robust programming model to derive the optimal decisions from the information. More specifically, data serves as input to a data-driven system. **Figure 1** presents the data-driven optimization paradigm framework. After that, the data-driven method constructs the uncertainty set to extract information from historical data fully. Constructing the uncertainty sets based upon historical data can be considered as an unsupervised learning problem from an ML perspective. So, data-driven robust optimization is a hybrid system that utilizes ML techniques to design data-driven uncertainty sets and develops a robust optimization problem from the data-driven set. Different effective unsupervised learning models such as the Dirichlet process mixture model, maximum likelihood estimation, principal component analysis, regular and conservative support vector clustering, Bayesian ML, and kernel density estimation were employed for uncertainty constructing, which could provide powerful representations of data distributions [38, 40, 41]. Uncertainty set is the set that can offer robust solutions with a conservatism level. Furthermore, this uncertainty set is finally given to the model-based system based on robust optimization to obtain robust solutions under uncertainty.

ML methods of support vector clustering-based uncertainty set (SVCU) and conservative support vector clustering-based uncertainty set (CSVCU) have been applied to finding an enclosed hypersphere with minimum volume which is able to cover all data samples as tightly as possible as uncertainty sets. Conservative support vector clustering is the most suitable choice for obtaining robust solutions in cases with sufficient data to construct an uncertainty set enclosing future data with a high confidence level [42]. Furthermore, it is the most effective choice for obtaining lower

**Figure 1.**

*The schematic of the data-driven optimization paradigm framework.*

#### *Artificial Intelligence and Its Application in Optimization under Uncertainty DOI: http://dx.doi.org/10.5772/intechopen.98628*

conservative solutions. On the other hand, CSVCU is suitable for highly conservative decision-makers since it is the only set that can offer robust solutions with a high conservatism level, particularly when there is limited data [42]. A data-driven robust optimization under correlated uncertainty was proposed to hedge against the fluctuations generated from continuous production processes in an ethylene plant [43]. For capturing and enrich the valid information of uncertainties, a copula-based method is introduced to estimate the joint probability distribution and simulate mutual scenarios for uncertainties. A deterministic and data-driven robust optimization framework was proposed for energy systems optimization under uncertainty. The uncertainty set is constructed by support vector clustering based on real industrial data [39]. A datadriven robust optimization was applied to design and optimize the entire wastewater sludge to-biodiesel supply chain [42]. They develop a conservative support vector clustering (CSVS) method to construct an uncertainty set from limited data. The developed uncertainty set encloses the fuzzy support neighborhood of data samples, making it practical even when the available data is limited.

#### **4.3 Data-driven chance-constrained program**

Chance constrained programming is a practical and convenient approach to control risk in decision-making under uncertainty. However, due to unknown probability distributions of uncertainty parameters, the solution obtained from a chance-constrained optimization problem can be biased. In practice, instead of knowing the actual distribution of an uncertainty parameter, only a set of historical/ or real-time uncertainty data, which can be considered as samples taken from the actual (while ambiguous) distribution, can be observed and stored. On the other hand, even if the probability distribution of an uncertainty parameter is available, the chance-constrained program is computationally cumbersome. Motivated by Chance constrained programming weaknesses, data-driven chance-constrained optimization emerges as a new data-driven optimization paradigm. The data-driven stochastic programming approach is a data-driven risk-averse strategy to handle uncertainties in the era of big data effectively.

In contrast to the data-driven stochastic programming approach, data-driven chance-constrained programming is another paradigm focusing on chance constraint satisfaction under the worst-case probability instead of optimizing the worst-case expected objective. Although both data-driven chance-constrained programs and DRO adopt ambiguity sets in the uncertainty models, they have distinct model structures. Specifically, the data-driven chance-constrained program features constraints subject to uncertainty in probability distributions. Simultaneously, DRO typically only involves the worst-case expectation of an objective function concerning a family of probability distributions [28]. In the data-driven stochastic programming approach, historical data is utilized to learn the uncertain parameters' distributions.

Data-driven chance-constrained programs with moment-based ambiguity sets, distance-based ambiguity set, Prohorov metric-based ambiguity sets [44], φ-divergence based ambiguity set [45], kernel smoothing method [46], Wasserstein ambiguity set [47].

Ghosal and Wiesemann [48] applied for Data-driven chance-constrained programs in the capacitated vehicle routing problem (CVRP), which asks for the costoptimal delivery of a single product geographically dispersed customers through a fleet of capacity-constrained vehicles. They model the customer demands as a random vector whose distribution is only known to belong to an ambiguity set.

#### **4.4 Leveraging DL techniques in the data-driven optimization**

The recent development in the data science field, AI, and ML techniques have enabled intelligent and automated DSS and real-time analytics coupled with computing power improvements. Thus, AI techniques are applied to big data sources to extract the knowledge-based rules or identify the underlying rules and patterns by ML techniques, to drive the systems toward set objectives. DL is an ML technique that can extract high levels of information and knowledge from massive data volumes. DL algorithms consist of multiple processing layers to learn representations of data with multiple abstraction levels [26]. For example, recently, DL techniques have been used to accurately forecasting customer demand, price, and inventory leading to optimization of supply chain performance. An intelligent forecasting system leads to optimize performance, reduce costs, and increase sales and profit. DL techniques can apply deep neural network architectures to solve various complex problems. The DL paradigm requires high computing power and a large amount of data for training. The recent advances in parallel architectures and GUP (Graphical Processing Unit) enabled the necessary computing power required in deep neural networks (DNN). The emergence of advanced IoT and blockchain technologies has also solved the need for a large amount of data to learn. IoT and blockchain result in massive amounts of streaming real-time data often referred to as "big data," which brings new opportunities to control and manage supply chains [49]. Optimizing the parameters in DNN is a challenging undertaking. Several optimization algorithms such as Adam, Adagrad, RMSprop, have been proposed to optimize the network parameters in DNN and improve generalizability. This technique, which stabilizes the optimization, paved the way for learning deeper networks [50]. In real applications, uncertainty data exhibit very complex and highly nonlinear characteristics. DNN can be used to uncover useful patterns of uncertainty data for optimizing under uncertainty [28]. Deep datadriven optimization could be considered a "hybrid" system that integrates the deep data-driven system based on DL to forecast the uncertainty parameters. The modelbased system is based on mathematical programming to drive the optimal decisions from predicted parameters (the deep data-driven system). In the DL-based system, DNN has been applied to analyze features, complex interactions, and relationships among features of a problem from samples of the dataset and learn model, which can be used for demand, inventory, and price forecasting. Kilimci et al. [51] developed an intelligent demand forecasting system based on the analysis and interpretation of the historical data using different forecasting methods, including support vector regression algorithm, time series analysis techniques, and DL models. In a study, the Auto-Regressive Integrated the backpropagation (BP) network method, recurrent neural network (RNN) method, and Moving Average (ARIMA) model were tested to forecast the price of agricultural products [52]. Yu et al. [53] developed an online big-data-driven forecasting model of Google trends to improve oil consumption prediction. Their proposed forecasting model considers traditional econometric models (LogR and LR) and typical AI techniques (BPNN, SVM, DT, and ELM).

Accurate automatic optimization heuristics are necessary for dealing with the complexity and diversity of modern hardware and software. ML is a proven technique for learning such heuristics, but its success is bound by the quality of the features used. Developers must handcraft these features through a combination of expert domain knowledge and trial and error. This makes the quality of the final model directly dependent on the skill and available time of the system architect. DL techniques are a better way to build heuristics. A deep neural network can learn

#### *Artificial Intelligence and Its Application in Optimization under Uncertainty DOI: http://dx.doi.org/10.5772/intechopen.98628*

heuristics over raw code entirely without using code features. The neural network simultaneously constructs appropriate representations of the code and learns how best to optimize, removing the need for manual feature creation. DNN can improve the accuracy of models without the help of human experts. Generally, this approach is a fundamental way to integrate forecast approaches into mathematical optimization models. First, a probabilistic forecast approach for future uncertainties is given by exploiting the advanced DL structures. Second, a model-based system based on mathematical programming is applied to derive the optimal decisions from the forecasting data. Comparison and evaluation of the forecasting models are significant since DL models can have different performances depending on the properties of the data [54, 55]. The performances of DL models differ according to the forecasting time, training duration, target data, and simple or ensemble structure [56, 57].

In a study, Nam et al. [54, 55] applied DL-based models to forecast fluctuating electricity demand and generation in renewable energy systems. This study compares and evaluates DL models and conventional statistical models. The DL models include DNN, long short-term memory, gated recurrent unit, and the disadvantages of conventional statistical models such as multiple linear regression and seasonal autoregressive integrated moving average. In another study, the operation of a cryogenic NGL recovery unit for the extraction of NGL has been optimized by implementing data-driven techniques [58]. The proposed approach is based on an optimization framework that integrates dynamic process simulations with two DL-based surrogate models using a long short-term memory (LSTM) layout with a bidirectional recurrent neural network (RNN) structure. Kilimci et al. [51] developed an intelligent demand forecasting system. This improved model is based on analyzing and interpreting the historical data using different forecasting methods, including time series analysis techniques, support vector regression algorithm, and DL models.

Accessing a sufficient amount of data for some optimization models is a practical challenge. For example, the quality of scenario-based optimization frameworks strongly depends on access to a sufficient amount of uncertain data. However, in practice, the amount of uncertainty data sampled from the underlying distribution is limited. On the other hand, acquiring a sufficient amount of uncertainty data is extremely time-consuming and expensive in some cases, which leads to the limited application of some approaches [59]. To deal with the practical challenge of requiring an insufficient amount of data, deep generative models emerge as a new paradigm to generate synthetic uncertainty data with the aim of better decisions with insufficient uncertainty data. DL techniques could be applied to learn the useful intrinsic patterns from the available uncertainty data and generate synthetic uncertainty data. More specifically, in deep generative models, the correct data distribution is mimicked either implicitly or explicitly by the DL techniques. Then the learned distribution is used to generate new data points referred to as synthetic data [28]. After that, these synthetic data serve as input to an optimizing model to derive the optimal decisions. Some of the most commonly used deep generative models are variational autoencoders generative and adversarial networks [26]. These synthetic uncertainty data generated by the DL techniques can be potentially useful in the scenario-based optimization model.

## **4.5 Deep data-driven models**

DL models are a class of approximate models proven to have strong predictive capabilities for representing complex phenomena [60]. Approximate models are

currently experiencing a radical shift due to the advent of DL. However, our research into the existing literature reveals a scarcity of research utilizing DL in approximate modeling. The introduction of DL models into an optimization formulation provides a means to reduce the problem complexity and maintain model accuracy [60]. Recently it has been shown that DL models in the form of neural networks with rectified linear units can be exactly recast as a mixed-integer linear programming formulation. DL is a method to approximate complex systems and tasks by exploiting large amounts of data to develop rigorous mathematical models [60].

Using DNN to model real-world problems is a powerful tool, as they provide an efficient abstraction that can be used to analyze the structure of the task at hand. The rigorous mathematical model is developed based on neural networks modeling complex systems and optimizing their operations in the deep data-driven model framework. This approximate model is developed by exploiting large amounts of data using DL techniques. Then the solving method is applied to obtain the optimal solutions of the developed optimization model. Developing an optimal solution to the approximate model remains challenging [60].

Pfrommer et al. [61] utilized a stochastic genetic algorithm to optimize a composite textile draping process where a neural network was utilized as a surrogate model. Marino et al. [62] presented an approach for modeling and planning under uncertainty using deep Bayesian neural networks (DBNNs). They use DBNNs to learn a stochastic model of the system dynamics. Planning is addressed as an open-loop trajectory optimization problem. In the study, DL-based surrogate modeling and optimization were proposed for microalgal biofuel production and photobioreactor design [63]. This surrogate model is built upon a few simulated results from the physical model to learn the sophisticated hydrodynamic and biochemical kinetic mechanisms; then adopts a hybrid stochastic optimization algorithm to explore untested processes and find optimal solutions. Tang & Zhang [64] developed a deep data-driven framework for modeling combustion systems and optimizing their operations. First, they developed a deep belief network to model the combustion systems. Next, they developed a multi-objective optimization model by integrating the deep belief network-based models, the considered operational constraints, and the control variable constraints.

#### **4.6 Online learning-based data-driven optimization**

In conventional data-driven optimization frameworks, a set of uncertainty data serves as input to the data-driven system, in which learning typically takes place once by using learning techniques. This approach fails to account for real-time uncertainty data [28]. For example, in the DRO method, the uncertainty set of probability distributions is constructed from uncertainty data. Once the uncertainty sets of probability distributions are obtained, they remain fixed for the model-based system based on mathematical programming and are not updated or refined. However, in real practice, a vast number of uncertainty data are generated and collected sequentially in an online fashion; therefore, data-driven systems should be developed to analyze the real-time data. An online-learning-based data-driven optimization framework emerges as a new data-driven optimization paradigm. Learning takes place iteratively to account for real-time data, and the data-driven system is updated in an online fashion. The framework of online-learning-based data-driven optimization could be considered a hybrid system that integrates the online data-driven and model-based systems. In the online data-driven system, the real-time uncertainty data should

#### *Artificial Intelligence and Its Application in Optimization under Uncertainty DOI: http://dx.doi.org/10.5772/intechopen.98628*

be saved and analyzed sequentially based on ML to extract sequentially useful and relevant information from the real-time data. The online data-driven system (such as the uncertainty sets, probability distributions sets, and forecasting data) that serve as input to a model-based system should be updated in an online fashion. Then in the model-based system, the optimal decisions are made sequentially from the real-time information based on mathematical programming. There is a "feedback" channel for information flow returning from the model-based system to the data-driven system in this framework. The information flow is fed into the mathematical programming problem from the ML results. Using the feedback control strategy delivers amazingly superior system performance (e.g., stability, robustness to disturbances, and safety) [28]. **Figure 2** presents the potential schematic of the online learning-based datadriven optimization system.

The online-learning-based data-driven optimization framework, updating the data-driven systems, and developing efficient algorithms to solve online learningbased mathematical programming problems have become challenging.

#### **4.7 Leveraging RL techniques for optimization**

RL has transformed AI, especially after the success of Google DeepMind. This branch of ML epitomizes a step toward building autonomous systems by understanding the visual world. Deep RL is currently applied to different sorts of problems that were previously obstinate. In this subsection, the authors will analyze Deep RL and its applications in optimization.

RL is one of the ML areas recently applied to tackle complex sequential decision problems. RL is concerned with how a software agent should choose an action to maximize a cumulative reward. RL is considered an optimal solution in addressing challenges where many factors must be taken into account, like supply chain management. For example, Q-learning is a type of RL algorithm that is applied to tackle simple optimization problems. In this approach, the Q-value has been applied to any state of the system. Although the classical RL algorithms guarantee optimal policy, these algorithms cannot promptly solve large states or actions. Many problems in the real world have large and action spaces. Applying RL algorithms for solving large problems would be nearly impossible, as these models would be costly to train. Therefore, deep RL emerges as a new method in which DNN is used to approximate any of the following RL components. Recently, deep Q-network (DQN) algorithms have been used in different areas. For example, deep Q-network (DQN) algorithms have been applied to solve supply chain optimization problems. These DQNs operate as the decision-maker of each agent. That results in a competitive game in which each DQN agent plays independently to minimize its own cost. Instead, recently a unified

#### **Figure 2.**

*The schematic of the "closed-loop" online learning-based data-driven optimization framework.*

framework has been proposed in which the agents still play independently from one another. Still, in the training phase, this model uses a feedback scheme so that the DQN agent learns the total cost for the whole network and, over time, learns to minimize it.

Like other types of reinforcement ML technique, multi-agent RL is a system of agents (e.g., robots, machines, and cars) interacting within a common environment. Each agent decides each time-step and works along with the other agent(s) to achieve a given goal. The agents are learnable units that want to learn policy on the fly to maximize the long-term reward through the interaction with the environment. Recently the multi-agent RL techniques have been applied to develop the supply chain management (SCM) systems that perform optimally for each entity in the chain. A supply chain can be defined as a network of autonomous business entities collectively responsible for procurement, manufacturing, storing, and distribution [65]. Entities in a supply chain have different sets of environmental constraints and objectives.

One of the biggest challenges of the development of MAS based supply chain is designing agent policies. To address designing agent policies, recently, automatic policy designing by RL has drawn attention. RL is considered an optimal solution in addressing challenges where a huge number of factors must be taken into account, like SCM. RL technique does not require datasets covering all environments, constraints, operations, and entity operation results. A multi-agent RL (MARL)-based SCM system can enable agents to learn automatically policies that optimize the supply chain performance using RL concerning certain constraints, environments, and objectives to optimize the performance. More specifically, the RL technique enables an agent to learn a policy by correcting necessary data itself during trial-and-error on the content of operations [66]. All agents also simultaneously cooperate to optimize the performances of the entire supply chain. RL technique can be applied for a certain problem when all processes concerning the problem satisfy a Markov property. Environmental change for a certain agent depends on the previous state of the environment and the agent's action. It is impossible to assume the Markov property because an agent's environmental change depends on the previous state for the agent and the other agent's actions.

There are two problems in developing a MARL technique for SCM: Building Markov decision processes for a supply chain and then avoiding learning stagnation among agents in learning processes. For solving these problems, a learning management method with deep neural network (DNN)-weight evolution (LM-DWE) has been applied [67]. Fuji et al. [67] developed a multi-agent RL technique to develop a supply chain management (SCM) system that enables agents to learn policies that optimize SC performance. They applied a learning management method with deepneural-network (DNN)-weight evolution (LM-DWE) in the MARL for SCM. An RL framework-FeedRec has been used in a study to optimize long-term user engagement [68]. They used hierarchical LSTM to design the Q-Network to model the complex user behaviors; they also used Q Network to simulate the environment. Zhang et al. [69] proposed a multi-agent learning (MAL) algorithm and applied it for optimizing online resource allocation in cluster networks.

#### **4.8 Deep RL for solving NP-hard problems**

Optimization in current DSS has a highly interdisciplinary nature related to integrating different techniques and paradigms for solving complex real-world problems. The design of efficient NP-hard combinatorial optimization problems is a

#### *Artificial Intelligence and Its Application in Optimization under Uncertainty DOI: http://dx.doi.org/10.5772/intechopen.98628*

fascinating issue and often requires significant specialized knowledge and trial-anderror. NP-hard problems are solved with exact methods, heuristic algorithms, or a combination of them. Although exact methods provide optimal answers, they have the limitation of performing inefficiently in time complexity. Heuristics are used to improve computational time efficiency and provide decent or near-optimal solutions [70]. According to the definition of Burke et al. [71], a hyper-heuristic is a searching mechanism that aims to select or generate appropriate heuristics to solve an optimization problem. However, the effectiveness of general heuristic algorithms is dependent on the problem being considered, and high levels of performance often require extensive tailoring and domain-specific knowledge. ML strategies have become a promising route to addressing these challenges, which led to the development of meta-algorithms to various combinatorial problems.

Solution approaches meta-heuristics and hyper-heuristics have been developed to tackle the NP-hard combinatorial optimization problem [72]. Recently, hyperheuristics arise in this context as efficient methodologies for selecting or generating (meta) heuristics to solve NP-hard optimization problems. Hyper-heuristics are categorized into heuristic selection (Methodologies to select) and heuristic generation (Methodologies to generate) [71]. Deep RL is a possible learning method that can automatically solve various optimization problems [73]. Encouragingly, characteristics of the deep RL method have been found in comparison with classical methods, e.g., strong generalization ability and fast solving speed. RL methods can be used at different levels to solve combinatorial optimization problems. They can be applied directly to the problem, as part of a meta-heuristic, or as part of hyper-heuristics [74]. Utilizing advanced computation power with meta-heuristics algorithms and massivedata processing techniques has successfully solved various NP-hard problems. However, meta-heuristic approaches find good solutions which, do not guarantee the determination of the global optimum. Meta-heuristics still face the limitations of exploitation and exploration, which consists of choosing between a greedy search and a wider exploration of the solution space.

A way to guide Meta-heuristic algorithms during the search for better solutions is to generate the initial population of a genetic algorithm by using a technique of Q-Learning algorithm.

The hyper-heuristic for heuristic selection can use RL algorithms, enabling the system to autonomously select the meta-heuristic to use in the optimization process and the respective parameters. For example, Falcão et al. [74] proposed a hyperheuristic module for solving scheduling problems in manufacturing systems. The proposed hyper-heuristic module uses an RL algorithm, which enables the system to autonomously select the meta-heuristic to use in the optimization process and the respective parameters. Cano-Belmán et al. [75] proposed a heuristic generation scatter search algorithm to address a mixed-model assembly line sequencing problem. Khalil et al. (Dai et al., 2017) developed a neural combinatorial optimization framework that utilizes neural networks and RL to tackle combinatorial optimization problems. The developed meta-algorithm automatically learns good heuristics for a diverse range of optimization problems over graphs. Mosadegh et al. [72] proposed novel hyper-simulated annealing (HSA) to tackle the NP-hard problem. They developed new mathematical models to describe a mixed-model sequencing problem with stochastic processing times (MMSPSP). The HSA applies a Q-learning algorithm to select appropriate heuristics through its search process [72]. The main idea is to conduct simulated annealing (SA)-based algorithms to find a suitable heuristic among available ones creating a neighbor solution(s).

#### **Case study 1: Data-driven robust optimization under correlated uncertainty.**

The first case study focuses on the production schedule. The data-driven robust optimization applied for an ethylene plant is predicted to hedge against the fluctuations generated from continuous production processes. For capturing and enrich the valid information of uncertainties, copulas are introduced to estimate the joint probability distribution and simulate mutual scenarios for uncertainties [43]. For this purpose, cutting planes are generated to remove unnecessary uncertain scenarios in the uncertainty sets. Then robust formulations induced by the cut set are proposed to reduce conservatism and improve the robustness of scheduling solutions. They consider the robust counterpart induced by the classical uncertainty set, where the difference to the best possible solution over all scenarios is to be minimized. Instead of focuses on simple uncertainty sets that are either finite or hyperboles, they considered problems with more flexible and realistic ellipsoidal uncertainty sets. In this research, the cut sets of flexible uncertainty sets are proposed. They used the historical data to correct the uncertainties and drive the reformulation of constraints with uncertainties. The new robust formulations induced by cut sets are derived for linear programming (LP) and mixed-integer linear programming (MILP) problems. Through the real-world ethylene plant example, the correlations between uncertain consumption rates of furnaces are analyzed.

In this research, Decision-makers prefer to obtain robust solutions immune to most high-frequency uncertain scenarios. Since in production scheduling problems, many uncertainties are associated with the entire production network, a process, or equipment, which makes them correlated and difficult to be separated. So, in this optimization research, uncertainties are assumed to be dependent. In this research, the cut sets of flexible uncertainty sets are proposed.

Deterministic solutions are regarded as theoretically optimal at most times, and robust solutions provide references for decision-makers, which may not be optimal but feasible and applicable. It is always neglected that stricter descriptions of uncertainties could also create great profits. The full coverage of uncertain values usually leads to unpractical and conservative results. The improper simplification of uncertainty scenarios will cause infeasibility when the solutions are implemented in the volatile production process. Thus, historical data should be introduced to correct the uncertainties and drive the reformulation of constraints with uncertainties. For eliminating the worst-case formulation scenario for robust optimization and decrease conservatism, the cut set of flexible uncertainty sets is constructed by introducing cutting planes. Cutting planes are generated to construct cut sets for the outer approximation of most uncertain scenarios. Since the size of the uncertainty set directly influences the quality of robust solutions, in this research, the more uncertain values are considered.

They stated that utilizing the data-driven robust optimization approach causes the decision-makers to have the ability to decide how many uncertain scenarios are considered in the model and to provide effective, economical, and robust scheduling plans. Finally, it causes fluctuations in the production performance captured and controlled below a lower level of conservatism.

#### **Case study 2: wastewater sludge-to-biodiesel supply chain design.**

Designing and optimizing the wastewater sludge-to-biodiesel supply chain facilitates the development of its large-scale production [42]. Hence, this case study evaluates Data-driven robust optimization for supply chain designing and optimization. The entire wastewater sludge-to-biodiesel supply chain over multiple periods is systematically designed and optimized based on the uncertainty sets constructed from

#### *Artificial Intelligence and Its Application in Optimization under Uncertainty DOI: http://dx.doi.org/10.5772/intechopen.98628*

the data of uncertain parameters. In this research, a data-driven robust optimization has been adopted, which constructs the uncertainty sets from the data of uncertain parameters utilizing support vector clustering. In contrast, the conventional uncertainty sets are driven without incorporating the data, which results in a high cost of robustness. The developed uncertainty set in this research encloses the fuzzy support neighborhood of data samples that makes it practical even when the available data is limited. The research results show that the proposed data-driven robust optimization approach can yield robust supply chain decisions with the same degree of robustness but at a lower cost than robust conventional optimization approaches.
