**5.1. Algorithm validation**

The set of constraints remains the same as they were already defined for each period *t*. Nevertheless, special considerations should be made to calculate de risk and to fulfill the riskconstrained optimization. As *Vt* only account for the expected profit, risk functions *Rt* should also similarly be calculated for the same state space in order to enforce the risk constraint.

116 Dynamic Programming and Bayesian Inference, Concepts and Applications

By definition, the linear regression will deliver an approximation that minimizes the mean square error on the entire dataset. In a stochastic setting where the same inputs leads to several different outputs, the regression will accurately estimate the expected value for a given set of inputs, provided the sample size and the approximation order are appropriate. This fits perfect for the case of the value function but for the approximation of the risk function some problems

Let suppose that the CVaR is chosen as risk metric. As the algorithm progresses, new data points are collected, i.e. a set of state variables and its corresponding simulated profits for the period. For approximating the CVaR associated to a particular point in the state space, one approach is to select a subset from the dataset whose input variables are "close" to the point. Then the CVaR is calculated first by sorting the profit values and then taking the mean of those below the specified α-quantile. Now, let suppose that a new data point is simulated and an update of the CVaR approximation is needed. To do so, the process described must be repeated, but now including the new data point. This simple approach has large disadvantages: all the data points must be stored and the mean is not easily updated as old data may be excluded or included of the zone below the α-quantile. These drawbacks are caused by the fact that the CVaR is quantile-based. To solve these difficulties, another solution is envisioned. Instead of approximating directly the CVaR, another risk measure is used to approximate the CVaR within the space state. The risk measure used is called Relative Lower Semideviation (RLS) and it is moment based instead. Hence, it can be updated more easily and without needing to store the entire dataset. These types of risk measures are described in detail in [13-16] and for

s

( )

where the equation (77) is the negative semideviation of degree *p* of the stochastic profit *Pt*.

It can be proven that these moment-based risk measures are coherent if 0≤*a* ≤1 and *p* ≥1. To approximate a CVaR with a 5%-quantile, in this work the parameters used are *a* =1 and *p* =9.5. To compute the approximation, two linear regressions were used to calculate the expectations on the profit and on the negative deviations, which can be updated using the same method proposed for the value functions. In a Monte Carlo scheme, a large amount of data is needed to compute reliable risk estimations. Therefore, the risk constraint should not

,0 - æ ö é ù éù éù <sup>=</sup> - ëû ëû ç ÷ ê ú è ø ë û


1

*p p p t B max B B* E E *t t* (77)

arise.

a stochastic profit *Pt* have the form:

( )

s

With the objective of validating the results of the proposed ADP algorithm, a first simple exemplary case is considered in which a thermal generator sells energy in the spot market and in a future contract. The results of the ADP algorithm were compared with the results of a conventional DP algorithm for which the space-state was discretized appropriately.

The fractions of energy sold in the spot market and in a quarter future contract were optimized considering that the future can be traded during the delivery period. The optimization determines three decision stages during this period, one at the beginning of each month, consisting on sell or buy energy in the future market based on the previous state. The spacestate previous to each decision is defined only by the level of future already sold, in order to keep tractable the DP problem.
