4. Scalable control for distributed energy resources

#### 4.1. Overview

using the network model of Hutcheon and Bialek [30], technical generator information provided to the authors by ENGIE, and multiarea demand and renewable energy information collected from national system operators (see [31] for details). We consider eight representative day types, one weekday and one weekend day per season, as being representative of the

We consider 4 day-ahead scheduling models: the DUCR model and the SUC model with 30 (SUC30), 60 (SUC60), and 120 (SUC120) scenarios. The sizes of the different day-ahead scheduling models are presented in Table 1, where the size of the stochastic models refers to the size of the extensive form. While the DUCR model is of the scale of problems that fit in the memory of a single machine and can be solved by a commercial solver, the SUC models in extensive

Table 2 presents the solution time statistics for all day-ahead scheduling policies. In the case of SUC, we report these results for the two dual initialization alternatives proposed in Section 3.2. The results of Table 2 indicate that the OPF initialization significantly outperforms the LP approach in terms of termination time. This is mainly due to the fact that the OPF approach provides nontrivial lower bounds including information for all scenarios much faster than the LP approach. On the other hand, the solution times of SUC60 and DUCR indicate that, using distributed computing, we can solve SUC at a comparable run time to that required by commercial solvers for DUCR on large-scale systems. Moreover, as shown in Table 3, for a given hard constraint on solution wall time such as 2 h (which is common for day-ahead power system operations), the proposed distributed algorithm provides solutions to SUC with up to 60 scenarios within 2% of optimality, which is acceptable for operational purposes.

Model Scenarios Variables Constraints Integers DUCR 1 570.432 655.784 9.552 SUC30 30 13334.400 16182.180 293.088 SUC60 60 26668.800 32364.360 579.648 SUC120 120 53337.600 64728.720 1152.768

Model Nodes used Initialization Running time [h] avg. (min.–max.) Worst final gap [%]

10 OPF 0.8 (0.3–1.8) 1.00

10 OPF 1.5 (0.6–4.7) 0.97

10 OPF >3.0 (0.6–10.0) 1.07

DUCR 1 – 1.9 (0.6–4.2) 0.95 SUC30 10 LP 1.1 (0.7–2.2) 0.93

SUC60 10 LP 3.2 (1.1–8.4) 1.00

SUC120 10 LP >6.1 (1.6–10.0) 1.68

different conditions faced by the system throughout the year.

32 Recent Progress in Parallel and Distributed Computing

form are beyond current capabilities of commercial solvers.

Table 1. Problem sizes.

Table 2. Solution time statistics over 8 day types.

Residential demand response has gained significant attention in recent years as an underutilized source of flexibility in power systems, and is expected to become highly valuable as a balancing resource as increasing amounts of renewable energy are being integrated into the grid. However, the mobilization of demand response by means of real-time pricing, which represents the economists' gold standard and can be traced back to the seminal work of Schweppe et al. [32], has so far fallen short of expectations due to several obstacles, including regulation issues, market structure, incentives to consumers, and technological limitations.

The ColorPower architecture [7, 8, 9] aims at releasing the potent power of demand response by approaching electricity as a service of differentiated quality, rather than a commodity that residential consumers are willing to trade in real time [33]. In this architecture, the coordination problem of determining which devices should consume power at what times is solved through distributed aggregation and stochastic control. The consumer designates devices or device modes using priority tiers (colors). These tiers correspond to "service level" plans, which are easy to design and implement: we can simply map the "color" designations of electrical devices into plans. A "more flexible" color means less certainty of when a device will run (e.g., time when a pool pump runs), or lower quality service delivered by a device (e.g., wider temperature ranges, slower electrical vehicle charging). These types of economic decision-making are eminently compatible with consumer desires and economic design, as evidenced by the wide range of quality-of-service contracts offered in other industries.

Furthermore, the self-identified priority tiers of the ColorPower approach enable retail power participation in wholesale energy markets, lifting the economic obstacles for demand response: since the demand for power can be differentiated into tiers with a priority order, the demand in each tier can be separately bid into the current wholesale or local (DSO level) energy markets. The price for each tier can be set according to the cost of supplying demand response from that tier, which in turn is linked to the incentives necessary for securing customer participation in the demand response program. This allows aggregated demand to send price signals in the form of a decreasing buy bid curve. Market information thus flows bidirectionally. A small amount of flexible demand can then buffer the volatility of the overall power demand by yielding power to the inflexible devices as necessary (based upon the priority chosen by the customer), while fairly distributing power to all customer devices within a demand tier.

Technological limitations to the massive deployment of demand response are dealt with by deploying field-proven stochastic control techniques across the distribution network, with the objective of subtly shifting the schedules of millions of devices in real time, based upon the conditions of the grid. These control techniques include the CSMA/CD algorithms that permit cellular phones to share narrow radio frequency bands, telephone switch control algorithms, and operating system thread scheduling, as well as examples from nature such as social insect hive behaviors and bacterial quorum sensing. Moreover, the ubiquity of Internet communications allows us to consider using the Internet platform itself for end-to-end communications between machines.

At a high level, the ColorPower algorithm operates by aggregating the demand flexibility state information of each agent into a global estimate of total consumer flexibility. This aggregate and the current demand target are then broadcast via IP multicast throughout the system, and every local controller (typically one per consumer or one per device) combines the overall model and its local state to make a stochastic control decision. With each iteration of aggregation, broadcast, and control, the overall system moves toward the target demand, set by the utility or the ISO, TSO, or DSO, allowing the system as a whole to rapidly achieve any given target of demand and closely tracking target ramps. Note that aggregation has the beneficial side-effect of preserving the privacy of individual consumers: their demand information simply becomes part of an overall statistic.

The proposed architectural approach supplements the inadequacy of pure market-based control approaches by introducing an automated, distributed, and cooperative communications feedback loop between the system and large populations of cooperative devices at the edge of the network. TSO markets and the evolving DSO local energy markets of the future will have both deep markets and distributed control architecture pushed out to the edge of the network. This smart grid architecture for demand response in the mass market is expected to be a key asset in addressing the challenges of renewable energy integration and the transition to a low-carbon economy.

#### 4.2. The ColorPower control problem

A ColorPower system consists of a set of n agents, each owning a set of electrical devices organized into k colors, where lower-numbered colors are intended to be shut off first (e.g., 1 for "green" pool pumps, 2 for "green" HVAC, 3 for "yellow" pool pumps, etc.), and where each color has its own time constants.

Within each color, every device is either Enabled, meaning that it can draw power freely, or Disabled, meaning that has been shut off or placed in a lower power mode. In order to prevent damage to appliances and/or customer annoyance, devices must wait through a Refractory period after switching between Disabled and Enabled, before they return to being Flexible and can switch again. These combinations give four device states (e.g., Enabled and Flexible, EF), through which each device in the ColorPower system moves according to the modified Markov model of Figure 2: randomly from EF to DR and DF to ER (becoming disabled with probability poff and enabled with probability pon) and by randomized timeout from ER to EF and DR to DF (a fixed length of T�<sup>F</sup> plus a uniform random addition of up to T�<sup>V</sup>).

The ColorPower control problem can then be stated as dynamically adjusting pon and pof f for each agent and color tier, in a distributed manner, so that the aggregate consumption of the system follows a demand goal given by the operator of the high-voltage network.

#### 4.3. The ColorPower architecture

form of a decreasing buy bid curve. Market information thus flows bidirectionally. A small amount of flexible demand can then buffer the volatility of the overall power demand by yielding power to the inflexible devices as necessary (based upon the priority chosen by the customer), while fairly distributing power to all customer devices within a demand tier.

Technological limitations to the massive deployment of demand response are dealt with by deploying field-proven stochastic control techniques across the distribution network, with the objective of subtly shifting the schedules of millions of devices in real time, based upon the conditions of the grid. These control techniques include the CSMA/CD algorithms that permit cellular phones to share narrow radio frequency bands, telephone switch control algorithms, and operating system thread scheduling, as well as examples from nature such as social insect hive behaviors and bacterial quorum sensing. Moreover, the ubiquity of Internet communications allows us to consider using the Internet platform itself for end-to-end communications

At a high level, the ColorPower algorithm operates by aggregating the demand flexibility state information of each agent into a global estimate of total consumer flexibility. This aggregate and the current demand target are then broadcast via IP multicast throughout the system, and every local controller (typically one per consumer or one per device) combines the overall model and its local state to make a stochastic control decision. With each iteration of aggregation, broadcast, and control, the overall system moves toward the target demand, set by the utility or the ISO, TSO, or DSO, allowing the system as a whole to rapidly achieve any given target of demand and closely tracking target ramps. Note that aggregation has the beneficial side-effect of preserving the privacy of individual consumers: their demand information sim-

The proposed architectural approach supplements the inadequacy of pure market-based control approaches by introducing an automated, distributed, and cooperative communications feedback loop between the system and large populations of cooperative devices at the edge of the network. TSO markets and the evolving DSO local energy markets of the future will have both deep markets and distributed control architecture pushed out to the edge of the network. This smart grid architecture for demand response in the mass market is expected to be a key asset in addressing the challenges of renewable energy integration and the transition to a low-carbon economy.

A ColorPower system consists of a set of n agents, each owning a set of electrical devices organized into k colors, where lower-numbered colors are intended to be shut off first (e.g., 1 for "green" pool pumps, 2 for "green" HVAC, 3 for "yellow" pool pumps, etc.), and where

Within each color, every device is either Enabled, meaning that it can draw power freely, or Disabled, meaning that has been shut off or placed in a lower power mode. In order to prevent damage to appliances and/or customer annoyance, devices must wait through a Refractory period after switching between Disabled and Enabled, before they return to being Flexible and can switch again. These combinations give four device states (e.g., Enabled and Flexible, EF),

between machines.

ply becomes part of an overall statistic.

34 Recent Progress in Parallel and Distributed Computing

4.2. The ColorPower control problem

each color has its own time constants.

The block diagram of the ColorPower control architecture is presented in Figure 3. Each ColorPower client (i.e., the controller inside a device) regulates the state transitions of the devices under its control. Each client state sðt, aÞ is aggregated to produce a global state estimate ^sðtÞ, which is broadcasted along with a goal gðtÞ (the demand target set by the utility or the ISO, TSO, or DSO), allowing clients to shape demand by independently computing the control state cðt, aÞ.

The state sðt, aÞ of a client a at time t sums the power demands of the device(s) under its control, and these values are aggregated using a distributed algorithm (e.g., a spanning tree in Ref. [7]) and fed to a state estimator to get an overall estimate of the true state ^sðtÞ of total demand in each state for each color. This estimate is then broadcast to all clients (e.g., by gossip-like diffusion in Ref. [7]), along with the demand shaping goal gðtÞ for the next total Enabled demand over all colors. The controller at each client a sets its control state cðt, aÞ, defined as the set of transition probabilities pon,i, <sup>a</sup> and poff,i, <sup>a</sup> for each color i. Finally, demands move through their states according to those transition probabilities, subject to exogenous disturbances such as changes in demand due to customer override, changing environmental conditions, imprecision in measurement, among others.

Figure 2 Markov model-based device state switching [8, 9].

Figure 3 Block diagram of the control architecture [8, 9].

Note that the aggregation and broadcast algorithms must be chosen carefully in order to ensure that the communication requirements are lightweight enough to allow control rounds that last for a few seconds on low-cost hardware. The choice of algorithm depends on the network structure: for mesh networks, for example, spanning tree aggregation and gossipbased broadcast are fast and efficient (for details, see [7]).

#### 4.4. ColorPower control algorithm

The ColorPower control algorithm, determines the control vector cðt, aÞ by a stochastic controller formulated to satisfy four constraints:

Goal tracking: The total Enabled demand in sðtÞ should track gðtÞ as closely as possible: i.e., the sum of Enabled demand over all colors i should be equal to the goal. This is formalized as the equation:

$$\mathbf{g}(t) = \sum\_{i} (|EF\_i| + |ER\_i|).$$

Color priority: Devices with lower-numbered colors should be shut off before devices with higher-numbered colors. This is formalized as:

$$|E F\_i| + |E R\_i| = \begin{cases} D\_i - D\_{i+1} & \text{if } D\_i \preceq g(t) \\ g(t) - D\_{i+1} & \text{if } D\_{i+1} \preceq g(t) < D\_i \\ 0 & \text{otherwise} \end{cases}$$

so that devices are Enabled from the highest color downward, where Di is the demand for the ith color and above:

$$D\_{\mathbf{i}} = \sum\_{j \ge i} (|EF\_j| + |ER\_{\mathbf{j}}| + |DF\_{\mathbf{j}}| + |DR\_{\mathbf{j}}|).$$

Fairness: When the goal leads to some devices with a particular color being Enabled and other devices with that color being Disabled, each device has the same expected likelihood of being Disabled. This means that the control state is identical for every client.

Cycling: Devices within a color trade-off which devices are Enabled and which are Disabled such that no device is unfairly burdened by initial bad luck. This is ensured by asserting the constraint:

$$(|EF\_i| > 0) \cap (|DF\_i| > 0) \Rightarrow (p\_{\text{on},i,a} > 0) \cap (p\_{\text{off},i,a} > 0) \therefore$$

This means that any color with a mixture of Enabled and Disabled Flexible devices will always be switching the state of some devices. For this last constraint, there is a tradeoff between how quickly devices cycle and how much flexibility is held in reserve for future goal tracking; we balance these with a target ratio f of the minimum ratio between pairs of corresponding Flexible and Refractory states.

Since the controller acts indirectly, by manipulating the pon and poff transition probabilities of devices, the only resource available for meeting these constraints is the demand in the flexible states EF and DF for each tier. When it is not possible to satisfy all four constraints simultaneously, the ColorPower controller prioritizes the constraints in order of their importance. Fairness and qualitative color guarantees are given highest priority, since these are part of the contract with customers: fairness by ensuring that the expected enablement fraction of each device is equivalent (though particular clients may achieve this in different ways, depending on their type and customer settings). Qualitative priority is handled by rules that prohibit flexibility from being considered by the controller outside of contractually allowable circumstances. Constraints are enforced sequentially. First comes goal tracking—the actual shaping of demand to meet power schedules. Second is the soft color priority, which ensures that in those transient situations when goal tracking causes some devices to be in the wrong state, it is eventually corrected. Cycling is last, because it is defined only over long periods of time and thus is the least time critical to satisfy. A controller respecting the aforementioned constraints is described in Ref. [8].

#### 4.5. Numerical experiment

Note that the aggregation and broadcast algorithms must be chosen carefully in order to ensure that the communication requirements are lightweight enough to allow control rounds that last for a few seconds on low-cost hardware. The choice of algorithm depends on the network structure: for mesh networks, for example, spanning tree aggregation and gossip-

The ColorPower control algorithm, determines the control vector cðt, aÞ by a stochastic con-

Goal tracking: The total Enabled demand in sðtÞ should track gðtÞ as closely as possible: i.e., the sum of Enabled demand over all colors i should be equal to the goal. This is formalized as

Color priority: Devices with lower-numbered colors should be shut off before devices with

so that devices are Enabled from the highest color downward, where Di is the demand for

ðjEFijþjERijÞ:

Di � Diþ<sup>1</sup> if Di ≤ gðtÞ gðtÞ � Diþ<sup>1</sup> if Diþ<sup>1</sup> ≤ gðtÞ < Di 0 otherwise,

<sup>g</sup>ðtÞ ¼ <sup>X</sup> i

> 8 < :

based broadcast are fast and efficient (for details, see [7]).

4.4. ColorPower control algorithm

the equation:

the ith color and above:

troller formulated to satisfy four constraints:

Figure 3 Block diagram of the control architecture [8, 9].

36 Recent Progress in Parallel and Distributed Computing

higher-numbered colors. This is formalized as:

jEFijþjERij ¼

We have implemented and tested the proposed demand response approach into the ColorPower software platform [8]. Simulations are executed with the following parameters: 10 trials per condition for 10,000 controllable devices, each device consumes 1 kW of power (for a total of 10 MW demand), devices are 20% green (low priority), 50% yellow (medium priority) and 30% red (high priority), the measurement error is ε = 0.1% (0.001), the rounds are 10 seconds long and all the Refractory time variables are 40 rounds. Error is measured by taking the ratio of the difference of a state from optimal versus the total power.

The results of the simulation test are shown in Figure 4. When peak control is desired, the aggregate demand remains below the quota, while individual loads are subjected stochastically

Figure 4 Simulation results with 10,000 independently fluctuating power loads. Demand is shown as a stacked graph, with enabled demand at the bottom in dark tones, disabled demand at the top in light tones, and Refractory demand cross hatched. The goal is the dashed line, which coincides with the total enabled demand for the experiment. The plot illustrates a peak shaving case where a power quota, the demand response target that may be provided from an externally-generated demand forecast, is used as a guide for the demand to follow.

to brief curtailments. Post-event rush-in, a potentially severe problem for both traditional demand response and price signal-based control systems, is also managed gracefully due to the specific design of the modified Markov model of Figure 2.

Taken together, these results indicate that the ColorPower approach, when coupled with an appropriate controller, should have the technological capability to flexibly and resiliently shape demand in most practical deployment scenarios.

### 5. Conclusions

We present two applications of distributed computing in power systems. On the one hand, we optimize high-voltage power system operations using a distributed asynchronous algorithm capable of solving stochastic unit commitment in comparable run times to those of a deterministic unit commitment model with reserve requirements, and within operationally acceptable time frames. On the other hand, we control demand response at the distribution level using stochastic distributed control, thereby enabling large-scale demand shaping during real-time operations of power systems. Together, both applications of distributed computing demonstrate the potential for efficiently managing flexible resources in smart grids and for systematically coping with the uncertainty and variability introduced by renewable energy.

### Acknowledgements

The authors acknowledge the Fair Isaac Corporation FICO for providing licenses for Xpress, and the Lawrence Livermore National Laboratory for granting access and computing time at the Sierra cluster. This research was funded by the ENGIE Chair on Energy Economics and Energy Risk Management and by the Université catholique de Louvain through an FSR grant.

A Distributed Computing Architecture for the Large-Scale Integration of Renewable Energy and Distributed… http://dx.doi.org/10.5772/67791 39

#### Nomenclature

to brief curtailments. Post-event rush-in, a potentially severe problem for both traditional demand response and price signal-based control systems, is also managed gracefully due to

Figure 4 Simulation results with 10,000 independently fluctuating power loads. Demand is shown as a stacked graph, with enabled demand at the bottom in dark tones, disabled demand at the top in light tones, and Refractory demand cross hatched. The goal is the dashed line, which coincides with the total enabled demand for the experiment. The plot illustrates a peak shaving case where a power quota, the demand response target that may be provided from an

Taken together, these results indicate that the ColorPower approach, when coupled with an appropriate controller, should have the technological capability to flexibly and resiliently

We present two applications of distributed computing in power systems. On the one hand, we optimize high-voltage power system operations using a distributed asynchronous algorithm capable of solving stochastic unit commitment in comparable run times to those of a deterministic unit commitment model with reserve requirements, and within operationally acceptable time frames. On the other hand, we control demand response at the distribution level using stochastic distributed control, thereby enabling large-scale demand shaping during real-time operations of power systems. Together, both applications of distributed computing demonstrate the potential for efficiently managing flexible resources in smart grids and for systemat-

The authors acknowledge the Fair Isaac Corporation FICO for providing licenses for Xpress, and the Lawrence Livermore National Laboratory for granting access and computing time at the Sierra cluster. This research was funded by the ENGIE Chair on Energy Economics and Energy Risk Management and by the Université catholique de Louvain through an FSR grant.

ically coping with the uncertainty and variability introduced by renewable energy.

the specific design of the modified Markov model of Figure 2.

externally-generated demand forecast, is used as a guide for the demand to follow.

38 Recent Progress in Parallel and Distributed Computing

shape demand in most practical deployment scenarios.

5. Conclusions

Acknowledgements

#### Deterministic and stochastic unit commitment



#### Asynchronous distributed algorithm for stochastic unit commitment


#### Distributed control for demand response

Parameters


Variables

