**Design of Robust Policies for Uncertain Natural Resource Systems: Application to the Classic Gordon-Schaefer Fishery Model**

Armando A. Rodriguez1, Jeffrey J. Dickeson2, John M. Anderies3 and Oguzhan Cifdaloz4 *Arizona State University USA* 

## **1. Introduction**

414 Challenges and Paradigms in Applied Robust Control

Lawley, M. (1999). Deadlock Avoidance for Production Systems with Flexible Routing. *IEEE* 

Lawley, M. (2002). Control of Deadlock and Blocking for Production Systems with

Lawley, M. & Reveliotis, S. (2001). Deadlock Avoidance for Sequential Resource Allocation

Lawley, M.; Reveliotis, S. & Ferreira, P. (1998). Application and Evaluation of Banker's

Lawley, M. & Sulistyono, W. (2002). Robust Supervisory Control Policies for Manufacturing

Papadimitriou, C. (1982). *Combinatorial Optimization: Algorithms and Complexity*, Prentice-

Park, S. & Lim, J. (1999). Fault-tolerant Robust Supervisor for Discrete Event Systems with

Reveliotis, S. (1999). Accommodating FMS Operational Contingencies through Routing

Reveliotis, S. (2000). Conflict Resolution in AGV Systems. *IIE Transactions*, Vol.32, No.7, (July

Wang, S.; Chew, S. & Lawley, M. (2008). Using Shared-Resource Capacity for Robust

*and Automation*, Vol.15, No.2, (April 1999), pp. 386–391, ISSN 1042-296X Ramadge, P. & Wonham, W. (1987). Supervisory Control of a Class of Discrete Event

*Systems,* Vol.13, No.4, (October 2001), pp. 385-404, ISSN 0920-6299

(November 2002), pp. 4563-4582, ISSN 0020-7543

(February 1998), pp. 73–100, ISSN 0920-6299

Hall, ISBN 0486402584, New Jersey, USA

206230, ISSN 0363-0129

1999), pp. 3–19, ISSN 1042-296X

2000), pp. 647-659, ISSN 0740-817X

Vol.18, No.3, (June 2002), pp. 346-359, ISSN 1042-296X

No.23, (December 2009), pp. 6499-6524, ISSN 0020-7543

1042-296X

*Transactions on Robotics and Automation*, Vol.15, No.3, (June 1999), pp. 497-510, ISSN

Unreliable Resources. *International Journal of Production Research,* Vol.40, No.17,

Systems: Hard and Easy Cases. *International Journal of Flexible Manufacturing* 

Algorithm for Deadlock-free Buffer Space Allocation in Flexible Manufacturing Systems. *International Journal of Flexible Manufacturing Systems,* Vol.10, No.1,

Systems with Unreliable Resources. *IEEE Transactions on Robotics and Automation*,

Model Uncertainty and Its Application to a Workcell. *IEEE Transactions on Robotics* 

Processes. S*IAM Journal on Control and Optimization,* Vol.25, No.1, (March 1985), pp.

Flexibility. *IEEE Transactions on Robotics and Automation*, Vol.15, No.1, (February

Control of Failure-Prone Manufacturing Systems. *IEEE Transactions on Systems, Man and Cybernetics, Part A*, Vol.38, No.3, (May 2008), pp. 605-627, ISSN 1083-4427 Wang, S.; Chew, S. & Lawley, M. (2009). Guidelines for Implementing Robust Supervisors in

Flexible Manufacturing Systems. *International Journal of Production Research,* Vol.47,

**Introduction.** A critical challenge faced by sustainability science is to develop robust strategies to cope with highly uncertain social and ecological dynamics. The increasing intensity with which human societies utilize (limited) natural resources is fueling the global debate and urging the development of resource management methodologies/policies to effectively deal with very demanding socio-bio-economical issues. Unfortunately, despite concerted efforts by governments, many natural resources continue to be poorly managed. The collapse of many fisheries worldwide is the most notable example (Clark, 2006; Clark et al., 2006; Holland, Gudmundsson; Myers, Worm 2003; Sethi et al., 2005) but other examples include forests (Moran, Ostrom), groundwater basins (Shah, 2000), and soils (ISRIC, 1990). The suggested causes are varied but (Clark, 2006) highlights two: (1) lack of consideration of economic incentives actually faced by economic agents and (2) uncertainty associated with the dynamics of biological populations. In the case of fisheries, Clark notes that "complexity and uncertainty will always limit the extent to which the effects of fishing can be understood or predicted" (Clark, 2006, p. 98). This suggests that we need policies capable of effectively managing natural resource systems despite the fact that we understand them poorly at best.

**Real-World Management Issues.** Real-world resource management must address three components: goal setting, practical (robust) implementation, and learning. Clark and others (Clark, 2007; 2006; Clark et al., 2006) have recently noted that practical implementation issues are frequently at the root of fishery management failures. For most fisheries, the necessary institutional contexts exist (Wilen, Homans) and we know what to do, yet management efforts fail. This suggests a need to focus on the actual *process* of resource management. For example, how can managers make decisions with incomplete information concerning how the resource and the resource users will respond to management actions?

<sup>1</sup>Electrical Engineering, Ira A. Fulton School of Engineering

<sup>2</sup>Electrical Engineering, Ira A. Fulton School of Engineering

<sup>3</sup>School of Human Evolution and Social Change, School of Sustainability

<sup>4</sup>ASELSAN, Inc. Microelectronics, Guidance and Electro-Optics Division, Turkey

cycle behavior (nonlinear oscillations) when parametric uncertainty is present. As such, it cannot be implemented in practice (because of prohibitively expensive switching costs). This motivates the need for robust policies that (1) do not exhibit limit cycle behavior and (2) offer performance (returns) as close to the optimal perfect information policy as model parameter (and derived fishery biomass target) uncertainty permits. Given the state of most world fisheries, our presentation focuses on a fishery that is nominally (i.e. believed to be) biologically over exploited (BOE); i.e. the optimal equilibrium biomass lies below the maximum sustainable yield biomass (Clark, 2006; Clark et al., 2006; Clark, 1990; Holland, Gudmundsson; Myers, Worm 2003; Sethi et al., 2005). By so doing, we directly address a globally critical renewable resource management problem. As in our prior work (Anderies et al., 2007), (Rodriguez et al., 2010), we do not seek "a best policy." Instead, we seek families of policies that are robust with respect to uncertainties that are likely to occur. Such families can, in principle, be used by a fishery manager to navigate the many tradeoffs (biological, ecological, social, economic, political) that must be confronted. More specifically, our effort to seek robust performance focuses on reducing the worst case downside performance; i.e. maximizing returns when we have the worst case combination of parameters. Such worst case (conservative) planning is critical to avoid/minimize the possibility of major regional/societal economical shortfalls; case in point, the recent "Great Recession." It is important to note that the simplicity of our model (vis-a-vis our performance objective of maximizing the net present value of returns) permits us to readily determine the worst case combination of model parameters (i.e. growth rate, carrying capacity, catchability, discount rate, price, cost of harvesting). Given this, we seek control laws that do not exhibit limit cycle behavior and whose returns are close (modulo limitations imposed by uncertainty) to that of the worst case perfect information optimal control policy - the best we could do in terms of return if we knew the parameters perfectly. Other design strategies are also examined; e.g. designing for the best case set of parameters. "Blended strategies" that attempt to do well for the worst case downside perturbation (i.e. minimize the economic downside) as well as the best case upside perturbation (i.e. maximize the economic upside) are also discussed. Such strategies seek to flatten the return-uncertainty characteristics over a broad range of likely parameters. The above optimal control (derived) policies are used as performance benchmarks/targets for the development of robust control policies. While our focus is on bounded deterministic parametric uncertainty, we also show how probability distributions for uncertain model parameters can be exploited to help in the selection of benchmark (optimal) policies. After targeting a suitable optimal (benchmark) policy, we show how robust policies can be used to approximate the benchmark (as closely as the uncertainty will permit) in order to achieve desired performance-robustness-vulnerability

<sup>417</sup> Design of Robust Policies for Uncertain Natural Resource Systems:

Application to the Classic Gordon-Schaefer Fishery Model

tradeoffs; e.g. have a return that is robust to worst case parameter perturbations.

methodology (framework) for robust control policies.

While the presentation is intended to provide an introduction into how concepts from optimal and robust control may be used to address critical issues associated with renewable resource management, the presentation also attempts to shed light on challenges for the controls community. Although the presentation builds on the prior work presented in (Anderies et al., 2007), (Rodriguez et al., 2010), the focus here is more on defining the problem, describing the many issues, and sufficiently narrowing the scope to permit the presentation of a design

Finally, it must be noted that the robust policies that we present are not intended to be viewed as final policies to be implemented. Rather, they should be viewed as policy targets -

When managers can't learn fast enough, yet still must make decisions, how should they proceed?

**Stochastic Optimization.** A common approach to such policy<sup>1</sup> problems is stochastic optimization. Examples include studies of the performance of management instruments in the face of a single source of specific uncertainty such as in the size of the resource stock (Clark, Kirkwood; Koenig, 1984), the number of new recruits (Ludwig, Walters; Weitzman, 2002), or price (Andersen, 1982). Unfortunately, because they require assigning probabilities to possible outcomes, the insights from stochastic optimization techniques can be somewhat restricted. As Weitzman puts it, "The most we can hope to accomplish with such an approach is to develop a better intuition about the direction of the pure effect of the single extra feature being added...when the rest of the model is isolated away from all other forms of fisheries uncertainty" (Weitzman, 2002, p. 330). Such models generate interesting insights regarding how uncertain resources *should* be managed, but they contribute little to improving actual resource management practice. In our presentation, we attempt to provide some guidance through the development and application of a set of tools for practical (robust) policy implementation decisions in situations with multiple sources of uncertainty. While our approach is fundamentally deterministic, we show how probabilistic information can be accommodated within our framework.

**Literature Survey.** Several different threads concerning practical policy implementation challenges have emerged in the literature. Adaptive management (Walters, 1986) and resilience-based management (Holling, Gunderson; 1986; 1973; Ludwig et al., 1997) are examples from ecology. In parallel, robust control ideas from engineering (Zhou, Doyle) have begun to permeate macroeconomics (Hansen, Sargent; Kendrick, 2005) and there is recent work on resource management problems in the engineering literature (Belmiloudi, 2006; 2005; Dercole et al., 2003). A concept of robust optimization has also been developed in the operations research and management science literature (Ben-Tal, Nemirovski; Ben-Tal et al., 2000; Ben-Tal, Nemirovski) with some specific applications of these ideas to environmental problems (Babonneu et al., 2010; Lempert et al., 2006; 2000). The overarching theme of robust optimization is to select the best solution from those "immunized" against data uncertainty, i.e. solutions that remain feasible for all realizations of the data (Ben-Tal, Nemirovski).

**Our Approach: Exploiting Concepts from Robust Control.** This chapter presents a sensitivity-based robustness-vulnerability framework for the study of policy implementation in highly uncertain natural resource systems in which uncertainty is characterized by parameter bounds (not probability distributions). This approach is motivated by the fact that probability distributions are often difficult to obtain. Despite this, it is shown how one might exploit distributions for uncertain model parameters within the presented framework. The framework is applied to parametric uncertainty in the classic Gordon-Schaefer fishery model to illustrate how performance (income) can be sacrificed (traded-off) for reduced sensitivity, and hence increased robustness, with respect to model parameter uncertainty. Our robustness-vulnerability approach provides tools to systematically compare policy uncertainty-performance properties so that policy options can be systematically discussed.

More specifically, within this chapter, we exploit concepts from robust control in order to analyze the classic Gordon-Schaefer fishery model (Clark, 1990). Classic maximization of net present revenue is shown to result in an optimal control law that exhibits limit

<sup>1</sup> We use the terms "policies" and "control laws" interchangeably in this presentation.

2 Will-be-set-by-IN-TECH

When managers can't learn fast enough, yet still must make decisions, how should they

**Stochastic Optimization.** A common approach to such policy<sup>1</sup> problems is stochastic optimization. Examples include studies of the performance of management instruments in the face of a single source of specific uncertainty such as in the size of the resource stock (Clark, Kirkwood; Koenig, 1984), the number of new recruits (Ludwig, Walters; Weitzman, 2002), or price (Andersen, 1982). Unfortunately, because they require assigning probabilities to possible outcomes, the insights from stochastic optimization techniques can be somewhat restricted. As Weitzman puts it, "The most we can hope to accomplish with such an approach is to develop a better intuition about the direction of the pure effect of the single extra feature being added...when the rest of the model is isolated away from all other forms of fisheries uncertainty" (Weitzman, 2002, p. 330). Such models generate interesting insights regarding how uncertain resources *should* be managed, but they contribute little to improving actual resource management practice. In our presentation, we attempt to provide some guidance through the development and application of a set of tools for practical (robust) policy implementation decisions in situations with multiple sources of uncertainty. While our approach is fundamentally deterministic, we show how probabilistic information can be

**Literature Survey.** Several different threads concerning practical policy implementation challenges have emerged in the literature. Adaptive management (Walters, 1986) and resilience-based management (Holling, Gunderson; 1986; 1973; Ludwig et al., 1997) are examples from ecology. In parallel, robust control ideas from engineering (Zhou, Doyle) have begun to permeate macroeconomics (Hansen, Sargent; Kendrick, 2005) and there is recent work on resource management problems in the engineering literature (Belmiloudi, 2006; 2005; Dercole et al., 2003). A concept of robust optimization has also been developed in the operations research and management science literature (Ben-Tal, Nemirovski; Ben-Tal et al., 2000; Ben-Tal, Nemirovski) with some specific applications of these ideas to environmental problems (Babonneu et al., 2010; Lempert et al., 2006; 2000). The overarching theme of robust optimization is to select the best solution from those "immunized" against data uncertainty, i.e. solutions that remain feasible for all realizations of the data (Ben-Tal, Nemirovski). **Our Approach: Exploiting Concepts from Robust Control.** This chapter presents a sensitivity-based robustness-vulnerability framework for the study of policy implementation in highly uncertain natural resource systems in which uncertainty is characterized by parameter bounds (not probability distributions). This approach is motivated by the fact that probability distributions are often difficult to obtain. Despite this, it is shown how one might exploit distributions for uncertain model parameters within the presented framework. The framework is applied to parametric uncertainty in the classic Gordon-Schaefer fishery model to illustrate how performance (income) can be sacrificed (traded-off) for reduced sensitivity, and hence increased robustness, with respect to model parameter uncertainty. Our robustness-vulnerability approach provides tools to systematically compare policy uncertainty-performance properties so that policy options can be systematically discussed. More specifically, within this chapter, we exploit concepts from robust control in order to analyze the classic Gordon-Schaefer fishery model (Clark, 1990). Classic maximization of net present revenue is shown to result in an optimal control law that exhibits limit

<sup>1</sup> We use the terms "policies" and "control laws" interchangeably in this presentation.

proceed?

accommodated within our framework.

cycle behavior (nonlinear oscillations) when parametric uncertainty is present. As such, it cannot be implemented in practice (because of prohibitively expensive switching costs). This motivates the need for robust policies that (1) do not exhibit limit cycle behavior and (2) offer performance (returns) as close to the optimal perfect information policy as model parameter (and derived fishery biomass target) uncertainty permits. Given the state of most world fisheries, our presentation focuses on a fishery that is nominally (i.e. believed to be) biologically over exploited (BOE); i.e. the optimal equilibrium biomass lies below the maximum sustainable yield biomass (Clark, 2006; Clark et al., 2006; Clark, 1990; Holland, Gudmundsson; Myers, Worm 2003; Sethi et al., 2005). By so doing, we directly address a globally critical renewable resource management problem. As in our prior work (Anderies et al., 2007), (Rodriguez et al., 2010), we do not seek "a best policy." Instead, we seek families of policies that are robust with respect to uncertainties that are likely to occur. Such families can, in principle, be used by a fishery manager to navigate the many tradeoffs (biological, ecological, social, economic, political) that must be confronted. More specifically, our effort to seek robust performance focuses on reducing the worst case downside performance; i.e. maximizing returns when we have the worst case combination of parameters. Such worst case (conservative) planning is critical to avoid/minimize the possibility of major regional/societal economical shortfalls; case in point, the recent "Great Recession." It is important to note that the simplicity of our model (vis-a-vis our performance objective of maximizing the net present value of returns) permits us to readily determine the worst case combination of model parameters (i.e. growth rate, carrying capacity, catchability, discount rate, price, cost of harvesting). Given this, we seek control laws that do not exhibit limit cycle behavior and whose returns are close (modulo limitations imposed by uncertainty) to that of the worst case perfect information optimal control policy - the best we could do in terms of return if we knew the parameters perfectly. Other design strategies are also examined; e.g. designing for the best case set of parameters. "Blended strategies" that attempt to do well for the worst case downside perturbation (i.e. minimize the economic downside) as well as the best case upside perturbation (i.e. maximize the economic upside) are also discussed. Such strategies seek to flatten the return-uncertainty characteristics over a broad range of likely parameters. The above optimal control (derived) policies are used as performance benchmarks/targets for the development of robust control policies. While our focus is on bounded deterministic parametric uncertainty, we also show how probability distributions for uncertain model parameters can be exploited to help in the selection of benchmark (optimal) policies. After targeting a suitable optimal (benchmark) policy, we show how robust policies can be used to approximate the benchmark (as closely as the uncertainty will permit) in order to achieve desired performance-robustness-vulnerability tradeoffs; e.g. have a return that is robust to worst case parameter perturbations.

While the presentation is intended to provide an introduction into how concepts from optimal and robust control may be used to address critical issues associated with renewable resource management, the presentation also attempts to shed light on challenges for the controls community. Although the presentation builds on the prior work presented in (Anderies et al., 2007), (Rodriguez et al., 2010), the focus here is more on defining the problem, describing the many issues, and sufficiently narrowing the scope to permit the presentation of a design methodology (framework) for robust control policies.

Finally, it must be noted that the robust policies that we present are not intended to be viewed as final policies to be implemented. Rather, they should be viewed as policy targets -

where

function

expected).

**2.3 LTI small signal model**

present value of future returns:

*<sup>F</sup>*(*x*) = *rx*

sat(*x*; *<sup>x</sup>*min, *<sup>x</sup>*max) def

Application to the Classic Gordon-Schaefer Fishery Model

The feasibility condition can then be written in terms of (3), i.e.

*<sup>J</sup>*(*up*) def

constant. (We will use *T* = ∞ to develop the optimal control law.)

**2.2 Equilibrium analysis of bioeconomic model**

= *<sup>T</sup>*

(set) points. Given this, the set of equilibria for the nonlinear model are as follows:

*when ue* ∈ (0, 1] *xe* = *k*

=

where *u* is the control signal. When there is no risk of confusion, we will write *up* = sat(*u*). **Performance Measure.** The fishery performance measure to be used, denoted *J*, is the net

where price *p*, cost per unit effort *c*, discount rate *δ*, and planning horizon *T* are assumed

One of the desired control objectives will be for the fishery to operate at specific equilibrium

Observe that as the equilibrium effort increases, the equilibrium biomass decreases (as

To further understand the local characteristics of the above nonlinear model, we can linearize it about equilibria. Doing so yields the following small signal linear time invariant (LTI)

<sup>1</sup> <sup>−</sup> *<sup>x</sup> k* 

*<sup>x</sup>*min <sup>−</sup><sup>∞</sup> <sup>&</sup>lt; *<sup>x</sup>* <sup>&</sup>lt; *<sup>x</sup>*min

*up* ∈ [0, *u*max] ⇔ *up* = sat(*u*; 0, *umax*) (4)

*xe* = *o ue* = 0 (6)

 <sup>1</sup> <sup>−</sup> *<sup>q</sup> r ue* 

. (7)

*x x*min <sup>≤</sup> *<sup>x</sup>* <sup>≤</sup> *<sup>x</sup>*max *xmax <sup>x</sup>*max <sup>&</sup>lt; *<sup>x</sup>* <sup>&</sup>lt; <sup>∞</sup>. (3)

<sup>0</sup> *<sup>e</sup>*−*δτ*(*pqx* <sup>−</sup> *<sup>c</sup>*)*up <sup>d</sup><sup>τ</sup>* (5)

represents the natural regeneration rate of the resource and *x*, *xo*, and *up* represent resource biomass, initial resource biomass, and harvesting effort, respectively. The parameters *r*, *k*, and *q*, retain their traditional definitions of intrinsic growth rate, carrying capacity, and catchability, respectively. Table 1 in Section 2.5 summarizes model parameter definitions, units, nominal values, and ranges. Model uncertainty will be addressed in Section 2.6. **Saturating Nonlinearity.** Typically, effort is bounded above by some maximum and below by zero, i.e. *up* ∈ [0, *u*max]. Typically, this physical constraint is implicitly taken into account when the optimal control problem is solved. However, a more general family of controls may generate control signals outside the allowable range, and it is important to be explicit about how these signals are "clipped" by physical constraints. We thus define the saturation

<sup>419</sup> Design of Robust Policies for Uncertain Natural Resource Systems:

(2)

providing guidance to resource managers for the development of final implementable policies (based on taxes, quotas, etc. (Clark, 1990, Chapter 8)) that will (in some sense) approximate our robust policies. While our focus has been on parametric uncertainty, it must be noted that robustness to unmodeled dynamics (e.g. lags, time delays) is also important. While some discussion on this is provided, this will be examined in future work.

**Contributions of Work.** The main contributions of this chapter are as follows:


**Organization of Chapter.** The remainder of the chapter is organized as follows.


## **2. Nonlinear bioeconomic model**

In this section, we describe the nonlinear bioeconomic model to be used for control design. The model is then analyzed.

#### **2.1 Description of bioeconomic model**

The nonlinear Gordon-Schaefer bioeconomic model (Clark, 1990; Gordon, 1954; Schaefer, 1957) is now described.

#### **Nonlinear Gordon-Schaefer Bioeconomic Model.**

The nonlinear model to be used is as follows:

$$\dot{\mathbf{x}} = F(\mathbf{x}) - q\mathbf{x}u\_p \qquad\qquad \mathbf{x}(0) = \mathbf{x}\_{0\prime} \tag{1}$$

where

4 Will-be-set-by-IN-TECH

providing guidance to resource managers for the development of final implementable policies (based on taxes, quotas, etc. (Clark, 1990, Chapter 8)) that will (in some sense) approximate our robust policies. While our focus has been on parametric uncertainty, it must be noted that robustness to unmodeled dynamics (e.g. lags, time delays) is also important. While some

• *Benefits of Robust Control in Renewable Resource Management.* The chapter shows how robust control laws can be used to eliminate the limit cycle behavior of the optimal control law while increasing robustness to parametric uncertainty and achieving a return that is close (modulo limitations imposed by uncertainty) to the perfect information optimal control law. Special attention is paid to minimizing worst case economic downside. As such, the policies presented shed light on fundamental performance limitations in the presence of (parametric) uncertainty. The policies presented are intended to serve as targets/guidelines that fishery managers may try to approximate using available tools

• *Tutorial/Introductory Value.* The chapter serves as an introduction for the controls community to a very important resource management problem in the area of global sustainability. As such, the chapter offers a myriad of challenging problems for the controls

• Section 2 describes the classic Gordon-Schaefer nonlinear fishery model (Clark, 1990) to be

• Section 3 describes the optimal control law and its properties. The latter motivates the need for robust control laws for fishery management - laws that try to achieve robust near

• Section 5 contains the main results of the work - comparing the properties of the optimal

In this section, we describe the nonlinear bioeconomic model to be used for control design.

The nonlinear Gordon-Schaefer bioeconomic model (Clark, 1990; Gordon, 1954; Schaefer,

*x*˙ = *F*(*x*) − *qxup x*(0) = *xo*, (1)

• Finally, Section 6 summarizes the chapter and presents directions for future research.

**Organization of Chapter.** The remainder of the chapter is organized as follows.

• Section 4 describes a class of robust control laws to be examined.

policy to those of the robust policies being considered.

discussion on this is provided, this will be examined in future work.

(e.g. taxes, quotas, etc. (Clark, 1990, Chapter 8).

community to address in future work.

optimal performance in some sense.

**2. Nonlinear bioeconomic model**

**2.1 Description of bioeconomic model**

**Nonlinear Gordon-Schaefer Bioeconomic Model.** The nonlinear model to be used is as follows:

The model is then analyzed.

1957) is now described.

used.

**Contributions of Work.** The main contributions of this chapter are as follows:

$$F(\mathbf{x}) = r\mathbf{x}\left(1 - \frac{\mathbf{x}}{k}\right) \tag{2}$$

represents the natural regeneration rate of the resource and *x*, *xo*, and *up* represent resource biomass, initial resource biomass, and harvesting effort, respectively. The parameters *r*, *k*, and *q*, retain their traditional definitions of intrinsic growth rate, carrying capacity, and catchability, respectively. Table 1 in Section 2.5 summarizes model parameter definitions, units, nominal values, and ranges. Model uncertainty will be addressed in Section 2.6.

**Saturating Nonlinearity.** Typically, effort is bounded above by some maximum and below by zero, i.e. *up* ∈ [0, *u*max]. Typically, this physical constraint is implicitly taken into account when the optimal control problem is solved. However, a more general family of controls may generate control signals outside the allowable range, and it is important to be explicit about how these signals are "clipped" by physical constraints. We thus define the saturation function

$$\text{sat}(\mathbf{x}; \mathbf{x}\_{\text{min}}, \mathbf{x}\_{\text{max}}) \stackrel{\text{def}}{=} \begin{cases} \mathbf{x}\_{\text{min}} - \infty & < \mathbf{x} < \mathbf{x}\_{\text{min}} \\ \mathbf{x} & \mathbf{x}\_{\text{min}} \le \mathbf{x} \le \mathbf{x}\_{\text{max}} \\ \mathbf{x}\_{\text{max}} \ge \mathbf{x}\_{\text{max}} < \mathbf{x} < \mathbf{x} \end{cases} \tag{3}$$

The feasibility condition can then be written in terms of (3), i.e.

$$
\mu\_p \in [0, \mu\_{\text{max}}] \Leftrightarrow \mathfrak{u}\_p = \mathfrak{sat}(\mathfrak{u}; 0, \mathfrak{u}\_{\text{max}}) \tag{4}
$$

where *u* is the control signal. When there is no risk of confusion, we will write *up* = sat(*u*). **Performance Measure.** The fishery performance measure to be used, denoted *J*, is the net present value of future returns:

$$J(u\_p) \stackrel{\text{def}}{=} \int\_0^T e^{-\delta \tau} (pq\infty - c) u\_p \, d\tau \tag{5}$$

where price *p*, cost per unit effort *c*, discount rate *δ*, and planning horizon *T* are assumed constant. (We will use *T* = ∞ to develop the optimal control law.)

#### **2.2 Equilibrium analysis of bioeconomic model**

One of the desired control objectives will be for the fishery to operate at specific equilibrium (set) points. Given this, the set of equilibria for the nonlinear model are as follows:

$$\mathfrak{x}\_{\ell} = \mathfrak{o} \tag{6}$$

$$
\text{when} \quad \mathsf{u}\_{\mathcal{C}} \in \{0, 1\} \tag{7}
$$

$$
\mathsf{x}\_{\mathcal{C}} = k \left(1 - \frac{q}{r} \mathsf{u}\_{\mathcal{C}}\right). \tag{7}
$$

Observe that as the equilibrium effort increases, the equilibrium biomass decreases (as expected).

#### **2.3 LTI small signal model**

To further understand the local characteristics of the above nonlinear model, we can linearize it about equilibria. Doing so yields the following small signal linear time invariant (LTI)

5. Attenuate high frequency sensor noise so that it does not significantly impact control action 6. Ensure that the fishery biomass overshoot to step reference biomass commands is suitably

<sup>421</sup> Design of Robust Policies for Uncertain Natural Resource Systems:

**Symbol Description Unit Nominal Range**

*umin* Minimum harvesting effort *fleet* · *power* · *year*/*year* 0 *umax* Maximum harvesting effort *fleet* · *power* · *year*/*year* 1 *r* Intrinsic growth rate 1/*year* 0.3 [0.15,0.45] *q* Catchability 1/ *fleet* · *power* · *year* 0.3 [0.15,0.45] *k* Carrying capacity *KT* 100 [50,150]

*p* Resource market price *M\$ per kiloton* 10 [5,15] *c* Cost of harvesting per effort *M\$ per year* 13.24 [6.62, 19.86] *δ* Annual discount rate *1/year* 0.1 [0.05,0.15] *T* Planning horizon *years* 50 N/A

A planning horizon of *T* = 50 years was selected because the nominal discount rate is *δ* = 0.1

• *BOE with the 'low cost' c* = 13.24*.* BOE occurs when the cost is sufficiently small. For the

*<sup>e</sup>* <sup>=</sup> 0.75 · *<sup>x</sup>*MSY <sup>=</sup> 37.5 <sup>&</sup>lt; *<sup>x</sup>*MSY <sup>=</sup> *<sup>k</sup>*

i.e. the optimal equilibrium biomass is below the maximum sustainable yield biomass.

Within this presentation, we focus on uncertainty associated with the nominal model parameters: *r*, *k*, *q*, *p*, *c*, *δ*. The following uncertainty will not be addressed in this presentation

1. The structure of *F* may be different than considered above. For example, if *F* has the form *F*(*x*)*geq*0 for *x* ∈ [*kc*, *k*] and *F*(*x*) < 0 for *x* ∈ (0, *kc*) where *F*(0) = 0 and *F*(*k*) = 0, then we say that the fishery exhibits *critical depensation* (Clark, 1990, p. 17). In short, this implies that if *x* ever drops below the critical depensation parameter *kc* > 0, then *x* will decrease

<sup>2</sup> = 50

*<sup>δ</sup>* = 50 years, the integrand within *J* is negligible. **Focus of Work: Biologically Exploited (BOE) Fishery.** The focus of our presentation will be on a fishery that biologically overly expoilted (BOE) as opposed to biologically under exploited (BUE). This is because most of the world's critical fisheries are overly exploited

*xo* Initial resource biomass *Kilotons*, *KT* varies [0.5*xo*, 1.5*xo*]

*θ θo*

7. Robustness with respect to model parametric uncertainty

Application to the Classic Gordon-Schaefer Fishery Model

Nominal parameter values to be used are given below in Table 1.

bounded

**2.5 Nominal model parameters**

Biological Parameters

Economic Parameters

and in roughly *T* = <sup>5</sup>

(Clark, 1990).

but it is duly noted:

Table 1. Nominal Parameter Values Used

parameters indicated, it can be shown that: *x*∗

**2.6 Model uncertainty and scope of presentation**

toward zero regardless of *u*; i.e. the fishery will be lost.

model:

$$
\delta \dot{\mathbf{x}} = \mathbf{a} \,\delta \mathbf{x} + \underbrace{\mathbf{b} \,\delta \mathbf{u}}\_{\tag{8}} \tag{8}
$$

$$f(\mathbf{x}) = r\mathbf{x}\left(1 - \frac{\mathbf{x}}{k}\right) - q\mathbf{x}u \tag{9}$$

$$a = \left[\frac{\partial f}{\partial \mathbf{x}}\right]\_{\left(\mathbf{x}\_{\ell}, \boldsymbol{\mu}\_{\ell}\right)} = r - \frac{2r\mathbf{x}\_{\ell}}{k} - q\boldsymbol{\mu}\_{\ell} = -\left(\frac{r}{k}\right)\mathbf{x}\_{\ell} \qquad \qquad b = \left[\frac{\partial f}{\partial \boldsymbol{\mu}}\right]\_{\left(\mathbf{x}\_{\ell}, \boldsymbol{\mu}\_{\ell}\right)} = -q\mathbf{x}\_{\ell} \tag{10}$$

$$\delta \mathbf{x}(t) = \mathbf{x}(t) - \mathbf{x}\_{\ell} \qquad \delta \mathbf{x}(0) = \mathbf{x}(0) - \mathbf{x}\_{\ell} = \mathbf{x}\_{0} - \mathbf{x}\_{\ell} \qquad \delta u(t) = u(t) - u\_{\ell} \tag{11}$$

The associated transfer function from *δu* to *δx* is given by:

$$P(s) = \frac{b}{s - a} \tag{12}$$

Since *<sup>a</sup>* <sup>=</sup> <sup>−</sup> *<sup>r</sup> k xe* < 0, it follows that the equilibrium point (*xe*, *ue*) is asymptotically stable with the rate of convergence (pole) being proportional to the equilibrium biomass *xe*, the fishery growth rate *r*, and inversely proportional to the fishery's carrying capacity *k*. The dc gain associated with *<sup>P</sup>* is *<sup>P</sup>*(0) = <sup>−</sup> *kq <sup>r</sup>* ; the minus sign implying that fishing reduces the equilibrium biomass.

*Utility of LTI Small Signal Model.* The above LTI model can be used to approximate the response *x* of the nonlinear model. If the response of the LTI model is denoted

$$
\hat{\mathfrak{x}} = \mathfrak{x}\_{\ell} + \delta \mathfrak{x} \tag{13}
$$

then *x*ˆ ≈ *x* when *u* ≈ *ue* (i.e. *δu*(*t*) ≈ 0) and *xo* ≈ *xe* (i.e. *δx*(0) = *xo* − *xe* ≈ 0).

#### **2.4 Control objectives**

The control objectives for the fishery may be summarized (roughly) as follows:

1. Maximize the net present value of future returns

$$\text{maximize } f \stackrel{\text{def}}{=} \int\_0^\infty e^{-\delta t} (pq\mathbf{x} - \mathbf{c})\mu\_P \, dt \tag{14}$$

Note: We would be willing to give up some return for increased robustness.

	- (a) Limit cycle behavior is not acceptable because it can have an prohibitively expensive implementation cost. While this is not captured in *J*, it could be addressed by introducing an additional *u*˙ *p* term within *J*.
	- (b) Closed loop responses should be "relatively smooth" (continuous) when we have nearly continuous sampling of the biomass *x*. It is understood that sampling is inevitable in practice; i.e. continuous sampling is prohibitively expensive and hence impossible. As such, closed loop responses should be robust with respect to some discrete sampling.

## **2.5 Nominal model parameters**

6 Will-be-set-by-IN-TECH

*<sup>k</sup>* <sup>−</sup> *que* <sup>=</sup> <sup>−</sup>

*<sup>P</sup>*(*s*) = *<sup>b</sup>*

with the rate of convergence (pole) being proportional to the equilibrium biomass *xe*, the fishery growth rate *r*, and inversely proportional to the fishery's carrying capacity *k*. The

*Utility of LTI Small Signal Model.* The above LTI model can be used to approximate the response

*δx*˙ = *a δx* + *b δu* (8)

 *r k* 

*δx*(*t*) = *x*(*t*) − *xe δx*(0) = *x*(0) − *xe* = *xo* − *xe δu*(*t*) = *u*(*t*) − *ue* (11)

*xe* < 0, it follows that the equilibrium point (*xe*, *ue*) is asymptotically stable

− *qxu* (9)

*xe b* =

 *∂ f ∂u* 

*<sup>s</sup>* <sup>−</sup> *<sup>a</sup>* (12)

*<sup>r</sup>* ; the minus sign implying that fishing reduces the

*x*ˆ = *xe* + *δx* (13)

(*pqx* − *c*)*up dt* (14)

(*xe*,*ue*)

= −*qxe* (10)

model:

*f*(*x*) = *rx*

*a* = *∂ f ∂x* 

> *k*

dc gain associated with *<sup>P</sup>* is *<sup>P</sup>*(0) = <sup>−</sup> *kq*

1. Maximize the net present value of future returns

introducing an additional *u*˙ *p* term within *J*.

Since *<sup>a</sup>* <sup>=</sup> <sup>−</sup> *<sup>r</sup>*

equilibrium biomass.

**2.4 Control objectives**

2. Closed loop stability

discrete sampling.

state

 <sup>1</sup> <sup>−</sup> *<sup>x</sup> k* 

(*xe*,*ue*)

<sup>=</sup> *<sup>r</sup>* <sup>−</sup> <sup>2</sup>*rxe*

*x* of the nonlinear model. If the response of the LTI model is denoted

then *x*ˆ ≈ *x* when *u* ≈ *ue* (i.e. *δu*(*t*) ≈ 0) and *xo* ≈ *xe* (i.e. *δx*(0) = *xo* − *xe* ≈ 0).

The control objectives for the fishery may be summarized (roughly) as follows:

= ∞ 0

Note: We would be willing to give up some return for increased robustness.

*e*−*δ<sup>t</sup>*

(a) Limit cycle behavior is not acceptable because it can have an prohibitively expensive implementation cost. While this is not captured in *J*, it could be addressed by

(b) Closed loop responses should be "relatively smooth" (continuous) when we have nearly continuous sampling of the biomass *x*. It is understood that sampling is inevitable in practice; i.e. continuous sampling is prohibitively expensive and hence impossible. As such, closed loop responses should be robust with respect to some

3. Follow (achievable) step biomass commands issued by the fishery manager in the steady

maximize *<sup>J</sup>* def

4. Reject additive step input and output disturbances in the steady state

The associated transfer function from *δu* to *δx* is given by:

Nominal parameter values to be used are given below in Table 1.


Table 1. Nominal Parameter Values Used

A planning horizon of *T* = 50 years was selected because the nominal discount rate is *δ* = 0.1 and in roughly *T* = <sup>5</sup> *<sup>δ</sup>* = 50 years, the integrand within *J* is negligible.

**Focus of Work: Biologically Exploited (BOE) Fishery.** The focus of our presentation will be on a fishery that biologically overly expoilted (BOE) as opposed to biologically under exploited (BUE). This is because most of the world's critical fisheries are overly exploited (Clark, 1990).

• *BOE with the 'low cost' c* = 13.24*.* BOE occurs when the cost is sufficiently small. For the parameters indicated, it can be shown that:

 $\mathbf{x}\_{\varepsilon}^{\*} = \mathbf{0}.\mathbf{7}$  $\mathbf{5} \cdot \mathbf{x}\_{\text{MSY}} = \mathbf{3}$  $\mathbf{7}.\mathbf{5} \prec \mathbf{x}\_{\text{MSY}} = \frac{k}{2} = \mathbf{50}$ 

i.e. the optimal equilibrium biomass is below the maximum sustainable yield biomass.

## **2.6 Model uncertainty and scope of presentation**

Within this presentation, we focus on uncertainty associated with the nominal model parameters: *r*, *k*, *q*, *p*, *c*, *δ*. The following uncertainty will not be addressed in this presentation but it is duly noted:

1. The structure of *F* may be different than considered above. For example, if *F* has the form *F*(*x*)*geq*0 for *x* ∈ [*kc*, *k*] and *F*(*x*) < 0 for *x* ∈ (0, *kc*) where *F*(0) = 0 and *F*(*k*) = 0, then we say that the fishery exhibits *critical depensation* (Clark, 1990, p. 17). In short, this implies that if *x* ever drops below the critical depensation parameter *kc* > 0, then *x* will decrease toward zero regardless of *u*; i.e. the fishery will be lost.

**Optimal Steady State Equilibrium Biomass.** When *F*(*x*) = *rx*(1 − *x*/*k*), the above equation

<sup>423</sup> Design of Robust Policies for Uncertain Natural Resource Systems:

*xMSY* <sup>=</sup> *<sup>k</sup>*

*<sup>x</sup>*<sup>∞</sup> <sup>=</sup> *<sup>c</sup>*

is the optimal equilibrium when *δ* = ∞; i.e. open-access equilibrium (Clark, 1990). The above

**Optimal Control Policy.** Define the *tracking error* as the difference between the desired

*<sup>u</sup>*(*t*) = <sup>−</sup><sup>∞</sup> when *<sup>e</sup>* <sup>&</sup>gt; <sup>0</sup> *u*∗

The saturation function is then applied to this control signal to capture the physical constraints

Below, we show that this policy (in general) exhibits limit cycle behavior in the presence of

*<sup>e</sup>* ≤ *xMSY* +

<sup>2</sup> <sup>−</sup> *xMSY <sup>δ</sup>*

*r*  <sup>+</sup> *xMSY*<sup>2</sup>

*<sup>e</sup>* depends on the three independent parameters *x*∞, *xMSY*,

*x*∞

<sup>2</sup> is the optimal *x*<sup>∗</sup>

*<sup>e</sup>* and combining (19) with (25) yields following expression for the control law:

*<sup>e</sup>* when *<sup>e</sup>* <sup>=</sup> <sup>0</sup> <sup>∞</sup> when *<sup>e</sup>* <sup>&</sup>lt; 0.

*e* :

+ 4*xMSYx*<sup>∞</sup>

<sup>2</sup> . (21)

<sup>2</sup> (22)

*pq* (23)

<sup>2</sup> (24)

. (25)

= *xref* − *x*. (26)

*<sup>e</sup>* , then set *up*(*t*) = sat(*u*<sup>∗</sup>

*<sup>e</sup>* for *δ* = 0. The associated

(27)

*<sup>e</sup>* , then set *up*(*t*) =

*e* ).

 *δ r* 

can be used to determine the optimal (steady state) equilibrium biomass *x*∗

+ *x*<sup>∞</sup>

<sup>+</sup> *xMSY*

*x*<sup>∞</sup> ≤ *x*<sup>∗</sup>

*u*∗ *e* def <sup>=</sup> *<sup>r</sup> q* <sup>1</sup> <sup>−</sup> *<sup>x</sup>*<sup>∗</sup> *e k* 

> *e* def

on the system, i.e. *up*(*t*) = sat(*u*(*t*)). This control law implies the following: • If *e* > 0, set *up*(*t*) = *umin* = 0, allow *x*(*t*) to increase until *x*(*t*) = *x*<sup>∗</sup>

*x*∗ *<sup>e</sup>* =

where

and *<sup>δ</sup>*

 *x*<sup>∞</sup>

<sup>2</sup> <sup>−</sup> *xMSY <sup>δ</sup>*

*r* 

Application to the Classic Gordon-Schaefer Fishery Model

is the maximum sustainable yield biomass and

for all *<sup>δ</sup>* <sup>∈</sup> [0, <sup>∞</sup>] where the quantity *xMSY* <sup>+</sup> *<sup>x</sup>*<sup>∞</sup>

(reference) state and the actual state, i.e.

Setting *xref* = *x*<sup>∗</sup>

sat(*u*∗ *e* ).

• If *e* = 0, set *up*(*t*) = sat(*u*∗

parameter uncertainty (see Figure 6).

optimal (steady state) equilibrium control is given by:

• If *e* < 0, set *up*(*t*) = *umax* until *x*(*t*) decreases to *x*∗

*e* ).

shows that the optimal biomass *x*∗

*<sup>r</sup>* . It can be shown that


In contrast to many control applications where the "controller" is implemented with great fidelity, fishery controllers are implemented by an organization. As such, there can be considerable implementation issues/uncertainty. This will be discussed further below.

#### **3. Optimal control law and properties**

Within this section, we present the optimal control problem, the associated solution (optimal control law), and the properties of the optimal control law.

#### **3.1 Optimal control law**

We begin with a brief derivation of the classical optimal control policy stated in a way that will facilitate comparison to the class of LTI policies described later in this section. The solution of the traditional optimal control problem:

$$\text{maximize } f \stackrel{\text{def}}{=} \int\_0^\infty e^{-\delta t} (pqx(t) - c) u\_p(t) \, dt \tag{15}$$

$$\text{s.t. } \dot{\mathbf{x}}(t) = F(\mathbf{x}(t)) - q\mathbf{x}(t)\boldsymbol{\mu}\_p(t) \qquad \mathbf{x}(0) = \mathbf{x}\_o \tag{16}$$

$$
\mu\_{\min} = 0 \le \mu\_p(t) \le \mu\_{\max} \tag{17}
$$

is obtained by forming the Hamiltonian:

$$\mathcal{H}(\mathbf{x}, u, \lambda) \stackrel{\text{def}}{=} e^{-\delta t} (p\mathbf{q}\mathbf{x} - \mathbf{c})\mathbf{u} + \lambda \left[\mathcal{F}(\mathbf{x}) - q\mathbf{x}u\right] = \mathcal{G}(\mathbf{x}, t)\mathbf{u} - \lambda \mathcal{F}(\mathbf{x}) \tag{18}$$

where *<sup>G</sup>*(*x*,*t*) def = *e*−*δ<sup>t</sup>* (*pqx* − *c*) − *λqx* and *λ* is the co-state variable. *Pontryagin's Maximum Principle* then implies that an optimal control policy will satisfy:

$$u(t) = \begin{cases} -\infty \text{ when} & \text{G}(\mathfrak{x}, t) < 0\\ \infty & \text{when} \quad \mathrm{G}(\mathfrak{x}, t) > 0. \end{cases} \tag{19}$$

Because the objective functional is linear, the Maximum Principle says nothing about the case when *<sup>G</sup>*(*x*, *<sup>t</sup>*) = 0. However, using the co-state variable relationship *<sup>λ</sup>*˙ <sup>=</sup> <sup>−</sup> *<sup>∂</sup>*<sup>H</sup> *<sup>∂</sup><sup>x</sup>* , the well-known implicit formula for the singular control path can be determined (Clark, 1990):

$$F'(\mathbf{x}) + \frac{cF(\mathbf{x})}{\mathbf{x}(pq\mathbf{x} - \mathbf{c})} = \delta \tag{20}$$

**Optimal Steady State Equilibrium Biomass.** When *F*(*x*) = *rx*(1 − *x*/*k*), the above equation can be used to determine the optimal (steady state) equilibrium biomass *x*∗ *e* :

$$\mathbf{x}\_{\varepsilon}^{\*} = \frac{\left[\frac{\mathbf{x}\_{\infty}}{2} - \mathbf{x}\_{\mathrm{MSY}}\left(\frac{\delta}{r}\right) + \mathbf{x}\_{\mathrm{MSY}}\right] + \sqrt{\left[\frac{\mathbf{x}\_{\infty}}{2} - \mathbf{x}\_{\mathrm{MSY}}\left(\frac{\delta}{r}\right) + \mathbf{x}\_{\mathrm{MSY}}\right]^2 + 4\mathbf{x}\_{\mathrm{MSY}}\mathbf{x}\_{\mathrm{os}}\left(\frac{\delta}{r}\right)}{2}. \tag{21}$$

where

8 Will-be-set-by-IN-TECH

2. All plant parameters are uncertain. They may even change with time. Moreover, the plant contains additional dynamics; e.g. it takes time for the fishery workers to mobilize. This can contribute additional lags, time delays, and rate limiters within the plant. One can use a decentralized or distributed model in order to capture the decision making made by

6. The output (biomass) is sampled at some rate; if this rate is not sufficiently high, it could cause aliasing (Ogata, 1995); the sampling rate should be (as a rule-of-thumb)greater than

In contrast to many control applications where the "controller" is implemented with great fidelity, fishery controllers are implemented by an organization. As such, there can be considerable implementation issues/uncertainty. This will be discussed further below.

Within this section, we present the optimal control problem, the associated solution (optimal

We begin with a brief derivation of the classical optimal control policy stated in a way that

(*pqx*(*t*) − *c*)*up*(*t*) *dt* (15)

s.t. *x*˙(*t*) = *F*(*x*(*t*)) − *qx*(*t*)*up*(*t*) *x*(0) = *xo* (16) *umin* = 0 ≤ *up*(*t*) ≤ *umax* (17)

(*pqx* − *c*)*u* + *λ* [*F*(*x*) − *qxu*] = *G*(*x*,*t*)*u* − *λF*(*x*) (18)

<sup>∞</sup> when *<sup>G</sup>*(*x*, *<sup>t</sup>*) <sup>&</sup>gt; 0. (19)

*<sup>x</sup>*(*pqx* <sup>−</sup> *<sup>c</sup>*) <sup>=</sup> *<sup>δ</sup>* (20)

*<sup>∂</sup><sup>x</sup>* , the

(*pqx* − *c*) − *λqx* and *λ* is the co-state variable. *Pontryagin's Maximum*

<sup>−</sup><sup>∞</sup> when *<sup>G</sup>*(*x*, *<sup>t</sup>*) <sup>&</sup>lt; <sup>0</sup>

Because the objective functional is linear, the Maximum Principle says nothing about the case when *<sup>G</sup>*(*x*, *<sup>t</sup>*) = 0. However, using the co-state variable relationship *<sup>λ</sup>*˙ <sup>=</sup> <sup>−</sup> *<sup>∂</sup>*<sup>H</sup>

well-known implicit formula for the singular control path can be determined (Clark, 1990):

(*x*) + *cF*(*x*)

will facilitate comparison to the class of LTI policies described later in this section.

*e*−*δ<sup>t</sup>*

individual fisher people (Clark, 1990, Ch. 8 & 9).

3. Input and output disturbances are uncertain.

5. The biomass is not known; it must be estimated

ten times the control system bandwidth.

**3. Optimal control law and properties**

**3.1 Optimal control law**

where *<sup>G</sup>*(*x*,*t*) def

control law), and the properties of the optimal control law.

The solution of the traditional optimal control problem:

= *e*−*δ<sup>t</sup>*

*Principle* then implies that an optimal control policy will satisfy:

*u*(*t*) =

*F*�

= ∞ 0

maximize *J* def

is obtained by forming the Hamiltonian:

<sup>H</sup>(*x*, *<sup>u</sup>*, *<sup>λ</sup>*) def

= *e*−*δ<sup>t</sup>*

4. Measurement noise is uncertain.

$$
\alpha\_{MSY} = \frac{k}{2} \tag{22}
$$

is the maximum sustainable yield biomass and

$$\mathbf{x}\_{\infty} = \frac{c}{pq} \tag{23}$$

is the optimal equilibrium when *δ* = ∞; i.e. open-access equilibrium (Clark, 1990). The above shows that the optimal biomass *x*∗ *<sup>e</sup>* depends on the three independent parameters *x*∞, *xMSY*, and *<sup>δ</sup> <sup>r</sup>* . It can be shown that

$$\begin{array}{ccccccccc}\chi\_{\infty} & \leq & \mathfrak{x}\_{\varepsilon}^{\*} & \leq & \mathfrak{x}\_{\text{MSY}} & + & \frac{\mathfrak{x}\_{\infty}}{2} \end{array} \tag{24}$$

for all *<sup>δ</sup>* <sup>∈</sup> [0, <sup>∞</sup>] where the quantity *xMSY* <sup>+</sup> *<sup>x</sup>*<sup>∞</sup> <sup>2</sup> is the optimal *x*<sup>∗</sup> *<sup>e</sup>* for *δ* = 0. The associated optimal (steady state) equilibrium control is given by:

$$
\mu\_{\varepsilon}^{\*} \stackrel{\text{def}}{=} \frac{r}{q} \left(1 - \frac{\chi\_{\varepsilon}^{\*}}{k}\right). \tag{25}
$$

**Optimal Control Policy.** Define the *tracking error* as the difference between the desired (reference) state and the actual state, i.e.

$$e \stackrel{\text{def}}{=} \mathfrak{x}\_{ref} - \mathfrak{x}.\tag{26}$$

Setting *xref* = *x*<sup>∗</sup> *<sup>e</sup>* and combining (19) with (25) yields following expression for the control law:

$$u(t) = \begin{cases} -\infty \text{ when} & e > 0\\ u\_{\varepsilon}^{\*} & \text{when} \quad e = 0\\ \infty & \text{when} \quad e < 0. \end{cases} \tag{27}$$

The saturation function is then applied to this control signal to capture the physical constraints on the system, i.e. *up*(*t*) = sat(*u*(*t*)). This control law implies the following:


Below, we show that this policy (in general) exhibits limit cycle behavior in the presence of parameter uncertainty (see Figure 6).

5. (*J*∗ *<sup>e</sup>* , *x*<sup>∗</sup>

*x*∗

*x*∗

uncertainty. *xo* = *x*∗

*p*, *c*, or *q*? This is because *x*∗

**Equilibrium.** In what follows, *x*∗

this, suppose that *xo* = *x*∗

*J*∗

very difficult to do.

*<sup>e</sup>* ) decrease while *u*<sup>∗</sup>

Application to the Classic Gordon-Schaefer Fishery Model

*<sup>e</sup>* is always uncertain. This is particularly crucial when *x*<sup>∗</sup>

and exciting to pursue, it is beyond the scope of our presentation.

**Impact of Extremal Parameter Uncertainty on Perfect Information (***J*∗

control law with good command following properties. Since *x*∗

*e* .

*<sup>e</sup>* increases with increasing *c*.

**Robustness with Respect to Parametric Uncertainty: Imperfect Versus Perfect Information.** Figure 1 shows how the optimal control law performs in the presence of parametric

<sup>425</sup> Design of Robust Policies for Uncertain Natural Resource Systems:

*<sup>e</sup>* for the imperfect information policy. The plots compare the performance of the optimal control law with imperfect parameter knowledge to that with perfect parameter knowledge. The perfect information optimal control law (by definition) results in the maximum achievable return. While it represents a suitable benchmark to compare with, it must be emphasized that

(reference command) for a robust control law (see Sections 4, 5) because an incorrect reference command *xref* will fundamentally limit the achievable performance. Moreover, no (inner loop) robust policy can address this. To properly address this, one needs some combination of parameter estimation, system identification, and learning coupled with some adaptive outer loop policy that adjusts the target based on collected information. While this is challenging

Figure 1 specifically shows the maximum theoretical (perfect information) return on the left in blue. The return associated with the imperfect information optimal policy (designed for nominal parameter values) is shown on the left in red. On the right in blue, we see how much the imperfect information optimal control law under performs the perfect information optimal control law. When *k* is perturbed by −30%, the imperfect law under performs the perfect information law by nearly 10%. Figure 1 shows that for a similar perturbation in *r*, the imperfect policy under performs by nearly 2%. It can be shown (figures not provided) that for a similar perturbation in *δ*, the imperfect policy under performs by less than 1%. It can be shown (figures not provided) that for similar perturbations in *p*, *c*, or *q* the imperfect policy under performs by a very small percentage. Why is it that the biological parameters *k* and *r* matter more in closing the perfect-imperfect information performance gap than *δ*,

under consideration. In short, the plots show that we should be concerned primarily with uncertainty in *k*. More generally, we seek (robust) policies that perform closer to the perfect information optimal policy for likely parametric modeling errors. Imperfect information obviously limits how close we can get. This and associated issues will be addressed below.

to understand how commanding an incorrect target will limit achievable performance. Given

We now ask, what is the worst case combination of perturbations for the model parameters (*r*,

*<sup>e</sup>* , in general, decreases when (*k*, *r*, *p*, *q*) are decreased and/or (*c*, *δ*) are increased. This result is independent of the initial condition *xo* for the BOE case under consideration. Given uncertainty bounds for each of the model parameters, this observation permits us to readily determine the worst case set of parameter perturbation - something that, in general, is

*k*, *q*, *p*, *c*, *δ*)? While an analytical proof is difficult, it can be shown (numerically) that

*<sup>e</sup>* for the perfect information policy. *xo* is at the unperturbed/nominal

*<sup>e</sup>* is more sensitive to uncertainty in *k* and *r* for the BOE case

*<sup>e</sup>* will be used as a reference command *xref* to a robust

*<sup>e</sup>* **,** *x*<sup>∗</sup> *<sup>e</sup>* **,** *u*<sup>∗</sup>

*<sup>e</sup>* is uncertain, it is important

*<sup>e</sup>* **) - At Optimal**

*<sup>e</sup>* is being used as the target biomass

#### **3.2 Nominal optimal control policy numerics**

The numerics for the nominal optimal perfect information control law are summarized in Table 2.


Table 2. Summary of Optimal Control Policy Numerics. BOE corresponds to *c* = 13.24 which yields *x*∗ *<sup>e</sup>* = 0.75 · *x*MSY. Small IC corresponds to *xo* = 0.5 · *x*<sup>∗</sup> *<sup>e</sup>* < *x*<sup>∗</sup> *<sup>e</sup>* . Large IC corresponds to *xo* = 1.5 · *x*<sup>∗</sup> *<sup>e</sup>* > *x*<sup>∗</sup> *<sup>e</sup>* . *<sup>x</sup>*<sup>∞</sup> = *<sup>c</sup> pq* and *<sup>u</sup>*<sup>∞</sup> <sup>=</sup> *<sup>r</sup> <sup>q</sup>* (<sup>1</sup> <sup>−</sup> *<sup>x</sup>*<sup>∞</sup> *<sup>k</sup>* ) correspond to infinite discounting (open-access); i.e. *δ* = ∞.

#### **3.3 Properties of the optimal control law**

In this section, we describe the properties of the optimal control law assuming perfect information (i.e. model parameters are known with no error) and imperfect information (i.e. model parameters are not perfectly known). Understanding the properties of the optimal policy is very important for several reasons. We wish to understand (1) the fundamental robustness properties (e.g. economic inefficiency) of an optimal policy (e.g. one based on nominal, worst case, or best case parameters); (2) implementation issues associated with the optimal control policy; (3) how the robustness properties for our robust policies compare to those of a particular optimal control policy; (4) how *x*∗ *<sup>e</sup>* depends on parameter perturbations. The latter is important because we will using *x*∗ *<sup>e</sup>* as the reference command *xref* for our robust control law policies. This is an issue because the optimal *x*∗ *<sup>e</sup>* (in general) is uncertain; i.e. *x*<sup>∗</sup> *<sup>e</sup>* is only known for specific value selections (e.g. nominal, worst case, best case). As such, we will have to address this uncertainty to clearly understand what our robust control policies (with built-in command following) will be driving the state of the fishery to.

In short, we show below that: (1) Since *x*∗ *<sup>e</sup>* is, in general, uncertain, if *x*<sup>∗</sup> *<sup>e</sup>* is the desired (reference) state, then we have a major issue in that we will be driving the fishery to the incorrect state. This can have severe economic as well as biological implications (e.g. driving *x* below the critical depensation parameter *kc*, will destroy the fishery). (2) The optimal policy exhibits limit cycle behavior when *x*∗ *<sup>e</sup>* is uncertain. Moreover, it is very sensitive to any discrete sampling. As such, the (imperfect information) optimal policy is prohibitively expensive to implement (see Figure 6).

**Optimal Perfect Information Control Law Sensitivity: Single Parameter Results (***xo* = *x*∗ *e* **).** The following shows how the the performance of the optimal perfect information control law changes with parameter perturbations. Results for our BOE fishery when *xo* = *x*∗ *<sup>e</sup>* are as follows:


#### 5. (*J*∗ *<sup>e</sup>* , *x*<sup>∗</sup> *<sup>e</sup>* ) decrease while *u*<sup>∗</sup> *<sup>e</sup>* increases with increasing *c*.

10 Will-be-set-by-IN-TECH

The numerics for the nominal optimal perfect information control law are summarized in

Table 2. Summary of Optimal Control Policy Numerics. BOE corresponds to *c* = 13.24 which

In this section, we describe the properties of the optimal control law assuming perfect information (i.e. model parameters are known with no error) and imperfect information (i.e. model parameters are not perfectly known). Understanding the properties of the optimal policy is very important for several reasons. We wish to understand (1) the fundamental robustness properties (e.g. economic inefficiency) of an optimal policy (e.g. one based on nominal, worst case, or best case parameters); (2) implementation issues associated with the optimal control policy; (3) how the robustness properties for our robust policies compare to

only known for specific value selections (e.g. nominal, worst case, best case). As such, we will have to address this uncertainty to clearly understand what our robust control policies (with

(reference) state, then we have a major issue in that we will be driving the fishery to the incorrect state. This can have severe economic as well as biological implications (e.g. driving *x* below the critical depensation parameter *kc*, will destroy the fishery). (2) The optimal policy

sampling. As such, the (imperfect information) optimal policy is prohibitively expensive to

**Optimal Perfect Information Control Law Sensitivity: Single Parameter Results (***xo* = *x*∗

*<sup>e</sup>* ) decrease with increasing *q*.

The following shows how the the performance of the optimal perfect information control law changes with parameter perturbations. Results for our BOE fishery when *xo* = *x*∗

*<sup>e</sup>* decreases with increasing *δ*.

*<sup>e</sup>* increases with increasing *p*.

*<sup>q</sup>* (<sup>1</sup> <sup>−</sup> *<sup>x</sup>*<sup>∞</sup>

⎧ ⎪⎨ ⎪⎩

*<sup>e</sup>* = 0.75 · *x*MSY. Small IC corresponds to *xo* = 0.5 · *x*<sup>∗</sup>

*pq* and *<sup>u</sup>*<sup>∞</sup> <sup>=</sup> *<sup>r</sup>*

*u*∗ *e* def = *<sup>F</sup>*(*x*<sup>∗</sup> *e* ) *qx*∗

*<sup>e</sup> x*<sup>∞</sup> *u*<sup>∞</sup> **Optimal Control Law** Optimal Return, *J*<sup>∗</sup>

0 *x*(*t*) < 37.5

*umax* = 1 *x*(*t*) > 37.5

*<sup>e</sup>* = 0.625 *<sup>x</sup>*(*t*) = 37.5

*<sup>e</sup>* < *x*<sup>∗</sup>

*<sup>k</sup>* ) correspond to infinite discounting

*o*

*<sup>e</sup>* is

*e* **).**

*<sup>e</sup>* are as

*<sup>e</sup>* is the desired

small IC large IC

451 782

*<sup>e</sup>* . Large IC corresponds to

*<sup>e</sup>* depends on parameter perturbations.

*<sup>e</sup>* (in general) is uncertain; i.e. *x*<sup>∗</sup>

*<sup>e</sup>* as the reference command *xref* for our robust

*<sup>e</sup>* is, in general, uncertain, if *x*<sup>∗</sup>

*<sup>e</sup>* is uncertain. Moreover, it is very sensitive to any discrete

**3.2 Nominal optimal control policy numerics**

BOE 37.5 0.625 4.41 0.987 *u*(*t*) =

*<sup>e</sup>* . *<sup>x</sup>*<sup>∞</sup> = *<sup>c</sup>*

**3.3 Properties of the optimal control law**

those of a particular optimal control policy; (4) how *x*∗

control law policies. This is an issue because the optimal *x*∗

built-in command following) will be driving the state of the fishery to.

*<sup>e</sup>* ) increase with increasing *k* or increasing *r*.

*<sup>e</sup>* increases and *u*<sup>∗</sup>

*<sup>e</sup>* decreases and *u*<sup>∗</sup>

*<sup>e</sup>* , *u*<sup>∗</sup>

The latter is important because we will using *x*∗

In short, we show below that: (1) Since *x*∗

exhibits limit cycle behavior when *x*∗

implement (see Figure 6).

*<sup>e</sup>* increases while (*x*<sup>∗</sup>

*<sup>e</sup>* decreases while *x*<sup>∗</sup>

*<sup>e</sup>* increases while *x*<sup>∗</sup>

follows: 1. (*J*∗ *<sup>e</sup>* , *x*<sup>∗</sup> *<sup>e</sup>* , *u*<sup>∗</sup>

2. *J*∗

3. *J*∗

4. *J*∗

*<sup>e</sup>* > *x*<sup>∗</sup>

(open-access); i.e. *δ* = ∞.

Table 2. **Case** *x*∗

yields *x*∗

*xo* = 1.5 · *x*<sup>∗</sup>

*<sup>e</sup> u*<sup>∗</sup>

**Robustness with Respect to Parametric Uncertainty: Imperfect Versus Perfect Information.** Figure 1 shows how the optimal control law performs in the presence of parametric uncertainty. *xo* = *x*∗ *<sup>e</sup>* for the perfect information policy. *xo* is at the unperturbed/nominal *x*∗ *<sup>e</sup>* for the imperfect information policy. The plots compare the performance of the optimal control law with imperfect parameter knowledge to that with perfect parameter knowledge. The perfect information optimal control law (by definition) results in the maximum achievable return. While it represents a suitable benchmark to compare with, it must be emphasized that *x*∗ *<sup>e</sup>* is always uncertain. This is particularly crucial when *x*<sup>∗</sup> *<sup>e</sup>* is being used as the target biomass (reference command) for a robust control law (see Sections 4, 5) because an incorrect reference command *xref* will fundamentally limit the achievable performance. Moreover, no (inner loop) robust policy can address this. To properly address this, one needs some combination of parameter estimation, system identification, and learning coupled with some adaptive outer loop policy that adjusts the target based on collected information. While this is challenging and exciting to pursue, it is beyond the scope of our presentation.

Figure 1 specifically shows the maximum theoretical (perfect information) return on the left in blue. The return associated with the imperfect information optimal policy (designed for nominal parameter values) is shown on the left in red. On the right in blue, we see how much the imperfect information optimal control law under performs the perfect information optimal control law. When *k* is perturbed by −30%, the imperfect law under performs the perfect information law by nearly 10%. Figure 1 shows that for a similar perturbation in *r*, the imperfect policy under performs by nearly 2%. It can be shown (figures not provided) that for a similar perturbation in *δ*, the imperfect policy under performs by less than 1%. It can be shown (figures not provided) that for similar perturbations in *p*, *c*, or *q* the imperfect policy under performs by a very small percentage. Why is it that the biological parameters *k* and *r* matter more in closing the perfect-imperfect information performance gap than *δ*, *p*, *c*, or *q*? This is because *x*∗ *<sup>e</sup>* is more sensitive to uncertainty in *k* and *r* for the BOE case under consideration. In short, the plots show that we should be concerned primarily with uncertainty in *k*. More generally, we seek (robust) policies that perform closer to the perfect information optimal policy for likely parametric modeling errors. Imperfect information obviously limits how close we can get. This and associated issues will be addressed below.

**Impact of Extremal Parameter Uncertainty on Perfect Information (***J*∗ *<sup>e</sup>* **,** *x*<sup>∗</sup> *<sup>e</sup>* **,** *u*<sup>∗</sup> *<sup>e</sup>* **) - At Optimal Equilibrium.** In what follows, *x*∗ *<sup>e</sup>* will be used as a reference command *xref* to a robust control law with good command following properties. Since *x*∗ *<sup>e</sup>* is uncertain, it is important to understand how commanding an incorrect target will limit achievable performance. Given this, suppose that *xo* = *x*∗ *e* .

We now ask, what is the worst case combination of perturbations for the model parameters (*r*, *k*, *q*, *p*, *c*, *δ*)? While an analytical proof is difficult, it can be shown (numerically) that

*J*∗ *<sup>e</sup>* , in general, decreases when (*k*, *r*, *p*, *q*) are decreased and/or (*c*, *δ*) are increased.

This result is independent of the initial condition *xo* for the BOE case under consideration. Given uncertainty bounds for each of the model parameters, this observation permits us to readily determine the worst case set of parameter perturbation - something that, in general, is very difficult to do.

The green curves within figures 2-3 represent actual optimal perfect information values. The

<sup>427</sup> Design of Robust Policies for Uncertain Natural Resource Systems:

BOE: J\* e = 620.4

−20 −15 −10 −5 0 5 10 15 20

200

0.55

0.575

u\*

e (M\$)

0.6

0.625

0.65

500

800

J\*

e (M\$)

1100

1400

1700

Parameter Perturbation (%)

Assuming ±20% uncertainty for each nominal parameter value, figure 2 shows that the worst case perfect information optimal return is \$215.6 M (65.25% below the nominal of

*<sup>e</sup>* = 620.4*M*). In contrast, the best case perfect information optimal return is \$1482M (138.95%

perfect information optimal return. Also note from figure 3 that the worst case parameter

x

e

u\*

e Perturbation (%)

−5

*<sup>e</sup>* , *u*<sup>∗</sup>

−2.5

0

2.5

5

\*

*<sup>e</sup>* = 620.4*M*) - a 687% improvement with respect to the worst case

*<sup>e</sup>* with respect to the nominal. From figure 3, we

BOE: u\* e = 0.625

−20 −10 0 10 20

Parameter Perturbation (%)

*<sup>e</sup>* ): Extremal Percent Parameter

Fig. 2. Perfect Information Optimal Control Law Returns: Extremal Percent Parameter

blue curves give the percent deviation with respect to the nominal value.

Application to the Classic Gordon-Schaefer Fishery Model

−100

Perturbations, *xo* = *x*∗

above the nominal of *J*∗

*e*

combination results in a 20% reduction in *x*∗

BOE: x\* e = 37.5

−20 −10 0 10 20

Parameter Perturbation (%)

*e*

Fig. 3. Perfect Information Optimal Control Law (*x*∗

*<sup>e</sup>* is increased by less than 1%.

−50

J\*

*J*∗

see that *u*∗

−40 −30 −20 −10 0 10 20 30 40

Perturbations, *xo* = *x*∗

x\*

e Perturbation (%)

e Perturbation (%)

0

50

100

150

Fig. 1. Economic Inefficiency for Imperfect Information (Nominal) Optimal Control Law: Capacity & Growth Rate Uncertainty

Consider figures 2-3 for (*J*∗ *<sup>e</sup>* , *x*<sup>∗</sup> *<sup>e</sup>* , *u*<sup>∗</sup> *<sup>e</sup>* ), respectively. Within these figures, *xo* = *x*<sup>∗</sup> *<sup>e</sup>* and perfect information is assumed. The figures show the dependence of the perfect information optimal control law on worst case and best case (extremal) parameter perturbations as defined below.

• *Worst Case Extremal Parameter Perturbations.* Within figures 2-3, negative (worst case extremal) parameter perturbations correspond to

$$\frac{\Delta r}{r\_o} = \frac{\Delta k}{k\_o} = \frac{\Delta q}{q\_o} = \frac{\Delta p}{p\_o} < 0 \qquad \text{and} \qquad \frac{\Delta c}{c\_o} = \frac{\Delta \delta}{\delta\_o} > 0 \tag{28}$$

i.e. equal parametric perturbations that result in a smaller return. Here, <sup>Δ</sup>*<sup>θ</sup>* def = *θ* − *θ<sup>o</sup>* represents a perturbation in the parameter *θ* with respect to the nominal parameter *θo*.

• *Best Case Extremal Parameter Perturbations.* Within figures 2-3, positive (best case extremal) parameter perturbations correspond to

$$\frac{\Delta r}{r\_0} = \frac{\Delta k}{k\_o} = \frac{\Delta q}{q\_o} = \frac{\Delta p}{p\_0} > 0 \qquad \text{and} \qquad \frac{\Delta c}{c\_0} = \frac{\Delta \delta}{\delta\_0} < 0 \tag{29}$$

i.e. equal parametric perturbations that result in a larger return.

12 Will-be-set-by-IN-TECH

−10

0 2

−10

*<sup>e</sup>* ), respectively. Within these figures, *xo* = *x*<sup>∗</sup>

*co*

*co*

<sup>=</sup> <sup>Δ</sup>*<sup>δ</sup> δo*

<sup>=</sup> <sup>Δ</sup>*<sup>δ</sup> δo*

<sup>&</sup>lt; 0 and <sup>Δ</sup>*<sup>c</sup>*

<sup>&</sup>gt; 0 and <sup>Δ</sup>*<sup>c</sup>*

−8 −6 −4 −2

%

Fig. 1. Economic Inefficiency for Imperfect Information (Nominal) Optimal Control Law:

information is assumed. The figures show the dependence of the perfect information optimal control law on worst case and best case (extremal) parameter perturbations as defined below. • *Worst Case Extremal Parameter Perturbations.* Within figures 2-3, negative (worst case

−8 −6 −4 −2

%

0 2

−30 −20 −10 0 10 20 30

−30 −20 −10 0 10 20 30

Δ r (%)

*<sup>e</sup>* and perfect

= *θ* − *θ<sup>o</sup>*

> 0 (28)

< 0 (29)

Inefficiency of Optimal Control Law

Δ k (%)

Inefficiency of Optimal Control Law

−30 −20 −10 0 10 20 30

Actual Return

Expected Return

Maximum Theoretical Return

Net Present Value

Δ k (%)

Net Present Value

−30 −20 −10 0 10 20 30

Actual Return

Expected Return

Maximum Theoretical Return

Δ r (%)

*<sup>e</sup>* , *x*<sup>∗</sup> *<sup>e</sup>* , *u*<sup>∗</sup>

extremal) parameter perturbations correspond to

<sup>=</sup> <sup>Δ</sup>*<sup>k</sup> ko*

<sup>=</sup> <sup>Δ</sup>*<sup>k</sup> ko*

<sup>=</sup> <sup>Δ</sup>*<sup>q</sup> qo*

<sup>=</sup> <sup>Δ</sup>*<sup>q</sup> qo*

i.e. equal parametric perturbations that result in a larger return.

<sup>=</sup> <sup>Δ</sup>*<sup>p</sup> po*

<sup>=</sup> <sup>Δ</sup>*<sup>p</sup> po*

i.e. equal parametric perturbations that result in a smaller return. Here, <sup>Δ</sup>*<sup>θ</sup>* def

represents a perturbation in the parameter *θ* with respect to the nominal parameter *θo*. • *Best Case Extremal Parameter Perturbations.* Within figures 2-3, positive (best case extremal)

Δ*r ro*

parameter perturbations correspond to

Δ*r ro*

Capacity & Growth Rate Uncertainty

Consider figures 2-3 for (*J*∗

300

300

J(up

)

J(up

)

The green curves within figures 2-3 represent actual optimal perfect information values. The blue curves give the percent deviation with respect to the nominal value.

Fig. 2. Perfect Information Optimal Control Law Returns: Extremal Percent Parameter Perturbations, *xo* = *x*∗ *e*

Assuming ±20% uncertainty for each nominal parameter value, figure 2 shows that the worst case perfect information optimal return is \$215.6 M (65.25% below the nominal of *J*∗ *<sup>e</sup>* = 620.4*M*). In contrast, the best case perfect information optimal return is \$1482M (138.95% above the nominal of *J*∗ *<sup>e</sup>* = 620.4*M*) - a 687% improvement with respect to the worst case perfect information optimal return. Also note from figure 3 that the worst case parameter combination results in a 20% reduction in *x*∗ *<sup>e</sup>* with respect to the nominal. From figure 3, we see that *u*∗ *<sup>e</sup>* is increased by less than 1%.

Fig. 3. Perfect Information Optimal Control Law (*x*∗ *<sup>e</sup>* , *u*<sup>∗</sup> *<sup>e</sup>* ): Extremal Percent Parameter Perturbations, *xo* = *x*∗ *e*

Now choose *xref* equal to the associated worst case *x*<sup>∗</sup>

Application to the Classic Gordon-Schaefer Fishery Model

the worst case parameters. By so doing, the actual *x*∗

our uncertainty bounds were not truly encompassing.

*ko* <sup>=</sup> <sup>Δ</sup>*<sup>r</sup>*

Now choose *xref* equal to the associated best case *x*<sup>∗</sup>

the best case parameters. By so doing, the actual *x*∗

uncertainty bounds were not truly encompassing.

perturbations occur. We shall illustrate this below.

*ro* <sup>=</sup> <sup>Δ</sup>*<sup>q</sup>*

perturbations occur; i.e. <sup>Δ</sup>*<sup>k</sup>*

value for J is given by *E*[*J*] =

nominal design (green).

and *x*∗

model and *x*∗

all of the parameters.)

*<sup>e</sup>* ; i.e. the *x*<sup>∗</sup>

*co* <sup>=</sup> <sup>Δ</sup>*<sup>δ</sup>*

only cases 1-3 can occur; i.e. cases 4-6 cannot occur. The only way, cases 4-6 can occur is if

2. **Best-Best Case Approach.** Assume that we have good bounds on parametric uncertainty (not necessarily tight, but encompassing)for the 6 model parameters under consideration: *k*, *r*, *q*, *p*, *c*, *δ*. Suppose that we design for the best best case scenario; i.e. try to approach the return of the perfect information optimal policy when the best case parameter

<sup>429</sup> Design of Robust Policies for Uncertain Natural Resource Systems:

*qo* <sup>=</sup> <sup>Δ</sup>*<sup>p</sup>*

case maximal parameter perturbations. (For simplicity, we assume that all parameters are perturbed by their maximum best case percentage and that this percentage is the same for

cases 4-6 can occur; i.e. cases 1-3 cannot occur. The only way, cases 1-3 can occur is if our

3. **Blended Best-Worst-Best-Best Approach.** One can also try to offer a blended approach that attempts to offer decent returns when either worse case or best case parameter

4. **Probabilistic Approach.** If a probability density function for the parameter percentage *θ* is available, it can be used to determine where to operate. Let *f<sup>θ</sup>* denote a density function for *θ*. This can be used to derive the density function *fJ* for *J*. Given this, the expected

*<sup>J</sup> J fJ*(*J*) *dJ* <sup>=</sup>

2. A Best-Worst Case Design: *xref* = 29.7, *uref* = 0.629 3. A Nominal Design Based on the Nominal Parameters: *xref* = 37.5, *uref* = 0.625 4. A Best-Best Case Design: *xref* = 48.6, *uref* = 0.595 The performance of the perfect information designs are always best (by definition). The performance of the best-worst case design (blue) duplicates that of the perfect information design for 20% worst case perturbations since it is based on the worst case parameter model

*<sup>e</sup>* . The performance of the best-best case design (blue) duplicates that of the perfect information design for 20% best case perturbations since it is based on the best case parameter

1. The best-worst case design does better than the best-best case design when its parameter assumptions are maximally incorrect; falling by less than 20% (with respect to perfect information optimal return) while the best best falls by more than 40% (with respect

*<sup>e</sup>* . The following key observations are in order within figure 4 (small IC case):

To summarize, the following specific optimal control laws were implemented: 1. Perfect Information Optimal Designs: *xref* = *x*<sup>∗</sup>

function for *θ* can be used to reflect what parameter perturbations are most likely to occur. The above expectation can then be used to choose *xref* to maximize the expectation. To illustrate the above ideas, consider figures 4-5 for small and large initial conditions, respectively under extremal parameter perturbations. The figures show results for the perfect information designs (black), best worst case design (blue), best best case design (red), and the

*po* <sup>&</sup>gt; 0 and <sup>Δ</sup>*<sup>c</sup>*

*<sup>e</sup>* ; i.e. the *x*<sup>∗</sup>

*<sup>θ</sup> J*(*θ*) *fJ*(*J*(*θ*)) *J*�

*<sup>e</sup>* that results from choosing

*<sup>δ</sup><sup>o</sup>* < 0. We assume best

*<sup>e</sup>* that results from choosing

(*θ*) *dθ*. The density

*<sup>e</sup>* , *uref* = *u*<sup>∗</sup>

*e*

*<sup>e</sup>* will be smaller than *xref* . As such, only

*<sup>e</sup>* will be greater than *xref* . As such,

**Dealing with Uncertain** *x*∗ *<sup>e</sup>* **and** *J*<sup>∗</sup> *<sup>e</sup>* **.** Let *xref* denote the reference biomass at which the fishery manager wishes to operate the fishery 2. How does a manager choose the target fishery biomass *xref* ? A biologically conservative manager may wish to keep the fishery at the maximum sustainable yield *xref* = *xMSY* = *<sup>k</sup>* <sup>2</sup> . A financially aggressive manager may chose to operate the fishery at the infinite discount (*δ* = ∞) ) optimal value *xref* = *x*<sup>∞</sup> = *<sup>c</sup> pq* . More generally, a manager could use the optimal value *xref* = *x*<sup>∗</sup> *<sup>e</sup>* as the point at which to operate. Given that *x*∗ *<sup>e</sup>* is known to within a percentage <sup>Δ</sup>*x*<sup>∗</sup> *e x*∗ *<sup>e</sup>* , it follows that a fishery manager might try to operate at (1) *x*∗ *<sup>e</sup>* − Δ*x*<sup>∗</sup> *<sup>e</sup>* if economic aggression is desired, or at (2) *x*<sup>∗</sup> *<sup>e</sup>* + Δ*x*<sup>∗</sup> *<sup>e</sup>* if biological conservatism is desired. The *x*∗ *<sup>e</sup>* concept gives the fishery manager a way to systematically think about fishery biomass targets.

**Uncertainty In (***xo***,** *x*∗ *<sup>e</sup>* **,** *xref***): 6 Cases.** In general, *xo* and *x*<sup>∗</sup> *<sup>e</sup>* are uncertain. How does one choose the target *xref* . We'd ideally like *xref* = *x*<sup>∗</sup> *<sup>e</sup>* , but *x*<sup>∗</sup> *<sup>e</sup>* is uncertain. What can a manager do? The table below contains the six possible relations that can exist amongst the three scalars (*xo*, *x*∗ *<sup>e</sup>* , *xref*) - from smallest to biggest. In general, we would (ideally) like the state to move from *xo* toward *xref* = *x*<sup>∗</sup> *<sup>e</sup>* . Since *x*<sup>∗</sup> *<sup>e</sup>* is uncertain, it follows that *xref* (in general) will differ from *x*∗ *<sup>e</sup>* . As such, it follows that we may issue reference commands *xref* that move the state *x* in an incorrect direction. Since the state moves from *xo* toward *xref* , it follows from the table below that in two cases the state moves in the incorrect direction. In the four other cases, the state moves in the correct direction.


Table 3. Six Possible Inequality Relations for (*xo*, *x*∗ *<sup>e</sup>* , *xref*)

To select *xref* , we offer the following approaches.

1. **Best-Worst Case Approach.** Assume that we have good bounds on parametric uncertainty (not necessarily tight, but encompassing) for the 6 model parameters under consideration: *k*, *r*, *q*, *p*, *c*, *δ*. Suppose that we design for the best worst case scenario; i.e. try to approach the return of the perfect information optimal policy when the worst case parameter perturbations occur; i.e. <sup>Δ</sup>*<sup>k</sup> ko* <sup>=</sup> <sup>Δ</sup>*<sup>r</sup> ro* <sup>=</sup> <sup>Δ</sup>*<sup>q</sup> qo* <sup>=</sup> <sup>Δ</sup>*<sup>p</sup> po* <sup>&</sup>lt; 0 and <sup>Δ</sup>*<sup>c</sup> co* <sup>=</sup> <sup>Δ</sup>*<sup>δ</sup> <sup>δ</sup><sup>o</sup>* > 0. We assume worst case maximal parameter perturbations. (For simplicity, we assume that all parameters are perturbed by their maximum worst case percentage and that this percentage is the same for all of the parameters.)

One could, for example, pick a worst case percentage which bounds all of the parameters. Doing so can be conservative. Parameter estimation can be used to narrow tighten this worst case percentage. If we have fixed percentage bounds for each of the parameters, our approach remains the same. (Recall: Determining the worst case perturbation in our problem is easy. This is not true in most practical scenarios.)

<sup>2</sup> It is understood that *xref* can change with time. For now, we assume *xref* is fixed.

14 Will-be-set-by-IN-TECH

manager wishes to operate the fishery 2. How does a manager choose the target fishery biomass *xref* ? A biologically conservative manager may wish to keep the fishery at the

> *e x*∗

*<sup>e</sup>* if economic aggression is desired, or at (2) *x*<sup>∗</sup>

*<sup>e</sup>* , but *x*<sup>∗</sup>

*<sup>e</sup>* Incorrect Way too much fishing (Way Too Aggressive)

*<sup>e</sup>* Correct Too much fishing (Moderately Aggressive)

*<sup>e</sup>* , *xref*)

*po* <sup>&</sup>lt; 0 and <sup>Δ</sup>*<sup>c</sup>*

*co* <sup>=</sup> <sup>Δ</sup>*<sup>δ</sup>*

*<sup>δ</sup><sup>o</sup>* > 0. We assume worst

*<sup>e</sup> xref* Correct Too little fishing (Moderately Conservative)

to operate the fishery at the infinite discount (*δ* = ∞) ) optimal value *xref* = *x*<sup>∞</sup> = *<sup>c</sup>*

*<sup>e</sup>* **,** *xref***): 6 Cases.** In general, *xo* and *x*<sup>∗</sup>

do? The table below contains the six possible relations that can exist amongst the three scalars

*<sup>e</sup>* . As such, it follows that we may issue reference commands *xref* that move the state *x* in an incorrect direction. Since the state moves from *xo* toward *xref* , it follows from the table below that in two cases the state moves in the incorrect direction. In the four other cases, the state

*<sup>e</sup> xo* Correct Too much fishing (Very Aggressive)

*<sup>e</sup> xo xref* Incorrect Way too little fishing (Way Too Conservative)

1. **Best-Worst Case Approach.** Assume that we have good bounds on parametric uncertainty (not necessarily tight, but encompassing) for the 6 model parameters under consideration: *k*, *r*, *q*, *p*, *c*, *δ*. Suppose that we design for the best worst case scenario; i.e. try to approach the return of the perfect information optimal policy when the worst case parameter

*qo* <sup>=</sup> <sup>Δ</sup>*<sup>p</sup>*

case maximal parameter perturbations. (For simplicity, we assume that all parameters are perturbed by their maximum worst case percentage and that this percentage is the same

One could, for example, pick a worst case percentage which bounds all of the parameters. Doing so can be conservative. Parameter estimation can be used to narrow tighten this worst case percentage. If we have fixed percentage bounds for each of the parameters, our approach remains the same. (Recall: Determining the worst case perturbation in our

*<sup>e</sup> xref xo* Correct Too little fishing (Very Conservative)

*<sup>e</sup>* , *xref*) - from smallest to biggest. In general, we would (ideally) like the state to move

*<sup>e</sup>* **.** Let *xref* denote the reference biomass at which the fishery

*<sup>e</sup>* concept gives the fishery manager a way to systematically

*<sup>e</sup>* is uncertain, it follows that *xref* (in general) will differ from

<sup>2</sup> . A financially aggressive manager may chose

*<sup>e</sup>* , it follows that a fishery manager might

*<sup>e</sup>* as the point at which to operate.

*<sup>e</sup>* + Δ*x*<sup>∗</sup>

*<sup>e</sup>* are uncertain. How does one

*<sup>e</sup>* is uncertain. What can a manager

*pq* . More

*<sup>e</sup>* if biological

**Dealing with Uncertain** *x*∗

Given that *x*∗

(*xo*, *x*∗

*x*∗

try to operate at (1) *x*∗

**Uncertainty In (***xo***,** *x*∗

from *xo* toward *xref* = *x*<sup>∗</sup>

moves in the correct direction.

*xref x*<sup>∗</sup>

*xo x*∗

perturbations occur; i.e. <sup>Δ</sup>*<sup>k</sup>*

for all of the parameters.)

*x*∗

*x*∗

*xref xo x*<sup>∗</sup>

*xo xref x*<sup>∗</sup>

conservatism is desired. The *x*∗

think about fishery biomass targets.

*<sup>e</sup>* **and** *J*<sup>∗</sup>

generally, a manager could use the optimal value *xref* = *x*<sup>∗</sup>

*<sup>e</sup>* is known to within a percentage <sup>Δ</sup>*x*<sup>∗</sup>

maximum sustainable yield *xref* = *xMSY* = *<sup>k</sup>*

*<sup>e</sup>* − Δ*x*<sup>∗</sup>

choose the target *xref* . We'd ideally like *xref* = *x*<sup>∗</sup>

Smallest → Biggest Direction

Table 3. Six Possible Inequality Relations for (*xo*, *x*∗

*ko* <sup>=</sup> <sup>Δ</sup>*<sup>r</sup>*

problem is easy. This is not true in most practical scenarios.)

<sup>2</sup> It is understood that *xref* can change with time. For now, we assume *xref* is fixed.

*ro* <sup>=</sup> <sup>Δ</sup>*<sup>q</sup>*

To select *xref* , we offer the following approaches.

*<sup>e</sup>* . Since *x*<sup>∗</sup>

Now choose *xref* equal to the associated worst case *x*<sup>∗</sup> *<sup>e</sup>* ; i.e. the *x*<sup>∗</sup> *<sup>e</sup>* that results from choosing the worst case parameters. By so doing, the actual *x*∗ *<sup>e</sup>* will be greater than *xref* . As such, only cases 1-3 can occur; i.e. cases 4-6 cannot occur. The only way, cases 4-6 can occur is if our uncertainty bounds were not truly encompassing.

2. **Best-Best Case Approach.** Assume that we have good bounds on parametric uncertainty (not necessarily tight, but encompassing)for the 6 model parameters under consideration: *k*, *r*, *q*, *p*, *c*, *δ*. Suppose that we design for the best best case scenario; i.e. try to approach the return of the perfect information optimal policy when the best case parameter perturbations occur; i.e. <sup>Δ</sup>*<sup>k</sup> ko* <sup>=</sup> <sup>Δ</sup>*<sup>r</sup> ro* <sup>=</sup> <sup>Δ</sup>*<sup>q</sup> qo* <sup>=</sup> <sup>Δ</sup>*<sup>p</sup> po* <sup>&</sup>gt; 0 and <sup>Δ</sup>*<sup>c</sup> co* <sup>=</sup> <sup>Δ</sup>*<sup>δ</sup> <sup>δ</sup><sup>o</sup>* < 0. We assume best case maximal parameter perturbations. (For simplicity, we assume that all parameters are perturbed by their maximum best case percentage and that this percentage is the same for all of the parameters.)

Now choose *xref* equal to the associated best case *x*<sup>∗</sup> *<sup>e</sup>* ; i.e. the *x*<sup>∗</sup> *<sup>e</sup>* that results from choosing the best case parameters. By so doing, the actual *x*∗ *<sup>e</sup>* will be smaller than *xref* . As such, only cases 4-6 can occur; i.e. cases 1-3 cannot occur. The only way, cases 1-3 can occur is if our uncertainty bounds were not truly encompassing.


To illustrate the above ideas, consider figures 4-5 for small and large initial conditions, respectively under extremal parameter perturbations. The figures show results for the perfect information designs (black), best worst case design (blue), best best case design (red), and the nominal design (green).

To summarize, the following specific optimal control laws were implemented:


The performance of the perfect information designs are always best (by definition). The performance of the best-worst case design (blue) duplicates that of the perfect information design for 20% worst case perturbations since it is based on the worst case parameter model and *x*∗ *<sup>e</sup>* . The performance of the best-best case design (blue) duplicates that of the perfect information design for 20% best case perturbations since it is based on the best case parameter model and *x*∗ *<sup>e</sup>* . The following key observations are in order within figure 4 (small IC case):

1. The best-worst case design does better than the best-best case design when its parameter assumptions are maximally incorrect; falling by less than 20% (with respect to perfect information optimal return) while the best best falls by more than 40% (with respect

• Although the best-best case design appears worse in terms of percentages at off design conditions, it has a higher expected return across all cases versus the best-worst case

<sup>431</sup> Design of Robust Policies for Uncertain Natural Resource Systems:

• A manager could readily design a policy that limited the worst case downside return to a certain percentage of the maximum possible. For example, if the manager wanted a worst case downside return no worse than 5% of the maximum possible, a policy should

• A manager may also be interested in implementing the following policy: max*<sup>θ</sup> E*[*J*(*θ*)]. **Sensitivity Analysis: Extremal Perturbation, Large Initial Condition.** The expected value

70

75

80

85

90

J / Jperfect

Fig. 5. Economic Inefficiencies for Various Optimal Control Laws: Extremal Perturbations

Finally, it should be noted that in contrast to the low initial condition study conducted, the Best-Best Case Design performs worse both in terms of the percentage possible and the

**Limit Cycles In the Presence of Uncertainty.** Finally, consider figure 6. The optimal control law is based on the nominal BOE parameters. The initial condition is above the uncertain *x*∗

The simulation is conducted with a truth plant possessing a 10% reduction in *k* - hence the limit cycle behavior. The figure shows that: (1) The optimal control policy (in general) will exhibit limit cycle behavior when we have imperfect information; i.e. model parameters are not known exactly. (2) The optimal control policy (in general) will be very sensitive to finer

95

100

105

−30 −20 −10 0 10 20 30

BOE: xo

= 56.25

Parameter Perturbation (%)

*e* .

be designed around roughly a −2 % parameter perturbation.

for each of the design cases considered are as follows:

Application to the Classic Gordon-Schaefer Fishery Model

= 56.25

BOE: xo

Perfect Information Design Best Worst Case Design Nominal Design Best Best Case Design

−30 −20 −10 0 10 20 30

Parameter Perturbation (%)

• *E*[*J*] = 1284.5 for the Best-Worst Case Design

• *E*[*J*] = 1259.3 for the Best-Best Case Design

• *E*[*J*] = 1336.8 for the Nominal Design

• *E*[*J*] = 1368.2 for the Perfect Information Optimal Designs

expected return when compared to the Best-Worst Case Design.

design.

0

(Large IC)

500

1000

J (M\$)

1500

2000

2500

to perfect information optimal return) when its parameter assumptions are maximally incorrect.

2. The nominal design can be viewed as a nice compromise or blend between the two prior policies. It is based on the nominal parameter model and *x*∗ *<sup>e</sup>* . Its returns deteriorates by a little more than 10% for worst case parameter uncertainty and by a little more than 5% for best case parameter uncertainty. In short, the returns associated with this nominal (blended) policy offers flatter returns over a wider range of extremal parameter perturbations.

Each of the above three approaches offer a specific design model (to base the control design upon) and a specific *x*∗ *<sup>e</sup>* to use as a target. Control laws are always evaluated with the true (nonlinear) plant. In what follows, we will use the above as benchmarks whose performance we shall target via robust control laws. Similar patterns are observed for the large IC case in figure 5.

**Sensitivity Analysis: Extremal Perturbations, Small Initial Condition.** The expected value for each of the design cases considered are as follows:

Fig. 4. Economic Inefficiencies for Various Optimal Control Laws: Extremal Perturbations (Small IC)


A uniform distribution has been assumed for the parameter uncertainty. The optimal perfect information control law is included for comparison purposes. Its performance can only be approximated over a range of parameter perturbations. This is because the (1) design plant parameters differ from those of the true plant and the desired target *xref* differs from the perfect information target *x*∗ *e* .

The following additional points are in order:

16 Will-be-set-by-IN-TECH

2. The nominal design can be viewed as a nice compromise or blend between the two prior

Each of the above three approaches offer a specific design model (to base the control design

(nonlinear) plant. In what follows, we will use the above as benchmarks whose performance we shall target via robust control laws. Similar patterns are observed for the large IC case in

**Sensitivity Analysis: Extremal Perturbations, Small Initial Condition.** The expected value

70

75

80

85

90

J / Jperfect

Fig. 4. Economic Inefficiencies for Various Optimal Control Laws: Extremal Perturbations

A uniform distribution has been assumed for the parameter uncertainty. The optimal perfect information control law is included for comparison purposes. Its performance can only be approximated over a range of parameter perturbations. This is because the (1) design plant parameters differ from those of the true plant and the desired target *xref* differs from the

95

100

105

*<sup>e</sup>* to use as a target. Control laws are always evaluated with the true

policies. It is based on the nominal parameter model and *x*∗

for each of the design cases considered are as follows:

= 18.75

BOE: xo

Perfect Information Design Best Worst Case Design Nominal Design Best Best Case Design

−30 −20 −10 0 10 20 30

Parameter Perturbation (%)

• *E*[*J*] = 772.0 for the Best-Worst Case Design

The following additional points are in order:

• *E*[*J*] = 838.0 for the Nominal Design • *E*[*J*] = 800.4 for the Best-Best Case Design

perfect information target *x*∗

• *E*[*J*] = 867.8 for the Perfect Information Optimal Designs

*e* .

incorrect.

perturbations.

figure 5.

(Small IC)

J (M\$)

upon) and a specific *x*∗

to perfect information optimal return) when its parameter assumptions are maximally

by a little more than 10% for worst case parameter uncertainty and by a little more than 5% for best case parameter uncertainty. In short, the returns associated with this nominal (blended) policy offers flatter returns over a wider range of extremal parameter

*<sup>e</sup>* . Its returns deteriorates

−30 −20 −10 0 10 20 30

BOE: xo

= 18.75

Parameter Perturbation (%)


**Sensitivity Analysis: Extremal Perturbation, Large Initial Condition.** The expected value for each of the design cases considered are as follows:

Fig. 5. Economic Inefficiencies for Various Optimal Control Laws: Extremal Perturbations (Large IC)


Finally, it should be noted that in contrast to the low initial condition study conducted, the Best-Best Case Design performs worse both in terms of the percentage possible and the expected return when compared to the Best-Worst Case Design.

**Limit Cycles In the Presence of Uncertainty.** Finally, consider figure 6. The optimal control law is based on the nominal BOE parameters. The initial condition is above the uncertain *x*∗ *e* . The simulation is conducted with a truth plant possessing a 10% reduction in *k* - hence the limit cycle behavior. The figure shows that: (1) The optimal control policy (in general) will exhibit limit cycle behavior when we have imperfect information; i.e. model parameters are not known exactly. (2) The optimal control policy (in general) will be very sensitive to finer

respect to parametric uncertainty, and achieve returns that are close to those of the perfect

<sup>433</sup> Design of Robust Policies for Uncertain Natural Resource Systems:

**Control System Structure.** The structure of the control system may be visualized as shown in

*u management action*

✲

❄❞✛

*<sup>e</sup>* . Parameter uncertainty prevents us from commanding the desired

 *pro <sup>s</sup>* <sup>+</sup> *pro*

–

*sat*(·) *Saturation (feasibility constraint)*

✲ *up*

provides high frequency roll-off to ensure that

*P Fishery being managed* ✲

(30)

(31)

*actual output*

*x*

information optimal control law.

*W Pre-filter*

✲ *xref f* ❞ ✲ –

to approximate our nonlinear plant.

pair will be stabilized. The term *pro*

high frequency sensor noise *n* is suitably attenuated.

like to use *xref* = *x*<sup>∗</sup>

suitably bounded.

*e*

Feedback System with a Pre-Filter and Anti-Windup Logic

✻

Application to the Classic Gordon-Schaefer Fishery Model

*error K*

✲

*Policy*

*GAW* ✛

Fig. 7. Renewable Resource Management Problem Represented as a Standard Negative

*<sup>K</sup>*(*s*) = *<sup>g</sup>*(*<sup>s</sup>* <sup>+</sup> *<sup>z</sup>*) *s*

*<sup>s</sup>*+*pro*

4. *Command Pre-Filter. W* is a reference command pre-filter possessing the structure:

with the anti-windup logic may be described by the following equations:

*<sup>W</sup>*(*s*) = *<sup>z</sup>*

This pre-filter can be used to ensure that the overshoot to step reference commands is

5. *Observer-Based Integrator Anti-Windup Logic.* Anti-windup logic is included so that the integrator in the PI controller does not windup. That is, the integrator is turned off so that it does not integrate constant errors which occur when the input to the plant is saturated (Aström, Hägglund). The structure of the anti-windup logic is as follows *x*˙*<sup>k</sup>* = *Akxk* + *Bke* + *L*(*sat*(*u*) − *u*) where *L* is an observer gain matrix. The PI controller

*s* + *z*

1. *Plant.* Here, *P* represents the plant under control. We shall use an LTI small signal model

2. *Reference State or Command. xref* is the desired reference biomass state. Ideally, we would

state. As such, we are forced to choose *xref* more judiciously. Given this, we will give special attention to maximizing our return under the worst case parameter uncertainty. 3. *PI Controller. K* is a proportional-plus-integral (PI) controller possessing the structure:

where *g* > 0, *z* > 0, and *pro* > 0. The integrator within the controller will ensure that step biomass commands are followed in the steady state while step input/output disturbances are rejected in the steady state. The (*s* + *z*) term will ensure that the LTI plant-integrator

Figure 7.

✲ *xref desired output*

sampling (Δ*T* smaller) under imperfect information; i.e. more oscillations (switching) will be exhibited as our *x* time samples are spaced closer together. The figure also shows that low pass filtering the optimal with a lag can be used to smooth oscillations a bit. To significantly reduce the oscillations, however, there is no easy fix. We either need a penalized *u*˙ *p* term within *J* to penalize switching or we need policies that are inherently more robust (like the ones we will describe subsequently). As such, this implies that, in practice, the optimal control policy is prohibitively expensive to implement in the presence of parametric uncertainty because of the inherent limit cycle behavior and the associated switching costs.

Fig. 6. Optimal Control Law Robustness: Limit Cycles In Presence of (-10% Capacity) Uncertainty

**Motivation for Robust Control Laws.** The above motivates the need for more robust control laws; i.e. control laws that (1) exhibit an acceptable return (i.e. return robustness) in the presence of anticipated (likely) parametric uncertainty; (2) do not exhibit limit cycle behavior in the presence of anticipated (likely) parametric uncertainty. As such, the above motivates the robust control laws to be considered in our presentation.

**Control Law Implementation Issues.** Unlike many control applications where controllers are implemented with great fidelity (within state-of-the-art digital computing units), controllers within a resource management system are implemented by an organization by setting rules for the fishery worker community (e.g. quotas, taxes (Clark, 1990, Chapter 8). As such, many types of uncertainties can be introduced by the organization. These could include any of the following: (1) parameter uncertainty, (2) additional uncertain actuation/sensing dynamics (e.g. lags, time delays, rate limiters, etc.), (3) nonlinearities (e.g. rate limiters, saturations, quantization, dead zones), (4) actuation/incentive errors (e.g. quota/tax miscalculations), (5) sensing, measurement, and estimation errors (e.g. sensor dynamics, biomass sampling/aliasing/quntization errors, noise, disturbances).

## **4. Robust control laws**

The model under consideration is very simple. Many tools from the controls literature may be applied (e.g. classical control (Rodriguez, 2003), H-infinity (Rodriguez, 2004), feedback linearization, SDRE's, etc.). Given the introductory/tutorial nature of the paper, the simplicity of the model being used, as well as the fact that this text covers advanced control methodologies, we shall focus on simple control strategies from classical control theory. We will show that such control laws can be used to avoid limit cycles, increase robustness with 18 Will-be-set-by-IN-TECH

sampling (Δ*T* smaller) under imperfect information; i.e. more oscillations (switching) will be exhibited as our *x* time samples are spaced closer together. The figure also shows that low pass filtering the optimal with a lag can be used to smooth oscillations a bit. To significantly reduce the oscillations, however, there is no easy fix. We either need a penalized *u*˙ *p* term within *J* to penalize switching or we need policies that are inherently more robust (like the ones we will describe subsequently). As such, this implies that, in practice, the optimal control policy is prohibitively expensive to implement in the presence of parametric uncertainty because of

0 0.5 1 1.5 2

0 0.5 1 1.5 2

time (yrs)

0 0.2 0.4 0.6 0.8 1

u<sup>p</sup>

time (yrs)

**Motivation for Robust Control Laws.** The above motivates the need for more robust control laws; i.e. control laws that (1) exhibit an acceptable return (i.e. return robustness) in the presence of anticipated (likely) parametric uncertainty; (2) do not exhibit limit cycle behavior in the presence of anticipated (likely) parametric uncertainty. As such, the above motivates

**Control Law Implementation Issues.** Unlike many control applications where controllers are implemented with great fidelity (within state-of-the-art digital computing units), controllers within a resource management system are implemented by an organization by setting rules for the fishery worker community (e.g. quotas, taxes (Clark, 1990, Chapter 8). As such, many types of uncertainties can be introduced by the organization. These could include any of the following: (1) parameter uncertainty, (2) additional uncertain actuation/sensing dynamics (e.g. lags, time delays, rate limiters, etc.), (3) nonlinearities (e.g. rate limiters, saturations, quantization, dead zones), (4) actuation/incentive errors (e.g. quota/tax miscalculations), (5) sensing, measurement, and estimation errors (e.g. sensor

The model under consideration is very simple. Many tools from the controls literature may be applied (e.g. classical control (Rodriguez, 2003), H-infinity (Rodriguez, 2004), feedback linearization, SDRE's, etc.). Given the introductory/tutorial nature of the paper, the simplicity of the model being used, as well as the fact that this text covers advanced control methodologies, we shall focus on simple control strategies from classical control theory. We will show that such control laws can be used to avoid limit cycles, increase robustness with

dynamics, biomass sampling/aliasing/quntization errors, noise, disturbances).

Fig. 6. Optimal Control Law Robustness: Limit Cycles In Presence of (-10% Capacity)

the inherent limit cycle behavior and the associated switching costs.

x

the robust control laws to be considered in our presentation.

37 37.5 38 38.5 39 39.5 40

0 10 20 30 40 50

Monthly ΔT Daily ΔT Daily ΔT w/lag

time (yrs)

Uncertainty

**4. Robust control laws**

J(up

)

respect to parametric uncertainty, and achieve returns that are close to those of the perfect information optimal control law.

**Control System Structure.** The structure of the control system may be visualized as shown in Figure 7.

Fig. 7. Renewable Resource Management Problem Represented as a Standard Negative Feedback System with a Pre-Filter and Anti-Windup Logic


$$K(s) = \frac{g(s+z)}{s} \left[\frac{p\_{ro}}{s+p\_{ro}}\right] \tag{30}$$

where *g* > 0, *z* > 0, and *pro* > 0. The integrator within the controller will ensure that step biomass commands are followed in the steady state while step input/output disturbances are rejected in the steady state. The (*s* + *z*) term will ensure that the LTI plant-integrator pair will be stabilized. The term *pro <sup>s</sup>*+*pro* provides high frequency roll-off to ensure that high frequency sensor noise *n* is suitably attenuated.

4. *Command Pre-Filter. W* is a reference command pre-filter possessing the structure:

$$\mathcal{W}(\mathbf{s}) = \left[\frac{z}{s+z}\right] \tag{31}$$

This pre-filter can be used to ensure that the overshoot to step reference commands is suitably bounded.

5. *Observer-Based Integrator Anti-Windup Logic.* Anti-windup logic is included so that the integrator in the PI controller does not windup. That is, the integrator is turned off so that it does not integrate constant errors which occur when the input to the plant is saturated (Aström, Hägglund). The structure of the anti-windup logic is as follows *x*˙*<sup>k</sup>* = *Akxk* + *Bke* + *L*(*sat*(*u*) − *u*) where *L* is an observer gain matrix. The PI controller with the anti-windup logic may be described by the following equations:

7. *Damping Factor from Percent Overshoot Specification.* Determine *ζ* from the overshoot

<sup>435</sup> Design of Robust Policies for Uncertain Natural Resource Systems:

8. *Undamped Natural Frequency from Settling Time Specification.* Determine *ωn* from the settling

*<sup>ω</sup><sup>n</sup>* <sup>=</sup> <sup>5</sup> *ζts*

9. *PI Controller Parameters.* Determine the PI controller gain *g* and zero *z* from:

*<sup>g</sup>* <sup>=</sup> <sup>2</sup>*ζω<sup>n</sup>* <sup>+</sup> *<sup>a</sup>*

10. *Controller Roll-Of Parameter.* Choose the roll-off parameter *pro* as follows:

the robust policies based upon the nominal LTI plant model *Pd* = *<sup>b</sup>*

steady state noise attenuation specification.

Application to the Classic Gordon-Schaefer Fishery Model

**5. Control law comparisons**

considered.)

the figure (see Figure 6).

**5.1 Sample control law time responses**

*<sup>π</sup>*<sup>2</sup> + |*lnMp*|<sup>2</sup>

*<sup>b</sup> <sup>z</sup>* <sup>=</sup> *<sup>ω</sup>*<sup>2</sup>

so that the added high frequency roll-off does not significantly degrade the nominal phase margin within the loop. It could also be selected in order to satisfy a specific sinusoidal

11. *Anti-Windup Gain.* Choose the anti-windup gain *GAW* > 0 to be sufficiently large so that the integrator suitably shuts down in order to "maximally recapture" the dominant second order response characteristics described above. A family of gains is examined below.

In this section, we compare the properties of the nominal optimal control law with those for

Within this section, sample time responses are provided for families of robust control laws (based upon the nominal LTI plant model) - families that approximate the performance of the nominal optimal control law. (Note: There will be an approximation gap when uncertainty is

**Reference Biomass Tracking: Anti-Windup Gain Study.** Figure 8 shows closed loop biomass tracking time responses for a family of robust control law designs where *ζ* = 1, *ts* = 1. The anti-windup gain *GAW* is varied to control how well the responses approximate that of the optimal with no limit cycle behavior. As the anti-windup gain *GAW* is increased, the responses come closer to the (nominal) optimal control law (with no limit cycle behavior). The limit cycle behavior of the (nominal) optimal has been cleaned up in order to improve the readability of

**A Note On Robustness with Respect to High Frequency Unmodeled Dynamics.** It should be noted that as the speed of a policy is increased, the significance of unmodeled high frequency dynamics within the fishery or within the policy implementing organization/evironment (e.g. lags, time delays, rate limiters) becomes an issue to consider in final policy evaluation.

*n*

*pro* = 10*ω<sup>n</sup>* (42)

*<sup>s</sup>*−*<sup>a</sup>* .

*bg* (41)

(39)

(40)

*<sup>ζ</sup>* <sup>=</sup> <sup>|</sup>*lnMp*<sup>|</sup> 

specification:

time specification:

$$\dot{\mathbf{x}}\_1 = \mathbf{e} + \mathbf{G}\_{AW}(sat(u) - u) \qquad \qquad \dot{\mathbf{x}}\_2 = \mathbf{g}z\mathbf{x}\_1 - p\_{ro}\mathbf{x}\_2 + \mathbf{g}\mathbf{e} \qquad \qquad u = p\_{ro}\mathbf{x}\_2 \tag{32}$$

where *GAW* is the anti-windup gain.

**Nominal Design Methodology.** The nominal design methodology can be described as follows (Rodriguez, 2003):


$$L = P\_d K \approx \frac{b g(s+z)}{s(s-a)} = \frac{n(s)}{d(s)}.\tag{33}$$

4. *Nominal Closed Loop Characteristic Equation.* Form the nominal closed loop characteristic equation

$$\Phi\_{cl}(\mathbf{s}) = d(\mathbf{s}) + \mathfrak{n}(\mathbf{s}) = \mathbf{s}^2 + (\mathbf{b}\mathbf{g} - \mathbf{a})\mathbf{s} + \mathfrak{b}\mathbf{g}\mathbf{z} = \mathbf{0} \tag{34}$$

This polynomial has the "standard second order form"

$$\Phi\_{cl}(\mathbf{s}) = \mathbf{s}^2 + 2\mathsf{Z}\omega\omega\_n\mathbf{s} + \omega\_n^2 \tag{35}$$

where *ζ* = *bg*−*<sup>a</sup>* 2 <sup>√</sup>*bgz* is the damping factor and *<sup>ω</sup><sup>n</sup>* <sup>=</sup> <sup>√</sup>*gz* is the undamped natural frequency. For stable nominal complex closed loop poles, we require 0 < *ζ* < 1.

5. *Closed Loop Poles.* Determine the nominal closed loop poles (assumed complex for rapid transient response):

$$s = -\zeta \omega\_n \pm j\omega\_n \sqrt{1 - \zeta^2} \tag{36}$$

Given this, we will have nominal (local) closed loop exponential stability with an associated time constant *τ* = <sup>1</sup> *ζω<sup>n</sup>* . The associated (approximate 1%) settling time is *ts* = 5*τ*.

6. *Standard Second Order Closed Loop Transfer Function and Percent Overshoot.* With the command pre-filter *W*, the associated closed loop transfer function takes the standard second order form:

$$T\_{\mathbf{x}\_{ref}} = \frac{\text{WPK}}{1 + \text{PK}} \approx \frac{\omega\_n^2}{s^2 + 2\Im\omega\_n s + \omega\_n^2} \tag{37}$$

As such, the associated percent overshoot to a step reference command is given by

$$M\_p = e^{-\frac{\tau}{\xi}\omega\_n t\_p} = e^{-\left(\frac{\xi\pi}{1-\xi^2}\right)}\tag{38}$$

where *tp* = *<sup>π</sup> ω<sup>n</sup>* <sup>√</sup>1−*ζ*<sup>2</sup> is the time at which the peak overshoot occurs. 20 Will-be-set-by-IN-TECH

**Nominal Design Methodology.** The nominal design methodology can be described as follows

to approximate our nonlinear plant. Here, *Pd* is referred to as the *design plant*; i.e. the plant upon which we will base our control law design. While any design we obtain can be evaluated using plant approximants such as *Pd*, control designs must be evaluated with

*<sup>s</sup>*(*<sup>s</sup>* <sup>−</sup> *<sup>a</sup>*) <sup>=</sup> *<sup>n</sup>*(*s*)

<sup>2</sup> + 2*ζωns* + *ω*<sup>2</sup>

<sup>√</sup>*bgz* is the damping factor and *<sup>ω</sup><sup>n</sup>* <sup>=</sup> <sup>√</sup>*gz* is the undamped natural frequency.

*d*(*s*)

3. *Nominal Open Loop Approximant.* Form the open loop transfer function approximant *<sup>L</sup>* <sup>=</sup> *PdK* <sup>≈</sup> *bg*(*<sup>s</sup>* <sup>+</sup> *<sup>z</sup>*)

4. *Nominal Closed Loop Characteristic Equation.* Form the nominal closed loop characteristic

5. *Closed Loop Poles.* Determine the nominal closed loop poles (assumed complex for rapid

Given this, we will have nominal (local) closed loop exponential stability with an

6. *Standard Second Order Closed Loop Transfer Function and Percent Overshoot.* With the command pre-filter *W*, the associated closed loop transfer function takes the standard

<sup>1</sup> <sup>+</sup> *PK* <sup>≈</sup> *<sup>ω</sup>*<sup>2</sup>

As such, the associated percent overshoot to a step reference command is given by

*Mp* = *e*−*ζωntp* = *e*

<sup>√</sup>1−*ζ*<sup>2</sup> is the time at which the peak overshoot occurs.

*s* = −*ζω<sup>n</sup>* ± *jω<sup>n</sup>*

1. *Plant Approximant.* The following small signal LTI model *P* ≈ *Pd*

2. *Controller Approximant.* Use the controller approximant *<sup>K</sup>* <sup>≈</sup> *<sup>g</sup>*(*s*+*z*)

Φ*cl*(*s*) = *d*(*s*) + *n*(*s*) = *s*

Φ*cl*(*s*) = *s*

For stable nominal complex closed loop poles, we require 0 < *ζ* < 1.

*Txref <sup>x</sup>* <sup>=</sup> *WPK*

This polynomial has the "standard second order form"

where *GAW* is the anti-windup gain.

the actual nonlinear plant model.

(Rodriguez, 2003):

equation

where *ζ* = *bg*−*<sup>a</sup>* 2

transient response):

second order form:

where *tp* = *<sup>π</sup>*

*ω<sup>n</sup>*

associated time constant *τ* = <sup>1</sup>

*x*˙1 = *e* + *GAW*(*sat*(*u*) − *u*) *x*˙2 = *gzx*<sup>1</sup> − *prox*<sup>2</sup> + *ge u* = *prox*<sup>2</sup> (32)

def = *<sup>b</sup>*

*<sup>s</sup>*−*<sup>a</sup>* will be used

*<sup>s</sup>* where *g* > 0 and *z* > 0.

. (33)

*<sup>n</sup>* (35)

<sup>1</sup> − *<sup>ζ</sup>*<sup>2</sup> (36)

(37)

(38)

*ζω<sup>n</sup>* . The associated (approximate 1%) settling time is *ts* = 5*τ*.

*n*

*n s*<sup>2</sup> + 2*ζωns* + *ω*<sup>2</sup>

> − *ζπ* <sup>1</sup>−*ζ*<sup>2</sup>

<sup>2</sup> + (*bg* <sup>−</sup> *<sup>a</sup>*)*<sup>s</sup>* <sup>+</sup> *bgz* <sup>=</sup> <sup>0</sup> (34)

7. *Damping Factor from Percent Overshoot Specification.* Determine *ζ* from the overshoot specification:

$$\zeta = \frac{|\ln \mathcal{M}\_p|}{\sqrt{\pi^2 + |\ln \mathcal{M}\_p|^2}}\tag{39}$$

8. *Undamped Natural Frequency from Settling Time Specification.* Determine *ωn* from the settling time specification:

$$
\omega\_{\text{ll}} = \frac{5}{\zeta t\_{\text{s}}} \tag{40}
$$

9. *PI Controller Parameters.* Determine the PI controller gain *g* and zero *z* from:

$$\mathbf{g} = \frac{2\mathbf{\tilde{J}}\omega\_{\text{ll}} + a}{b} \qquad \qquad \qquad \qquad z = \frac{\omega\_{\text{m}}^2}{b\mathbf{g}} \tag{41}$$

10. *Controller Roll-Of Parameter.* Choose the roll-off parameter *pro* as follows:

$$p\_{ro} = 10\omega\_n\tag{42}$$

so that the added high frequency roll-off does not significantly degrade the nominal phase margin within the loop. It could also be selected in order to satisfy a specific sinusoidal steady state noise attenuation specification.

11. *Anti-Windup Gain.* Choose the anti-windup gain *GAW* > 0 to be sufficiently large so that the integrator suitably shuts down in order to "maximally recapture" the dominant second order response characteristics described above. A family of gains is examined below.

## **5. Control law comparisons**

In this section, we compare the properties of the nominal optimal control law with those for the robust policies based upon the nominal LTI plant model *Pd* = *<sup>b</sup> <sup>s</sup>*−*<sup>a</sup>* .

#### **5.1 Sample control law time responses**

Within this section, sample time responses are provided for families of robust control laws (based upon the nominal LTI plant model) - families that approximate the performance of the nominal optimal control law. (Note: There will be an approximation gap when uncertainty is considered.)

**Reference Biomass Tracking: Anti-Windup Gain Study.** Figure 8 shows closed loop biomass tracking time responses for a family of robust control law designs where *ζ* = 1, *ts* = 1. The anti-windup gain *GAW* is varied to control how well the responses approximate that of the optimal with no limit cycle behavior. As the anti-windup gain *GAW* is increased, the responses come closer to the (nominal) optimal control law (with no limit cycle behavior). The limit cycle behavior of the (nominal) optimal has been cleaned up in order to improve the readability of the figure (see Figure 6).

**A Note On Robustness with Respect to High Frequency Unmodeled Dynamics.** It should be noted that as the speed of a policy is increased, the significance of unmodeled high frequency dynamics within the fishery or within the policy implementing organization/evironment (e.g. lags, time delays, rate limiters) becomes an issue to consider in final policy evaluation.

well as the undershoot. As the damping factor *ζ* is reduced, the response speeds up (getting closer to that of the (nominal) optimal with no limit cycle behavior), although the observed undershoot increases. The limit cycle behavior of the (nominal) optimal has been cleaned up

<sup>437</sup> Design of Robust Policies for Uncertain Natural Resource Systems:

**Reference Biomass Tracking: Settling Time Study.** Figure 10 shows closed loop biomass tracking time responses for a family of robust control law designs where *ζ* = 1 (critically damped, *Mp* = 0) and *GAW* = 3. As the settling time *ts* of the closed loop system is reduced, the responses come closer to the (nominal) optimal (with no limit cycle behavior). The limit cycle behavior of the (nominal) optimal has been cleaned up in order to improve the

37

0.4

0.6

0.8

u<sup>p</sup>

In this section, we try to shed light on the utility of our linear time invariant (LTI) based robust control system design methodology and how linear simulation can be used to approximate/predict the behavior of the nonlinear simulations. All designs are based upon

**Linear vs Nonlinear Biomass Tracking:** *xo* **Near** *xref* **.** Figure 11 compares linear and nonlinear closed loop biomass tracking simulations where the the initial condition (IC) is near the desired set point (target biomass). Four responses are shown for *x* and *up*: (1) purely linear; i.e. linear plant model, linear controller, and no saturation, (2) linear with plant saturation; i.e. linear plant model, linear controller, and plant saturation, (3) linear with anti-windup logic; i.e. linear plant model, linear controller, plant saturation, and anti-windup logic, (4) nonlinear; i.e. nonlinear plant model, linear controller, plant saturation, and anti-windup logic. Here, the reference command is very small (*xref* = 0.375), the control does not saturate, and all of the responses match one another. This shows that the "pure linear theory" suffices under small

1

38

39

x

40

41

0 0.5 1 1.5 2 2.5 3

t s = 0.5 t s = 0.75

t s = 1 t s = 1.5 optimal

time (yrs)

0 0.5 1 1.5 2 2.5 3

time (yrs)

in order to improve the readability of the figure (see Figure 6).

Application to the Classic Gordon-Schaefer Fishery Model

0 10 20 30 40 50

time (yrs)

0 0.5 1 1.5 2 2.5 3

time (yrs)

**5.2 Utility of linear design methodology**

nominal parameter values.

signal conditions (as expected).

Fig. 10. Reference Biomass Tracking: Settling Time Study (*GAW* = 3, *ζ* = 1)

readability of the figure.

0

0

1

2

u

3

4

200

400

J(up

)

600

Fig. 8. Reference Biomass Tracking: Anti-Windup Gain Study (*ζ* = 1, *ts* = 1)

It is well known from fundamental robustness theory (Rodriguez, 2004; 2003) that fast control laws can result in closed loop oscillatory responses or instability when high frequency unmodeled dynamics are "significantly excited." This issue will be examined in future work. **Reference Biomass Tracking: Damping Factor Study.** Figure 9 shows shows closed loop biomass tracking time responses for a family of robust control law designs where *ts* = 1, *GAW* = 3. The damping factor *ζ* is varied in order to control the speed of the response as

Fig. 9. Reference Biomass Tracking: Damping Factor Study (*GAW* = 3, *ts* = 1)

22 Will-be-set-by-IN-TECH

0.4

37

0.4

0.6

0.8

u<sup>p</sup>

1

38

39

x

40

41

0.6

0.8

u<sup>p</sup>

It is well known from fundamental robustness theory (Rodriguez, 2004; 2003) that fast control laws can result in closed loop oscillatory responses or instability when high frequency unmodeled dynamics are "significantly excited." This issue will be examined in future work. **Reference Biomass Tracking: Damping Factor Study.** Figure 9 shows shows closed loop biomass tracking time responses for a family of robust control law designs where *ts* = 1, *GAW* = 3. The damping factor *ζ* is varied in order to control the speed of the response as

1

x

0 0.5 1 1.5 2 2.5 3

GAW = 0 GAW = 0.5 GAW = 1 GAW = 3 Optimal

time (yrs)

0 0.5 1 1.5 2 2.5 3

time (yrs)

0 0.5 1 1.5 2 2.5 3

ζ = 0.5 ζ = 0.707 ζ = 0.83 ζ = 1 optimal

time (yrs)

0 0.5 1 1.5 2 2.5 3

time (yrs)

0 10 20 30 40 50

time (yrs)

0 0.5 1 1.5 2 2.5 3

time (yrs)

0 10 20 30 40 50

time (yrs)

0 0.5 1 1.5 2 2.5 3

time (yrs)

Fig. 9. Reference Biomass Tracking: Damping Factor Study (*GAW* = 3, *ts* = 1)

Fig. 8. Reference Biomass Tracking: Anti-Windup Gain Study (*ζ* = 1, *ts* = 1)

0

0 0.5 1 1.5 2 2.5

0

0 0.5 1 1.5 2 2.5

u

200

400

J(up

)

600

u

200

400

J(up

)

600

well as the undershoot. As the damping factor *ζ* is reduced, the response speeds up (getting closer to that of the (nominal) optimal with no limit cycle behavior), although the observed undershoot increases. The limit cycle behavior of the (nominal) optimal has been cleaned up in order to improve the readability of the figure (see Figure 6).

**Reference Biomass Tracking: Settling Time Study.** Figure 10 shows closed loop biomass tracking time responses for a family of robust control law designs where *ζ* = 1 (critically damped, *Mp* = 0) and *GAW* = 3. As the settling time *ts* of the closed loop system is reduced, the responses come closer to the (nominal) optimal (with no limit cycle behavior). The limit cycle behavior of the (nominal) optimal has been cleaned up in order to improve the readability of the figure.

Fig. 10. Reference Biomass Tracking: Settling Time Study (*GAW* = 3, *ζ* = 1)

## **5.2 Utility of linear design methodology**

In this section, we try to shed light on the utility of our linear time invariant (LTI) based robust control system design methodology and how linear simulation can be used to approximate/predict the behavior of the nonlinear simulations. All designs are based upon nominal parameter values.

**Linear vs Nonlinear Biomass Tracking:** *xo* **Near** *xref* **.** Figure 11 compares linear and nonlinear closed loop biomass tracking simulations where the the initial condition (IC) is near the desired set point (target biomass). Four responses are shown for *x* and *up*: (1) purely linear; i.e. linear plant model, linear controller, and no saturation, (2) linear with plant saturation; i.e. linear plant model, linear controller, and plant saturation, (3) linear with anti-windup logic; i.e. linear plant model, linear controller, plant saturation, and anti-windup logic, (4) nonlinear; i.e. nonlinear plant model, linear controller, plant saturation, and anti-windup logic. Here, the reference command is very small (*xref* = 0.375), the control does not saturate, and all of the responses match one another. This shows that the "pure linear theory" suffices under small signal conditions (as expected).

far from the desired set point (target biomass). Here, the reference command is large (*xref* = 18.75), the controls saturate, windup is exhibited in the linear w/Sat case, and we observe relatively good agreement between the linear (particularly linear w/AW) and

<sup>439</sup> Design of Robust Policies for Uncertain Natural Resource Systems:

**Biomass Tracking Robustness In Presence Of Capacity Uncertainty: Anti-Windup Gain Study.** Figure 13 shows how our robust control laws can be adjusted to achieve the "flatter" economic inefficiency of the nominal optimal control law (see Figures 4-5). We observe the

−10

−8

−6

−4

% Inefficiency of Optimal Control Law

Fig. 13. PI Biomass Tracking Robustness (Capacity Uncertainty): Anti-Windup Gain Study

• With an anti-windup implementation, a PI control law can come arbitrarily close to matching the performance of the nominal optimal control law with imperfect information. • Improving upon the nominal optimal control law with imperfect information requires some outer loop control logic as well as system identification to more appropriately select

The observed performance gap (or inefficiency) is fundamentally because the target *xref*

truth plant. Closing the observed performance gap further requires an outer loop controller and/or parameter estimation techniques in order to get a more accurate target *xref* that is

**Summary.** This chapter has shown how ideas from robust control may be applied to a fishery. It has been specifically shown how some small amount of income may be sacrificed

**Directions for Future Research.** Future work will examine more complex models (e.g. decentralized, distributed), pros/cons associated with parameter estimation schemes, more complex robust control laws (e.g. use of receding horizon control for long-term management), robustness with respect to plant and controller uncertainty (parametric and

*e* .

for increased robustness with respect to uncertain fishery parameters.

Maximum Return Actual Return GAW = 0 GAW = 0.5 GAW = 1 GAW = 3

−2

0

2

−30 −20 −10 0 10 20 30

Optimal GAW = 0 GAW = 0.5 GAW = 1 GAW = 3

Δ k (%)

*<sup>e</sup>* ; not because the nominal design plant differs from the

nonlinear responses.

−30 −20 −10 0 10 20 30

Application to the Classic Gordon-Schaefer Fishery Model

Δ k (%)

the reference/target biomass.

differs from the perfect information *x*∗

closer to the perfect information target *x*∗

**6. Summary and future directions**

following:

300

(*ζ* = 1, *ts* = 1)

dynamic).

400

500

600

J(up

)

700

800

900

Fig. 11. Linear vs Nonlinear Biomass Tracking: *xo* Near *xref*

**Linear vs Nonlinear Biomass Tracking:** *xo* **Far From** *xref* **.** Figure 12 compares linear and nonlinear closed loop biomass tracking simulations where the the initial condition (IC) is

Fig. 12. Linear vs Nonlinear Biomass Tracking: *xo* Far From *xref*

24 Will-be-set-by-IN-TECH

0

0.5

1

u<sup>p</sup>

**Linear vs Nonlinear Biomass Tracking:** *xo* **Far From** *xref* **.** Figure 12 compares linear and nonlinear closed loop biomass tracking simulations where the the initial condition (IC) is

Linear vs Nonlinear: Large Command Size

0 5 10 15

Time (yrs)

Linear vs Nonlinear: Large Command Size

0 5 10 15

Time (yrs)

Fig. 12. Linear vs Nonlinear Biomass Tracking: *xo* Far From *xref*

1.5

2

0 0.5 1 1.5 2

Linear vs Nonlinear: Large Command Size

Time (yrs)

Linear Linear w/Sat Linear w/AW Nonlinear

Linear Linear w/Sat Linear w/AW Nonlinear

0 0.5 1 1.5 2

Linear vs Nonlinear: Large Command Size

Linear Linear w/Sat Linear w/AW Nonlinear

Time (yrs)

Fig. 11. Linear vs Nonlinear Biomass Tracking: *xo* Near *xref*

37.3

2

0

0.5

1

u<sup>p</sup>

1.5

x

37.4

37.5

37.6

x

37.7

37.8

37.9

far from the desired set point (target biomass). Here, the reference command is large (*xref* = 18.75), the controls saturate, windup is exhibited in the linear w/Sat case, and we observe relatively good agreement between the linear (particularly linear w/AW) and nonlinear responses.

**Biomass Tracking Robustness In Presence Of Capacity Uncertainty: Anti-Windup Gain Study.** Figure 13 shows how our robust control laws can be adjusted to achieve the "flatter" economic inefficiency of the nominal optimal control law (see Figures 4-5). We observe the following:

Fig. 13. PI Biomass Tracking Robustness (Capacity Uncertainty): Anti-Windup Gain Study (*ζ* = 1, *ts* = 1)


The observed performance gap (or inefficiency) is fundamentally because the target *xref* differs from the perfect information *x*∗ *<sup>e</sup>* ; not because the nominal design plant differs from the truth plant. Closing the observed performance gap further requires an outer loop controller and/or parameter estimation techniques in order to get a more accurate target *xref* that is closer to the perfect information target *x*∗ *e* .

## **6. Summary and future directions**

**Summary.** This chapter has shown how ideas from robust control may be applied to a fishery. It has been specifically shown how some small amount of income may be sacrificed for increased robustness with respect to uncertain fishery parameters.

**Directions for Future Research.** Future work will examine more complex models (e.g. decentralized, distributed), pros/cons associated with parameter estimation schemes, more complex robust control laws (e.g. use of receding horizon control for long-term management), robustness with respect to plant and controller uncertainty (parametric and dynamic).

Holling, C.S., Gunderson, L.H. (2002). Resilience and adaptive cycles. *In: L.H. Gunderson, C.S.*

<sup>441</sup> Design of Robust Policies for Uncertain Natural Resource Systems:

Holling, C.S. (1986) The resilience of terrestrial ecosystems, local surprise and global change.

Holling, C.S.(1973). Resilience and stability of ecological systems. *Annual Review of Ecology and*

ISRIC. (1990). Global assessment of the status of human induced soil degradation

Kendrick, D.A. (2005). Stochastic control for economic models: past, present and the paths

Koenig, E.F. (1984). Controlling stock externalities in a common property fishery subject to uncertainty. *Journal of Environmental Economics and Management 11*(2), 124–138 Lempert, R.J., Groves, D.G., Popper, S.W. & Bankes, S.C. (2006) A general, analytic method

Lempert, R.J., Schlesinger, M.E. (2000). Robust strategies for abating climate change - an

Ludwig, D., Walker, B. & Holling, C.S. (1997). Sustainability, stability, and resilience. *Conservation Ecology 1*(1), (online) URL: http://www.consecol.org/vol1/iss1/art7/

Ludwig, D. & Walters, C. (1982). Optimal harvesting with imprecise parameter estimates.

Moran, E.F., Ostrom, E. (eds.) (2006). Seeing the Forest and the Trees: Human-Environment

Myers, R. & Worm, B. (2003). Rapid worldwide depletion of predatory fish communities.

Rodriguez, A.A., Cifdaloz, O., Anderies, J.M., Janssen, J. & Dickeson, J., (2010). Confronting

Rodriguez, A.A. (2004). Analysis and Design of Multivariable Feedback Control Systems.

Rodriguez, A.A. (2003). Analysis and Design of Feedback Control Systems. *Control3D, LLC.*,

Schaefer, M. (1957). Some considerations of population dynamics and economics in relation to

Sethi, G., Costello, C., Fisher, A., Hanemann, M. & Karp, L. (2005). Fishery management

Shah, T., Molden, D., Sakthivadivel, R., Seckler. (2000). The global groundwater situation:

Management Challenges in Highly Uncertain Natural REsource Systems: a Robustness-Vulnerability Trade-off Approach," *Environmental Modeling & Assessment*,

the management of the commercial marine fisheries. *Journal of the Fisheries Research*

under multiple uncertainty. *Journal of Environmental Economics and Management 50*(2),

Overview of opportunities and challenges. *Tech. rep., International Water Management*

Interactions in Forest Ecosystems. *MIT Press*, Cambridge, MA

Ogata, K. (1995). Discrete-Time Control Systems. *Prentice Hall*, (2nd Ed.), N.J.

ahead. *Journal of Economic Dynamics and Control 29*, 3–30

editorial essay. *Climatic Change 45*(3-4), 387–401

*Ecological Modelling 14*, 273–292

Volume 16, Issue 1 (2011), Page 15.

*Control3D, LLC.*, Tempe, AZ

*Board of Canada 14*(5), 669–681

*Institute, Colombo, Sri Lanka*

*Nature 423*, 280–283

Tempe, AZ

300–318

chap. 2. Island Press

*Systematics 4*, 1–23

Netherlands

514–528

Cambridge University Press, Cambridge

Application to the Classic Gordon-Schaefer Fishery Model

*Holling (eds.) Panarchy: Understanding Transformations in Systems of Humans and Nature*,

*In: C.W. C., R.E. Munn (eds.) Sustainable development of the biosphere*, pp. 292–317.

(glasdod). *dataset, International Soil Reference and Information Centre*, Wageningen, The

for generating robust strategies and narrative scenarios. *Management Science 52*(4),

#### **7. References**


26 Will-be-set-by-IN-TECH

Anderies, J.M., Rodriguez, A., Janssen, M. & Cifdaloz, O. (2007). Panaceas, uncertainty, and

Andersen, P. (1982). Commercial fisheries under price uncertainty. *Journal of Environmental*

Aström, K. & Hägglund, T. (1995). PID Controllers: Theory, Design, and Tuning. *Int. Society*

Babonneau, F., Vial, J.P. & Apparigliato, R.: Robust optimization for environmental and energy

Belmiloudi, A. (2006). Minimax control problems of periodic competing parabolic systems with logistic growth terms. *International Journal of Control 79*(2), 150–161 Belmiloudi, A. (2005). Nonlinear optimal control problems of degenerate parabolic equations

Ben-Tal, A., Nemirovski, A. (2008). Selected topics in robust convex optimization. *Mathematical*

Ben-Tal, A., Nemirovski, A. (2002). Robust optimization—methodology and applications. *Mathematical Programming 92*(3, Ser. B), 453–480. ISMP 2000, Part 2 (Atlanta, GA) Ben-Tal, A., El-Ghaoui L. & Nemirovski, A. (2000) Robust semidefinite programming.

Ben-Tal, A., Nemirovski, A.(1998). Robust convex optimization. *Mathematics of Operations*

Clark, C. (2007) The Worldwide Crisis in Fisheries: Economic Models and Human Behavior.

Clark, C. (2006). Fisheries bioeconomics: why is it so misunderstood?, In: *Population Ecology*

Clark, C., Munro, G. & U.R., S. (2006). Subsidies, buybacks, and sustainable fisheries. *Journal*

Clark, C.W. (1990). Mathematical Bioeconomics: The Optimal Management of Renewable

Clark, C. & Kirkwood, G.: On uncertain renewable resource stocks - optimal harvest policies

Dercole, F., Gragnani, A., Kuznetsov, Y.A. & Rinaldi, S. (2003) Numerical sliding bifurcation

Gordon, H. (1954) The economic theory of a common property resource: the fishery. *Journal of*

Hansen, L.P. & Sargent, T.J. (2007) Robustness. *Princeton University Press*, Princeton, NJ Holland, D. & Gudmundsson, E., J., G. (1999) Do fishing vessel buyback programs work? a

*Systems I-Fundamental Theory and Applications 50*(8), 1058–1063

and the value of stock surveys. *Journal of Environmental Economics and Management*

analysis: an application to a relay control system. *Ieee Transactions on Circuits and*

*for Measurement and Control*, (2nd Ed.), Research Triangle Park, N.C.

*Academy of Sciences USA 104*, 15,194–15,199

*Economics and Management 9*(1), 11–28

*Methods & Applications 63*(8), 1126–1152

*Cambridge University Press, Cambridge*, UK

Resources, 2 edn. *J. Wiley*, N.Y., U.S.A.

survey of the evidence. *Marine Policy 23*, 47–69

*of Environmental Economics and Management 50*, 47–58

*Programming 112*(1), 125–158

*Applications. Kluwer Dordrecht*

*Research 23*(4), 769–805

*13*(3), 235–244 (1986)

*Political Economy 62*, 124–142

*48*, 95-98

79–126. Springer (2010)

the robust control framework in sustainability science. *Proceedings of the National*

planning. *In: J.A. Filar, A. Haurie (eds.) Uncertainty and Environmental Decision Making, International Series in Operations Research & Management Science*, vol. 138, chap. 3, pp.

with logistic time-varying delays of convolution type. *Nonlinear Analysis-theory*

*In: R. Saigal, L. Vandenberghe, H. Wolkowicz (eds.) Semidefinite Programming and*

**7. References**


**20**

*Japan*

**Robustness and Security of H**∞**-Synchronizer in**

In recent years, a large amount of work on chaos-based cryptosystems has been published (Kocarev (2001); Millérioux et al. (2008)). A general methodology for designing chaotic and hyperchaotic cryptosystems has been developed using the control systems theory (Grassi et al. (1999); Liao et al. (1999); Yang et al. (1997a;b)). The chaotic communication system is closely related to the concept of chaos synchronization. An overview of chaotic secure communication systems can be found in (Yang (2004)). He classified the continuous-time chaotic secure communication systems into four generations. In the third generation, the combination of the classical cryptographic technique and chaotic synchronization is used to enhance the degree of security. Specifically, Yang *et al.* proposed a new chaos-based secure communication scheme in an attempt to thwart the attacks (Yang et al. (1997a;b)). They have combined both conventional cryptographic method and synchronization of chaotic systems. Their cryptographic method consists of an encryption function (the multi-shift cipher), a decryption function (the inverse of the encryption function), a chaotic encrypter that generates the key signal for the encryption function, and a decrypter that estimates the key signal. The approach has a limitation since the cryptosystem design may fail if different chaotic circuits are utilized. So far, this generation has the highest security in all the chaotic communication systems had been proposed and has not yet broken. From the control theoretic perspective, the transmitter and the receiver in the chaotic communication system can be considered as the nonlinear plant and its observer, respectively. Grassi *et al.* proposed a nonlinear-observer-based decrypter to reconstruct the state of the encrypter (Grassi et al. (1999); Liao et al. (1999)). They extended the Chua's oscillator to the observer-based decrypter. The cryptosystem does not require initial conditions of the encrypter and the decrypter belonging to the same basin of attraction. If we can design a decrypter without the knowledge of the parameters of the encrypter, the chaos-based secure communication systems are not secure, because the parameters of the encrypter is selected as static secret keys in the cryptosystem. Parameter identification and adaptive synchronization methods may be effective for intruders in building reconstruction mechanisms, even when a synchronizing system is not available. Therefore, it is important for secure issues to investigate whether adaptive identifiers without the system information of encrypter can be constructed or not. We have recently designed an observer-based chaotic communication system combining the cryptosystems proposed by Grassi *et al.* (Grassi et al. (1999)) and by Liao *et al.* (Liao et al. (1999)) that allows us to assign the relative degree and the zeros of its encrypter system (Matsuo et al. (2004)). Specifically, we constructed three cryptosystems based on a Chua's circuit by assigning its relative degree and zeros. The cryptosystem consists of

**1. Introduction**

**Chaotic Communication System**

*Oita University, Dannoharu, Oita*

Takami Matsuo, Yusuke Totoki and Haruo Suemitsu

Walters, C. (1986). Adaptive Management of Renewable Resources. *MacMillan Publishing Co.*

