Automatica, June 2008, Volume 44, No. 6 Table of Contents
| Dynamic systems, discrete time or continuous time,
deterministic or stochastic, arise in all walks of life. Optimal control
theory is an important technique dealing with optimization of these
systems. Until the late fifties or early sixties, the use of optimal
control methods was limited to problems in physical sciences and
engineering. Since then, optimal control has found many new areas of
applications. It is some of these new applications to which this special
issue of Automatica is devoted.
There is already a long tradition of stochastic and robust optimization in energy and manufacturing engineering. Related models have also flourished in economics and finance (e.g. portfolio management, pricing of derivatives, investment under uncertainty, industrial organization). The deregulation of energy markets associated with the creation of markets for tradable emission permits (Clean air act in the US, Kyoto agreement in Europe) and more generally, the growing pressure for implementing environmentally conscious economic and management policies, have opened new domains of investigation and applied research situated at the crossroads of engineering, environmental economics and finance. This special issue of Automatica collects contributions on advanced methods and applications in this area.
A first group of three papers are methodological and contain existence results respectively for a class of infinite dimensional control problems, for Markov decision processes with countable state set and for stochastic games with uncountable state set.
Arie Leizarowitz, in Turnpike properties of a class of aquifer control problems, studies the long-run behavior of a distributed control system and establishes convergence to a steady state of the solutions to this infinite horizon, nonlinear infinite dimensional optimal control problem. This is motivated by agricultural economics models concerned with the optimal management of a homogeneous, unconfined coastal aquifer. The controller can influence the state of the aquifer by determining the rate of water extraction at different locations and times. The state equation of the aquifer is then given by a non-linear parabolic equation subject to certain boundary conditions. The goal is to optimize a reward expression over an infinite time horizon. The optimality definition used is in the spirit of the overtaking optimality notion.
Lukasz Balbus and Andrzej S. Nowak, in Existence of perfect equilibria in a class of multigenerational stochastic games of capital accumulation, introduce a model of multigenerational stochastic game of capital accumulation where each generation consists of m different players. They prove the existence of a perfect stationary equilibrium in an infinite horizon game. A suitable change in the terminology used in this paper provides (in the case of perfect altruism between generations) a new Nash equilibrium theorem for standard stochastic games with uncountable state space.
Arie Leizarowitz and Adam Shwartz, in Exact finite approximations of average-cost countable Markov decision processes, consider a countable-state Markov decision process for which they introduce an embedding which produces a finite-state Markov decision process. The finite-state embedded process has the same optimal cost, and moreover, it has the same dynamics as the original process when restricting to the approximating set. The embedded process can be used as an approximation which, being finite, is more convenient for computation and implementation.
The next group of four papers is focused on control applications in economics and finance.
Carlo Carraro and Alessandra Sgobbi, in Modelling negotiated decision making in environmental and natural resource management, propose a computational model which simulates the process of negotiation among more than two players, who bargain over the sharing of more than one pie. Through numerical simulation of several multiple issues negotiation games among multiple players, they identify the main features of players’ optimal strategies and equilibrium agreements. As in most economic situations, uncertainty crucially affects also bargaining processes. Therefore, in this analysis, they introduce uncertainty over the size of the pies to be shared and assess the impact on players’ strategic behavior. The model proposed here can have several applications to natural resource and environmental management at the national or local level, where conflicts arise on how to share a resource of a finite size.
Hans M. Amman, David A. Kendrick, and Marco P. Tucci, in Solving the Beck and Wieland model with optimal experimentation in DualPC, compare various methods for solving economic models through optimal experimentation. They provide two versions of the Beck & Wieland (BW) model, as a basis for comparing numerical results across methods and codes. They also provide a quantitative analysis of the properties of the BW model when it is solved with adaptive control methods, and compare Optimal Feedback (OF), Expected Optimal Feedback (EOF) and Dual Control (DC) methods as applied to the two versions of the BW model. Finally they provide results that can be used to initiate the analysis of the comparative advantage of the various solution methods for adaptive control of economic environmental models.
Hanqin Zhang and Qing Zhang, in Trading a mean-reverting asset: Buy low and sell high, consider an optimal trading (buy and sell) rule. The underlying asset price is governed by a mean-reverting model. The objective is to buy and sell the asset so as to maximize the overall return. Slippage cost is imposed on each transaction. They show that the solution to the original optimal stopping problem can be obtained by solving two quasi-algebraic equations. Sufficient conditions are given in the form of a verification theorem. A numerical example is reported to demonstrate the applicability of these results to financial and energy markets.
André de Palma and Jean-Luc Prigent, in Hedging global environment risks: An option based portfolio insurance, introduce a financial hedging model for global environmental risks. Their approach is based on portfolio insurance under hedging constraints. Each investor is assumed to maximize the expected utility of his/her portfolio which includes financial and environmental assets. The optimal investment is determined for quite general utility functions and hedging constraints. The results show how and why derivative assets should be introduced in the portfolio to hedge environmental risks. The main conclusion of the paper is that new types of options which combine both equity and environmental assets should be used, contrary to the current practice which considers two separate option markets.
The next two papers propose stochastic control formulations for global environmental policy assessment.
David W.K. Yeung and Leon Petrosyan, in A cooperative stochastic differential game of transboundary industrial pollution, study a cooperative solution which is subgame consistent in a differential game of transboundary industrial pollution. In this solution the optimality principle is maintained in any subgame which starts at a later time with any feasible state brought about by prior optimal behaviors. The subgame consistent cooperative solution is derived in this stochastic differential game together with a payment distribution mechanism that supports the solution.
Olivier Bahn, Alain Haurie and Roland Malhamé, in A stochastic control model for optimal timing of climate policies, propose a stochastic control model as a paradigm for the optimal timing of greenhouse gases (GHG) emissions abatement. The resolution of uncertainty concerning climate sensitivity and the technological breakthrough providing access to a carbon-free production economy are modeled as controlled stochastic jump processes. The optimal policy is characterized using the dynamic programming solution to a piecewise deterministic optimal control problem. A numerical illustration is developed with a set of parameters calibrated on recently proposed models for integrated assessment of climate policies.
A group of two papers deal with new approaches to stochastic programming.
Yuri Nesterov and Jean-Philippe Vial, in Confidence level solutions for stochastic programming, propose an alternative approach to stochastic programming based on Monte-Carlo sampling and stochastic gradient optimization. The optimality concept is based on defining a probability small enough that the random algorithm produces a solution with an expected objective value departing from the optimal one by more than a given small parameter. They derive complexity bounds on the number of iterations of this process and show that by repeating the basic process on independent samples, the number of iterations can be significantly reduced. They also discuss the relevance of this approach to the important environmental issues of this century, in particular for the design of climate friendly economic policies.
Julien Thénié and Jean-Philippe Vial, in Step decision rules for multistage stochastic programming: A heuristic approach, propose an original method to produce efficient solutions to multistage stochastic optimization problems. SPSDR, like plain multistage Stochastic Programming (SP), operates on a Monte-Carlo “computing sample” of moderate size that approximates the stochastic process. Unlike SP, SPSDR does not strive to build a balanced event tree out of that sample. Rather, it defines a solution as a special type of decision rule, with the property that the decisions at each stage are piecewise constant functions on the sample of scenarios. The rule is constructed so that the non-anticipativity condition is met. Three methods, SPSDR, SP and Robust Optimization, are compared for the same 12-stage problem in supply chain management, using different objectives and performance criteria.
The application of stochastic programming and control theory to hydro power system management is considered in the next two papers.
Kristian Nolde, Markus Uhr, and Manfred Morari, in Medium term scheduling of a hydro-thermal system using stochastic model predictive control, consider a multistage stochastic programming formulation for monthly production planning of a hydro-thermal system. Stochasticity from variations in water reservoir inflows and fluctuations in demand of electric energy are considered explicitly. The problem is solved efficiently via nested Benders decomposition. The solution is implemented in a model predictive control setup and performance of this control technique is demonstrated in simulations.
Andrea Castelletti, Francesca Pianosi, and Rodolfo Soncini-Sessa, in Water reservoir control under economic, social and environmental constraints, review, in a strict control theory perspective, recent and significant advances in designing management policies for water reservoir networks, under economic, social and environmental constraints. A general problem formulation is provided, along with a description of traditional solution techniques, their limitations and possible alternative approaches.
A group of three papers deal with the modeling of emissions control in power systems and combustion plants as well as air quality control policies.
Pedro Linares, Francisco Javier Santos, Mariano Ventosa, and Luis lapiedra, in Incorporating oligopoly, CO2 emissions trading and green certificates into a power generation expansion model, present a generation expansion model for the power sector which incorporates several interesting features for its application to current electricity markets: it considers the possible oligopolistic behavior of firms, and incorporates relevant policy instruments, carbon emissions trading and tradable green certificates. The model is formulated as a Linear Complementarity Problem (LCP) which allows solving simultaneously the optimization problem for each firm considering the power, carbon and green certificate markets. The model has been applied to the Spanish power system.
Alexandra Grancharova, Jus Kocijan, and Tor Arne Johansen, in Explicit stochastic predictive control of combustion plants based on Gaussian process models, use a Nonlinear Model Predictive Control (NMPC) method for achieving an efficient combustion control. They consider the application of an explicit approximate approach for stochastic NMPC to the design of an explicit reference tracking NMPC controller for a combustion plant based on its Gaussian process model. The controller brings the air factor (respectively the concentration of oxygen in the flue gases) to its optimal value with every change of the load factor and thus an optimal operation of the combustion plant is achieved.
Claudio Carnevale, Enrico Pisoni, and Marialuisa Volta, in A multi-objective nonlinear optimization approach to design effective air quality control policies, present the implementation of a two-objective optimization methodology to select effective tropospheric ozone pollution control strategies on a mesoscale domain. The considered objectives are (a) the emission reduction cost and (b) the Air Quality Index. The control variables are the precursor emission reductions due to abatement technologies. The nonlinear relationship linking air quality objective and precursor emissions is described by artificial neural networks, identified by processing deterministic chemical transport modeling system simulations. The two-objective problem has been applied to a complex domain in Northern Italy.
Finally Jeffrey D. Azzato and Jacek B. Krawczyk, in Applying a finite-horizon numerical optimization method to a periodic optimal control problem, describe the application of a new method to solve a periodic optimal control problem. Approximately optimal feedback rules are computed that can control the system both on and off the optimal orbit. Periodic optimal control is especially applicable to situations requiring sensible long-term management of resources or externalities.
While for reasons of space limitation, the compilation of papers presented here can only be a restricted sample of the very diverse control theoretic and optimization research currently taking place at the interface between systems theory, energy economics and the environment, and the underlying methods, the guest editors hope that it will contribute in stimulating further work in this vibrant multidisciplinary area, of importance for our collective future.
We thank all our contributors.
Alain Haurie and Roland Malhamé