Automatica, August 2006, Volume 42, No. 8
Dynamic systems, discrete time or continuous time, deterministic or stochastic, arise in all walks of life. Optimal control theory is an important technique dealing with optimization of these systems. Until the late 1950s or early 1960s, the use of optimal control methods was limited to problems in physical sciences and engineering. Since then, optimal control has found many new areas of applications. It is some of these new applications to which this special issue of Automatica is devoted.
The objective of this special issue is to present state-of-the-art results in applications of optimal control theory to management science. The idea is to generate an interest in such applications among the readers of Automatica as well as to encourage researchers in management science to consider submitting their control applications papers to Automatica.
The Call for Papers for the special issue was well received, and a total of 32 papers were submitted. The papers were reviewed by experts in the area and 19 of these papers are appearing in this issue. The topics covered a broad range, from applications of deterministic and stochastic optimal control, robust control, and differential games to such areas as production and inventory management, supply chains, finance, marketing, and economics. Ten papers cover a variety of topics in operations management, five papers are in marketing, three papers deal with problems in finance, and one paper is devoted to a problem in environmental economics.
Ten papers deal with operations management problems (three in production planning and inventory control, four in supply chain management, two in maintenance, and one in mining operations planning):
L. Rodrigues and E.-K. Boukas: Piecewise-Linear H∞ Controller Synthesis with Applications to Inventory Control of Switched Production Systems
This paper focuses on inventory control of a production system. The production system is modeled as a constrained switching linear system. A feedback H∞ theory is used to treat the underlying control design so as to achieve desired quadratic stability. A state feedback controller is obtained that forces the stock level to be kept close to zero in the presence of random demand.
D. Bauso, F. Blanchini, and R. Pesenti: Robust Control Strategies for Multi-Inventory Systems with Average Flow Constraints
This paper is about multi-inventory systems facing random demands. Using a long-term average demand, a nominal flow that feeds the demand is selected. Control policies that meet at each time all possible current demands and achieve the nominal flow in the average are studied. Linear programming is used to provide necessary and sufficient conditions for such a strategy and to characterize the set of achievable flows. In the case of a static flow, it shows that the strategy must be affine and the dynamics problem can be solved by a linear saturated control strategy.
M. Chen, I.-K. Cho, and S.P. Meyn: Reliability by Design in Distributed Power Transmission Networks
This paper is concerned with a centralized optimal solution for a power network model. It provides a centralized optimal solution for a power network model by generalizing recent techniques for the centralized optimal control of demand-driven production systems. The optimal solution indicates how reserves must be adjusted according to environmental factors including variability, and the ramping-rate constraints on generation in order to respond to unforeseen events such as unexpected surge in demand. It also addresses transmission constraint sensitivity through the construction of an effective cost on an aggregate model.
G. Gaalman: Bullwhip Reduction for ARMA Demand: The Proportional Order-up-to Policy versus the Full-State-Feedback Policy
The author considers bullwhip reduction for ARMA demand at a single stage in a supply chain. The motivation comes from the fact that the spread of the bullwhip effect in a supply chain occurs even when each stage in the supply chain acts independently of the others. The author uses stochastic optimal control theory to compare a proportional order-up-to policy with a full-state-feedback order-up-to policy. Necessary conditions for an optimum of a weighted sum of the inventory and the ordering variance for both policies are derived, and a relatively simple expression for the full-state-feedback policy is obtained. The comparison between the two policies demonstrates that the proportional policy does not fulfill the objective of controlling both the inventory and ordering variance for all parameter values of the demand model as well as the full-state-feedback policy. The author concludes that the full-state-feedback policy outperforms the proportional policy in several aspects.
Y.-J. Chen and S. Seshadri: Supply Chain Structure and Demand Risk
Agrawal and Seshadri  had considered a problem in which a single risk neutral distributor supplies a short-lifecycle, long-leadtime product to several retailers that are identical except in their attitudes towards risk. They proved that the distributor should not offer the same terms to every retailer but instead offer less risky (from the demand risk perspective) contracts to more risk averse retailers. They did not prove the optimality of their menu. This paper reconstructs their results when the number of retailers is infinite and their coefficient of risk aversion is drawn from a continuous distribution. It is shown that this distribution uniquely determines the channel structure. Moreover, the optimal contract menu not only has the same structure as in Agrawal and Seshadri but is also optimal among nearly all contracts. The implications of these findings for channel design are discussed.
T. Hosoda and S.M. Disney: The Governing Dynamics of Supply Chains: The Impact of Altruistic Behaviour
The authors analyze an infinite horizon two-echelon supply chain inventory problem and show that a sequence of the optimum ordering policies does not yield globally optimal solutions for the overall supply chain. First-order autoregressive demand pattern is assumed and each participant adopts the order-up-to (OUT) policy with a minimum mean square error forecasting scheme to generate replenishment orders. To control the dynamics of the supply chain, a proportional controller is incorporated into the OUT policy, which is termed a generalized OUT policy. A two-echelon supply chain with this generalized OUT policy achieves over 10% inventory related cost reduction. To enjoy this cost saving, the attitude of first echelon player to cost increases is an essential factor. This attitude also reduces the bullwhip effect. An important insight revealed in the paper is that a significant amount of benefit comes from the player doing what is the best for the overall supply chain, rather than what is the best for local cost minimization.
J.D. Schwartz, W. Wang and D.E. Rivera: Simulation-Based Optimization of Process Control Policies for Inventory Management in Supply Chains
In this paper, a simulation-based optimization framework involving Simultaneous Perturbation Stochastic Approximation (SPSA) is presented as a means for optimally specifying parameters of Internal Model Control (IMC) and Model Predictive Control (MPC)-based decision policies for inventory management in supply chains under conditions involving supply and demand uncertainty. The effective use of the SPSA technique serves to enhance the performance and functionality of this class of decision algorithms and is illustrated with case studies involving the simultaneous optimization of controller tuning parameters and safety stock levels for supply chain networks inspired from semiconductor manufacturing. The results of the case studies demonstrate that safety stock levels can be significantly reduced and financial benefits achieved while maintaining satisfactory operating performance in the supply chain.
A. Gosavi: A Risk-Sensitive Approach to Total Productive Maintenance
This paper is about modeling and control of a risk-sensitive preventive maintenance problem. It employs the mean-variance of the Markowitz paradigm in which one seeks to optimize a function of the expected cost and its variance. It provides a result for a risk-sensitive approach in the settings of renewal processes and a result for solving a risk-sensitive Markov decision process. Computational results are provided to demonstrate the efficacy of these results.
A. Bemporad, D. Muņoz de la Peņa, and P. Piazzesi: Optimal Control of Investments for Quality of Supply Improvement in Electrical Energy Distribution Networks
This paper considers the problem of deciding multi-period investments for maintenance and upgrade of electrical energy distribution networks. After describing the network as a constrained hybrid dynamical system, optimal control theory is applied to optimize profit under a complex incentive/penalty mechanism imposed by public authorities. The dynamics of the system and the cost function are translated into a mixed integer optimization model, whose solution gives the optimal investment policy over the multi-period horizon. While for a reduced-size test problem the pure mixed-integer approach provides the best optimal control policy, for real-life large scale scenarios a heuristic solution is also introduced. Finally, the uncertainty associated with the dynamical model of the network is taken care of by adopting ideas from stochastic programming.
G.C. Goodwin, M.M. Seron, R.H. Middleton, M.Zhang, B.F. Hennessy, P.M. Stone, and M. Menabde: Receding Horizon Control Applied to Optimal Mine Planning
The authors show that the problem of optimal mine planning can be cast in the framework of receding horizon control. Traditional formulations of this problem have cast it in the framework of mixed integer linear programming. They present an alternative formulation of the mine planning problem using the "language" of control engineering, and show that this alternative formulation gives rise to new insights which have the potential to lead to improved computational procedures. The advantages are illustrated by an example incorporating many practical features of an actual mine planning problem.
Five papers are in the area of marketing (three devoted to finding optimal advertising policies, one dealing with pricing to maintain brand image, and one concerning pricing and location of production decisions):
A. Buratto, L. Grosset, and B. Viscolani: Advertising Channel Selection in a Segmented Market
The authors consider a market with a finite number of segments and assume that several advertising channels are available, with different diffusion spectra and efficiencies. The problem of the choice of an advertising channel to direct the pre-launch campaign for a new product is analyzed in two steps. First, an optimal control problem is solved explicitly in order to determine the optimal advertising policy for each channel. Then a maximum profit channel is chosen. In a simulation example, the authors consider the choice of a newspaper among six available, and analyze the relations among the firm target market and the advertising channels environment which induce the optimal decision.
S. Jørgensen, P.M. Kort, and G. Zaccour: Advertising an Event
The paper considers a problem of how to minimize advertising costs to sell seats for a particular event, for instance, a sports game, a rock concert or a ballet performance. The authors take into consideration a word-of-mouth effect, which means that people buying a ticket tell their friends about it, so that advertising is unnecessary to inform those people. The number of seats sold and the advertising effort of the organizers are the state and control variables, respectively. The authors show that, besides being dependent on the cost and revenue parameters, the optimal advertising policy is also affected by the length of the planning period and the relation between the number of seats and the total number of potential attendees.
K. Raman: Boundary Value Problems in Stochastic Optimal Control of Advertising
Temporal patterns for advertising include constant spending over time, decreasing spending over time, and increasing spending over time. This paper shows that all these spending patterns emerge at optimality for the same response function dynamics, due to differences in salvage value assumptions. The author uses these results to develop a methodology for determining the optimal planning horizon length for each pattern of spending.
P.M. Kort, J.P. Caulkins, R.F. Hartl, and G. Feichtinger: Brand Image and Brand Dilution in the Fashion Industry
The authors consider the problem of a fashion designer’s challenge of maintaining brand image in the face of short-term profit opportunities through expanded sales that risk brand dilution in the longer-run. The key state variable is the brand’s reputation, and the key decision is sales volume. Depending on the brand’s capacity to command higher prices, one of two regimes is observed. If the price mark-ups relative to production costs are modest, then the optimal solution may simply be to exploit whatever value can be derived from the brand in the short-run and retire the brand when that capacity is fully diluted. However, if the price markups are more substantial, then an existing brand should be preserved. It may even be worth incurring short-term losses while increasing the brand’s reputation, even if starting a new brand name from scratch is not optimal.
G.E. Fruchter, E.D. Jaffe, and I.D. Nebenzahl: Dynamic Brand-Image-Based Production Location Decisions
This paper studies the dynamic location decision of a manufacturer of a brand-name product. Brand-image is a form of goodwill in the context of Nerlove-Arrow dynamic model which is extended by incorporating country-image and price. For a group of countries in which the cost of production is increasingly convex with country-image, the authors develop optimal decision rules regarding the location of production and pricing over time. The resulting optimal policy has a interesting pattern. Assuming that the demand rises by more than the value of the new brand-image in percentage terms, then, if brand image is increasing toward a stationary value level, the optimal policy is to initially locate production in countries with high image and set a high price that signals high quality. Later, the production should gradually shift to countries with lower production costs and lower image and the price lowered until the stationary value level is reached. For brand-images beyond the stationary value level, the location of production should start in a country with low costs and country image while setting prices that signal relatively low quality. Over time, production should be shifted to countries with gradually higher costs and images while setting higher prices until the brand-image approaches the level of stationary value.
Three papers are devoted to problems in finance (one on estimating the volatility of short-term interest rates, one devoted to finding optimal portfolio and capital inflows over time involved in bank management, and one providing an algorithm for pricing American options:
R. Bhar, C. Chiarella, H. Hung, and W. Runggaldier: The Volatility of the Instantaneous Spot Interest Rate Implied by Arbitrage Pricing - A Dynamic Bayesian Approach
This paper considers the estimation of the volatility of the instantaneous short interest rate from a new perspective. Rather than using discretely compounded market rates as a proxy for the instantaneous short rate of interest, the authors derive a relationship between observed LIBOR rates and certain unobserved instantaneous forward rates. They determine the stochastic dynamics for these rates under the riskneutral measure and propose a filtering estimation algorithm for a time-discretised version of the resulting interest rate dynamics based on dynamic Bayesian updating in order to estimate the volatility function. Their time discretisation can be justified by the fact that data are observed discretely in time. The method is applied to US Treasury rates of various maturities to compute a (posterior) distribution for the parameters of the volatility specification.
J. Mukuddem-Petersen and M.A. Petersen: Bank Management via Stochastic Optimal Control
This paper examines a stochastic control approach for a bank management problem. The goal is to minimize the market risk and risks related to the capital adequacy of the bank that involves the safety of the securities held and the stability of sources of funds. The paper provides an optimal portfolio choice and rate of bank capital inflow that keeps the level of loan issuing as close as possible to an actuarially determined reference process. The setup leads to a nonlinear stochastic control problem which can be solved via the dynamic programming approach.
K. Zhang, X.Q. Yang, and K.L. Teo: Augmented Lagrangian Method Applied to American Option Pricing
This paper treats pricing of an American option with an augmented Lagrangian method by discretizing the associated variational inequality. It provides an algorithm for valuation of American options together with its convergence. Using empirical numerical experiments, it also shows the effectiveness and robustness of the ALM.
The last paper in this special issue deals with a problem in environmental economics:
A. Haurie and F. Moresino: A Stochastic Control Model of Economic Growth with Environmental Disaster Prevention
This paper considers a capital accumulation model with a random stopping time corresponding to the occurrence of an environmental catastrophe. The damage cost associated with the catastrophe varies depending on the preventive capital stock accumulated at the time of the catastrophe. Turnpike theory is used to analyze the long term behavior of the optimal accumulation path. The case where the catastrophe process is uncontrolled is distinguished from the case where there is an anthropogenic effect on the probability of an occurrence. Intergenerational equity issues are also discussed.
We would like to thank all the authors who have submitted papers to this special issue, and the associate editors and the many reviewers who were involved in the refereeing of the manuscripts.
Suresh P. Sethi, Qing Zhang