IJPAM: Volume 58, No. 1 (2010)

OPTIMAL POLICIES IN THE CLASS OF INFINITELY
DIFFERENTIABLE FUNCTIONS FOR DISCOUNTED
LINEAR-QUADRATIC MODELS

Raúl Montes-de-Oca$^1$, Hugo Cruz-Suárez$^2$
$^1$Departamento de Matemáticas
Universidad Autónoma Metropolitana-Iztapalapa
186, San Rafael Atlixco Av., Vicentina, México, D.F., 09340, MEXICO
e-mail: momr@xanum.uam.mx
$^2$Facultad de Ciencias Físico-Matemáticas
Benemérita Universidad Autónoma de Puebla
San Claudio Av., San Manuel, Puebla, 1152, MEXICO
e-mail: hcs@fcfm.buap.mx


Abstract.This paper deals with one-dimensional linear-quadratic (LQ) models. These models are presented as Markov decision processes with the total discounted cost as the objective function. The discussion about the determination of the optimal policy of LQ models is divided into two cases: the analysis of the deterministic LQ models, and the analysis of the stochastic case. Specifically, (a) assuming that the optimal value function of the deterministic LQ models is the class $C^{\infty }$, the optimal policy is obtained by means of the Euler equation, and (b) the optimal policy for stochastic LQ models is obtained from the dynamic programming equation (DPE), using as a fixed point of the DPE, the optimal value function for the deterministic case adjusted with a suitable additive constant.

Received: December 15, 2009

AMS Subject Classification: 90C40, 93E20

Key Words and Phrases: linear-quadratic model, discounted Markov decision process, optimal policy, Euler equation, Taylor's series around an equilibrium point

Source: International Journal of Pure and Applied Mathematics
ISSN: 1311-8080
Year: 2010
Volume: 58
Issue: 1