IJPAM: Volume 28, No. 2 (2006)

POINTWISE APPROXIMATIONS OF DISCOUNTED
MARKOV DECISION PROCESSES TO OPTIMAL POLICIES

Daniel Cruz-Suárez$^1$, Raúl Montes-de-Oca$^2$, Francisco Salem-Silva$^{3}$
$^1$División Académica de Ciencias Básicas
Universidad Juárez Autónoma de Tabasco
P.O. Box 5, Cunduacán, Tabasco, 86690, MEXICO
e-mail: daniel.cruz@basicas.ujat.mx
$^2$Departamento de Matemáticas
Universidad Autónoma Metropolitana-Iztapalapa
186 San Rafael Atlixco Avenue
Vicentina, México D.F., 09340, MEXICO
e-mail: momr@xanum.uam.mx
$^3$Facultad de Ciencias Físico Matemáticas
Benemérita Universidad Autónoma de Puebla
San Claudio y Rio Verde Avenue
San Manuel, CU, Puebla City, 72570, MEXICO
e-mail: fsalem@fcfm.buap.mx


Abstract.This paper deals with discrete-time Markov decision processes with Borel state and control spaces, with possibly unbounded costs and compact control constraint sets, and the expected total discounted cost criterion. Conditions that allow to detect a value iteration policy which is a pointwise approximation to the optimal policy are given. Besides, two illustrative examples are supplied.

Received: April 6, 2006

AMS Subject Classification: 90C40

Key Words and Phrases: discounted Markov decision process, optimality equation, value iteration algorithm, uniqueness of the optimal policy, approximation to the optimal policy

Source: International Journal of Pure and Applied Mathematics
ISSN: 1311-8080
Year: 2006
Volume: 28
Issue: 2