This book concerns continuous-time controlled Markov chains, also known as continuous-time
Markov decision processes. They form a class of stochastic control problems in which a …

This paper is a survey of recent results on continuous-time Markov decision processes (MDPs)
withunbounded transition rates, and reward rates that may beunbounded from above and …

We deal with a discrete-time finite horizon Markov decision process with locally compact
Borel state and action spaces, and possibly unbounded cost function. Based on Lipschitz …

We consider a Markov decision process (MDP) with constraints under the total expected
discounted cost optimality criterion. We are interested in proposing approximation methods of …

Given a Markov process, we are interested in the numerical computation of the moments of
the exit time from a bounded domain. We use a moment approach which, together with …

We present a new methodology for the numerical pricing of a class of exotic derivatives such
as Asian or barrier options when the underlying asset price dynamics are modeled by a …

This paper deals with continuous-time zero-sum two-person Markov games with denumerable
state space, general (Borel) action spaces and possibly unbounded transition and reward/…

We consider a discrete-time Markov decision process with Borel state and action spaces,
and possibly unbounded cost function. We assume that the Markov transition kernel is …

In this paper, we study discrete-time absorbing Markov Decision Processes (MDP) with
measurable state space and Borel action space with a given initial distribution. For such models, …

This paper deals with unichain ergodic continuous-time controlled Markov chains (CMCs)
with a denumerable state space and possibly unbounded reward rates as well as unbounded …