The paper deals with a class of discrete-time stochastic control processes under a discounted optimality criterion with random discount rate, and possibly unbounded costs. The state process { x_{t} } and the discount process { \alpha_{t} } evolve according to the coupled difference equations x_{t+1}=F(x_{t},\alpha _{t},a_{t},\xi _{t}), \alpha_{t+1}=G(\alpha _{t},\eta _{t}) where the state and discount disturbance processes { \xi _{t} } and { \eta _{t} } are sequences of i.i.d. random variables with densities \rho^{\xi } and \rho^{\eta } respectively. The main objective is to introduce approximation algorithms of the optimal cost function that lead up to construction of optimal or nearly optimal policies in the cases when the densities \rho ^{\xi } and \rho ^{\eta } are either known or unknown. In the latter case, we combine suitable estimation methods with control procedures to construct an asymptotically discounted optimal policy.
This paper deals with a class of partially observable discounted Markov decision processes defined on Borel state and action spaces, under unbounded one-stage cost. The discount rate is a stochastic process evolving according to a difference equation, which is also assumed to be partially observable. Introducing a suitable control model and filtering processes, we prove the existence of optimal control policies. In addition, we illustrate our results in a class of GI/GI/1 queueing systems where we obtain explicitly the corresponding optimality equation and the filtering process.
We are concerned with a class of GI/GI/1 queueing systems with controlled service rates, in which the waiting times are only observed when they take zero value. Applying a suitable filtering process, we show the existence of optimal control policies under a discounted optimality criterion.