TL;DR
It is shown that the approximate value functions obtained by value iteration and LP asymptotically converge to the optimal value function, and the policies generated by policy iteration converge in value to an optimal policy.
Motivated by many application problems, we consider Markov decision processes (MDPs) with a general loss function and unknown parameters. To mitigate the epistemic uncertainty associated with unknown parameters, we take a Bayesian approach to estimate the parameters from data and impose a coherent risk functional (with...
To the best of the knowledge, this work provides the first non-asymptotic convergence rates for solving infinite-horizon discounted inventory systems with Markov-modulated demand and stochastic cash-balance problems using policy gradient methods.
A Taylor series expansion is derived for the invariant probability measure of the singularly perturbed Markov chain and this expansion is applied to analyze the expected long-run average cost (EAC) for singularly perturbed Markov decision processes (MDPs).
Combining the policy-iteration error with a quantitative soft-max approximation bound, it is shown that the iterated policies constitute weighted $\varepsilon$-equilibria for the original unregularized problem and derive an explicit regret estimate.
It is shown that AMR is asymptotically optimal such that the sequence of the expected absolute errors approaches zero and its convergence rate depends on the number of visits to each reachable state at each stage from the initial state, essentially transforming the result of AMS into the RL setting.
We study linear programming (LP) representations and strongly polynomial algorithms for robust Markov decision processes (RMDPs) with rational polyhedral state-action rectangular uncertainty in rewards and transitions. By encoding a finite sequence of robust policy-iteration steps, we construct a single LP whose optima...
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
