Poster
Regret Bounds for Learning State Representations in Reinforcement Learning
Ronald Ortner · Matteo Pirotta · Alessandro Lazaric · Ronan Fruit · Odalric-Ambrym Maillard
East Exhibition Hall B, C #196
Keywords: [ Markov Decision Processes ] [ Reinforcement Learning and Planning ] [ Computational Complexity ] [ Algorithms -> Model Selection and Structure Learning; Algorithms -> Representation Learning; Theory ]
We consider the problem of online reinforcement learning when several state representations (mapping histories to a discrete state space) are available to the learning agent. At least one of these representations is assumed to induce a Markov decision process (MDP), and the performance of the agent is measured in terms of cumulative regret against the optimal policy giving the highest average reward in this MDP representation. We propose an algorithm (UCB-MS) with O(sqrt(T)) regret in any communicating Markov decision process. The regret bound shows that UCB-MS automatically adapts to the Markov model. This improves over the currently known best results in the literature that gave regret bounds of order O(T^(2/3)).
Live content is unavailable. Log in and register to view live content