Hierarchical Optimal Control of MDP's
McGovern, Amy , Doina Precup, Balaraman Ravindran, Satinder Singh, Richard S SuttonHierarchical Optimal Control of MDP's
Proceedings of the 10th Yale Workshop on Adaptive and Learning systems.
( gzipped Postscript - 600 KB )
Abstract: In this paper we survey a new approach to reinforcement learning
in which high and low-level decisions are treated uniformly. Each low-level
action and high-level couse of action is represented as an "option,"
a (sub)controller
and termination condition. The theory of options is based on the
theories of of Markov and semi-Markov decision processes, but extends
these in significant ways.