Back to the Future: Using Abstract Behaviors to Guide Short-Term Memory Selection in a Learning Task

Graduate

In many tasks, what we perceive at a given moment may not give us enough information to correctly decide what to do. For example, consider the task of understanding a person's gestures when I cannot see entire person with one glance. I may have to look at the person's face first, and then look to see what their hands are doing. At one moment, I may see that the person's hand is pointing at me. However, I do not understand that person's intent unless I remember that she had been smiling. In that case, I should probably smile back. However, if she had been scowling, I would want to plan an escape. In either case, my current observation had to be coupled with a previous observation to choose an appropriate action. Additionally, I somehow knew that I did not need to remember, say, the person's eye-color to determine their intent. This problem we face when two current situations look the same but require different actions is known as "perceptual aliasing." Not only is having some memory of previous experience useful for handling perceptual aliasing, but referring back to the right kind of experience is crucial. In this work, we show that decomposing a learning task into a hierarchy of behaviors helps a simulated robot to store experience at a useful level of abstraction. We apply our technique to a task in which a robot must learn to navigate around a corridor environment. The robot uses its abstract behaviors as a guide to storing those experiences relevant to the navigation task and ignoring experiences that are irrelevant. We show that this robot has greater success at learning the task than a robot without the behavioral structure and a robot without memory.

Return to Workshop Listing