As an agent’s representation of the state of the environment will affect how well it learns, an interesting research question is how an agent might autonomously change its representation to learn better. Although many researchers recognize that the choice of feature representation affects learning, it is often considered the responsibility of the human implementer to construct a good feature-set for a particular learning task. However, for a long-lived autonomous agent that will solve many related tasks that have not been fully specified before its deployment, the agent should be able to adapt and construct better features from its experience.

A substantial challenge is to have algorithms operate online at the same fast time scale as the agent’s behaviour and learning processes. Although dimensionality reduction techniques such as principal components analysis are popular, dimensionality expanding algorithms are potentially more powerful when abundant data is available. We are examining a variety of techniques to see how better representations might be fruitfully constructed from an agent’s sensorimotor experience. One approach is to construct features at random and see which perform well. Another approach is to construct new features that respond at times in the agent’s experience of high predictive error. A third approach is to construct features that reveal structure in the agent’s sensorimotor experience and enable qualitatively different kinds of generalization.