Show simple item record

dc.contributor.authorJung, Tobias
dc.contributor.authorPolani, D.
dc.contributor.authorStone, Peter
dc.date.accessioned2013-01-15T12:29:01Z
dc.date.available2013-01-15T12:29:01Z
dc.date.issued2011-02
dc.identifier.citationJung , T , Polani , D & Stone , P 2011 , ' Empowerment for continuous agent-environment systems ' , Adaptive Behavior , vol. 19 , no. 1 , pp. 16-39 . https://doi.org/10.1177/1059712310392389
dc.identifier.issn1741-2633
dc.identifier.otherPURE: 392720
dc.identifier.otherPURE UUID: 0cd504d2-af99-4174-9a1c-54ca76b627b7
dc.identifier.otherWOS: 000287809500002
dc.identifier.otherScopus: 79952169043
dc.identifier.otherORCID: /0000-0002-3233-5847/work/86098045
dc.identifier.urihttp://hdl.handle.net/2299/9651
dc.description“The final, definitive version of this article has been published in the Journal Adaptive behavior, 149 (1) 2011 © SAGE Publications Ltd, 2011: on SAGE Journals Online: http://online.sagepub.com/" [Full text of this article is not available in the UHRA]
dc.description.abstractThis article develops generalizations of empowerment to continuous states. Empowerment is a recently introduced information-theoretic quantity motivated by hypotheses about the efficiency of the sensorimotor loop in biological organisms, but also from considerations stemming from curiosity-driven learning. Empowerment measures, for agent-environment systems with stochastic transitions, how much influence an agent has on its environment, but only that influence that can be sensed by the agent sensors. It is an information-theoretic generalization of joint controllability (influence on environment) and observability (measurement by sensors) of the environment by the agent, both controllability and observability being usually defined in control theory as the dimensionality of the control/observation spaces. Earlier work has shown that empowerment has various interesting and relevant properties, for example, it allows us to identify salient states using only the dynamics, and it can act as intrinsic reward without requiring an external reward. However, in this previous work empowerment was limited to the case of small-scale and discrete domains and furthermore state transition probabilities were assumed to be known. The goal of this article is to extend empowerment to the significantly more important and relevant case of continuous vector-valued state spaces and initially unknown state transition probabilities. The continuous state space is addressed by Monte Carlo approximation; the unknown transitions are addressed by model learning and prediction for which we apply Gaussian processes regression with iterated forecasting. In a number of well-known continuous control tasks we examine the dynamics induced by empowerment and include an application to exploration and online model learning.en
dc.format.extent24
dc.language.isoeng
dc.relation.ispartofAdaptive Behavior
dc.subjectInformation theory
dc.subjectlearning
dc.subjectdynamical systems
dc.subjectself-motivated behavior
dc.titleEmpowerment for continuous agent-environment systemsen
dc.contributor.institutionSchool of Computer Science
dc.contributor.institutionScience & Technology Research Institute
dc.description.statusPeer reviewed
rioxxterms.versionVoR
rioxxterms.versionofrecordhttps://doi.org/10.1177/1059712310392389
rioxxterms.typeJournal Article/Review
herts.preservation.rarelyaccessedtrue


Files in this item

FilesSizeFormatView

There are no files associated with this item.

This item appears in the following Collection(s)

Show simple item record