Object Empowerment-Driven Tool Selection in Reinforcement Learning

Tool use is a hallmark of intelligent behavior, both in animals and humans, enabling problem-solving beyond immediate sensorimotor capabilities. Yet, the cognitive and computational mechanisms that give rise to and govern effective tool use remain only partially understood in both cognitive science and artificial intelligence. Discovering and mastering tool use presents significant challenges for learning systems, because it involves delayed rewards and multi-step behaviors whose benefits are not immediately obvious. This has prompted calls for additional intrinsic drives or biases that can can guide learning systems toward the discovery of complex skills like tool use. In this paper, we investigate how artificial agents can successfully learn to use tools to interact with the environment by optimizing object-centric intrinsic motivations, specifically object empowerment. Object empowerment measures an agent's potential influence over specific objects of the environment and provides a grounded signal for discovering functional tool-object relationships. Through reinforcement learning (RL) experiments in grid world-like environments featuring multiple tools and objects, we demonstrate that object empowerment facilitates effective tool selection, and supports generalization to multi-object tasks. Moreover, it reveals the ability of tools to exert a never-ending influence over an object and the range of their interaction. Finally, we show that agents guided by object empowerment learn more efficiently in sparse reward conditions than vanilla RL agents.