Adaptive Smart Environments: Detecting Human Behaviour from Multimodal Observation

Abstract

It is desirable to enhance the social capabilities of a smart home environment to become more aware of the context of the human occupants’ activities. By taking human behavioural and contextual information into account, this will potentially improve decision making by the various smart house systems. Full mesh Wireless Sensor Networks (WSN) can be used for passive localisation and tracking of people or objects within a smart home. By monitoring changes in the propagation field of the monitored area from the link quality measurements collected from all the nodes of the network, it is feasible to infer target locations. It is planned to apply techniques from Radio Tomographic Imaging (RTI) and machine vision methods, adapted to the idiosyncrasies of RTI, which will facilitate real-time multiple target tracking in the University of Hertfordshire Robot House (UHRH). Using the Robot Operating System (ROS) framework, these data may then be fused with concurrent data acquired from other sensor systems (e.g.) 3-D video tracking and ambient audio detection in order to develop a high level contextual data model for human behaviour in a smart environment. We present experimental results which could provide support for human activity recognition in smart environments.