Memristive PAD three-dimensional emotion generation system based on D-S evidence theory

Abstract

In this work, a Pleasure–Arousal–Dominance (PAD) three-dimensional brain-like emotion generation system is proposed by simulating the brain tissue structures involved in emotion generation in the brain’s limbic system. The system utilizes volatile memristors to simulate the activation and recovery process of neurons, and non-volatile memristors to simulate the synaptic weight changes. It combines the brain emotion learning model and the biological long short-term memory model to simulate the emotion generation process in the brain. The system employs the Dempster–Shafter (D–S) evidence theory for multimodal feature fusion, ultimately representing the generated human-like emotions in the PAD three-dimensional emotion expression space. Considering the differences in emotional information represented in each dimension of the PAD emotion expression space, this work proposes the use of the D–S evidence theory to calculate the weight values of multimodal evidence and each dimension of emotion signals. The system performs weighted summation for multimodal feature fusion, which is more biologically inspired and realistic. As a result, the generated emotion signals are more accurate, and the PAD three-dimensional emotion expression model enhances the capability and richness of emotion expression. The system processes multimodal input signals (text, speech, visual signals) to generate three-dimensional emotion signals (pleasure, arousal, and dominance signals), which correspond to specific emotions in a three-dimensional space. These signals can be visually represented as facial images using MATLAB. The simulation results from PSPICE indicate a nonlinear mapping relationship between the system’s input and output. It shows that different inputs can generate distinct human-like emotions.