Using BERT to Generate Contextualised Textual Images for Sentiment Analysis

Sentiment Analysis could be performed on textual data and indicates the general ‘tone’ or emotional state of the writing. It is important in business, for instance in marketing, to determine customer opinions and trends, and in analysing social media to help weed out inappropriate or discriminatory language. Recently improved performance has been obtained by first converting the text to a grayscale image and then using a BLSTM and deep CNN, specifically ResNet, to classify the data. This paper investigates the addition of more context to the original text using a pre-trained BERT model to produce contextualised textual images. This produces a marked improvement over the previous results. The proposed BERT-BLSTM-ResNet model outperforms the BERT model on smaller datasets and above a threshold data size, the BERT performance is comparable.