FeelingBlue: A Corpus for Understanding the Emotional Connotation of Color in Context
Published
2023-03-14
Amith Ananthram
,
Olivia Winn
,
Smaranda Muresan
Amith Ananthram
Columbia University
Olivia Winn
Columbia University
Smaranda Muresan
Columbia University
Abstract
While the link between color and emotion has been widely studied, how context-based changes in color impact the intensity of perceived emotions is not well understood. In this work, we present a new multimodal dataset for exploring the emotional connotation of color as mediated by line, stroke, texture, shape and language. Our dataset, FeelingBlue, is a collection of 19,788 4-tuples of abstract art ranked by annotators according to their evoked emotions and paired with rationales for those annotations. Using this corpus, we present a baseline for a new task: Justified Affect Transformation. Given an image I, the task is to 1) recolor I to enhance a specified emotion e and 2) provide a textual justification for the change in e. Our model is an ensemble of deep neural networks which takes I, generates an emotionally transformed color palette p conditioned on I, applies p to I, and then justifies the color transformation in text via a visual-linguistic model. Experimental results shed light on the emotional connotation of color in context, demonstrating both the promise of our approach on this challenging task and the considerable potential for future investigations enabled by our corpus. Our dataset, code and models are available at https://github.com/amith-ananthram/feelingblue.
Presented at ACL 2023
Article at MIT Press