Emotional State in Pictures: How People Recognize Emotions in Images

Contents:

• How we identify emotions in images: scientists' theories
• Why context is important: visual language and the Kuleshov effect
• What helps to identify emotions more accurately: a scientific experiment
• Why emotions are needed in commercial design: an art director's opinion

Researchers from the HSE Laboratory of Cognitive Psychology of Digital Interface Users conducted a scientific experiment to study how people perceive emotions depicted in character drawings. The goal of the study was to determine which graphic elements most influence the emotional perception and interpretation of the depicted characters. The results of the experiment can help to further improve the design of digital interfaces and increase their emotional expressiveness.

This article provides a brief overview of the theories and results of the HSE experiment. The findings will be especially useful for graphic designers and illustrators seeking to improve the quality of their work and master new approaches in visual art.

How We Identify Emotions in Images: Scientists' Theories

People primarily recognize emotional reactions visually. There are various hypotheses about what exactly contributes to this process: some researchers argue that individual elements of facial expression, such as the eyes, mouth, and cheeks, play a key role. Others believe that considering the entire body is essential for accurately perceiving emotions. Understanding these factors can help improve interpersonal communication and emotional intelligence.

Let's explain with specific examples.

Facial features and facial expressions form unique combinations that are easily recognized by most people. Each facial element serves as a source of information for our brain, and cognitive processes allow us to process these combinations and categorize them. Understanding facial expressions and facial expressions is essential in communication, as they help convey emotions and intentions. Effective perception of these signals plays a key role in interactions between people.

Scientists have conducted numerous studies to understand exactly how this process occurs, and as a result, they have identified two main approaches to its analysis.

• Feature-based: individual facial elements—the mouth or eyes—are sufficient for emotion recognition.
• Holistic: only the entire face acts as a source of information about emotions.

Scientific research supports both the feature-based and holistic approaches to emotion recognition. It is likely that the mechanism of emotion perception is actually combined. Cognitive processes do not depend solely on facial expressions but are also influenced by various factors, such as context, cultural characteristics, and individual differences. This highlights the importance of a holistic approach to studying emotions and recognizing them.

A holistic approach implies that the perception of emotions occurs not only through facial expressions, but also through awareness of body posture. Analysis of non-verbal cues, such as gestures and facial expressions, plays a key role in understanding a person's emotional state. It's important to consider that every detail, including the position of our arms and legs, can significantly alter our perception and interpretation of emotions. Using a holistic approach, we can more deeply understand emotional communication and interaction between people. Research on the recognition and categorization of emotions based on body posture leads to diverse conclusions. Research shows that body posture plays a significant role in interpreting a person's emotional state. Emotions can be expressed not only through words but also through gestures, facial expressions, and posture. Scientists note that people intuitively read others' emotions based on their physical expressions, highlighting the importance of nonverbal communication. Different cultures may have their own peculiarities in interpreting these signals, which opens new perspectives for the study of emotional literacy and intercultural communication.

• Like facial expressions, body postures associated with basic emotions are recognized even when a facial expression is not depicted.
• The accuracy of emotional signals from the body is comparable to the accuracy of emotion recognition from the face, and can sometimes be even higher.
• Body posture and facial expressions are more likely to be processed as a whole.
• Evaluation of the spinal position of drawn figures did not lead to a strong correlation between the recognition of body postures and specific emotions. Respondents associated one spinal position with a whole range of emotional states.

This finding indicates that although body posture does influence emotional assessment, it does not play as significant a role as facial expressions. Research shows that facial expressions are a key element in the perception of emotions. Therefore, when analyzing emotional reactions, more attention should be paid to facial expressions than to body posture.

When body posture and facial expressions are consistent and convey the same emotion, people are able to recognize that emotion more quickly. The importance of congruence in nonverbal communication is undeniable, as it promotes more effective interaction and understanding between people.

In drawn images, especially in comics, an important element is the conveyance of emotion and narrative support. For this purpose, various graphic symbols are used, such as dashes, asterisks, wavy lines, and other signs that are located around the heads or bodies of characters. These visual elements are called emanates and play a key role in expressing the feelings and mood of the characters, helping readers better understand their state. Effective use of emanates in comics enriches the plot and makes it more expressive.

Research on visual emanates and their perception by people has been conducted by various groups of scientists. These studies have revealed significant aspects of human interaction with visual images and their effect on information perception. Scientists note that visual emanates can significantly influence people's emotional state and behavior. The results of these studies emphasize the importance of visual elements in communication and their role in shaping perception. Understanding this process can help develop more effective methods for visually representing information, which is especially relevant in the fields of marketing, design, and education.

• Individual emanata do not symbolize specific emotions for people—each icon is endowed with dozens of meanings. For example, a droplet next to a head in an image might represent sweat, tears, or simply rain.
• People tend to associate these symbols with states rather than emotions: for example, an image of a spiral represents confusion. However, emanata help people recognize emotions in images.
• In general, emanata do not systematically categorize emotions. However, there was an experiment that showed that a vertical line drawn over an image of a face helps to recognize sadness.

Emanata is an additional element that enhances The emotional state of the character. However, this factor is not enough to accurately recognize emotions in an image. Effective interpretation of emotions requires a deeper analysis, including the context of the image and visual cues that help the viewer better understand the character's internal state.

Why Context Matters: Visual Language and the Kuleshov Effect

There is a theory of visual language developed by the American scientist Neil Cohn. According to this theory, images contain elements similar to those used in speech, such as suffixes, prepositions, and others. Visual "linguistic" elements are represented by various details in images, for example, lines that symbolize rapid movement, or an image of a light bulb above a head indicating an idea. Thanks to such graphic morphemes, artists and designers can create dynamic narratives even in static images. This concept emphasizes the importance of visual language in communication and understanding, allowing us to convey complex ideas through simple visual means.

Individual drawn elements do not form a complete story. We are able to perceive the visual language and meaning of an image only in the presence of grammar, which in this case is context. Without context, elements lose their significance and cannot effectively convey an idea or emotion. The correct combination of visual elements and context creates a holistic perception, allowing us to more deeply understand the author's intent.

Context, whether background, situation, or text, significantly influences our perception of emotion in illustrations. An example of this is the Kuleshov effect, which demonstrates how the perception of an image can change depending on the environment and the content presented. This effect shows that the same illustration can evoke different emotional responses in viewers depending on the context given to it. Understanding this dependence is key to creating effective visual content and its perception.

In the 1920s, Lev Kuleshov conducted a landmark experiment that clearly demonstrated how editing influences narrative perception, the meaning of scenes, and the emotional response of viewers. He used a shot of an actor's neutral expression, which alternated with various images: a bowl of soup, a girl in a coffin, and a beautiful woman on a couch. Although the shot of the actor remained unchanged, viewers interpreted his emotions differently: in the context of food, they felt impatience or anticipation, and in the context of other shots, sorrow or joy. This experiment became the basis for understanding how editing can shape the emotional context and perception of storytelling in film.

Context plays a key role in the perception of emotion. People are able to recognize emotional states even in images where it's not obvious. When an illustration combines rich narrative and visual elements such as facial expressions, postures, and emotional cues, viewers can identify emotional expressions much more quickly and accurately. This adds depth and layering to the image, allowing for the creation of richer and more memorable visual stories.

What helps to identify emotions more accurately: a scientific experiment

Olga Rubtsova and Elena Gorbunova from the National Research University Higher School of Economics conducted a behavioral experiment to find out how people perceive and classify emotions in images. The researchers hypothesized that emotion perception depends on various factors, such as the context of the image and the individual characteristics of the respondents. In their study, they used a variety of visual stimuli to analyze how participants react to different emotional states. The results of the experiment may have a significant impact on our understanding of emotional communication and the perception of visual content.

The more elements that convey information about emotions, the more accurate their identification should be. Emotions play a key role in communication, and their correct recognition requires attention to detail. Effective emotion identification relies on a combination of verbal and nonverbal cues, which allows for a deeper understanding of the emotional state. This knowledge is especially important in psychology, marketing, and other fields where interaction with people is a key aspect of the work. The better we understand emotions, the more successfully we can interact and build relationships.

Emanata, being one of the most ambiguous factors, can lead to an increase in the number of errors. Misunderstanding or misinterpretation of this element can significantly affect the final results. Therefore, it's important to pay special attention to the emanate and its influence on processes to minimize potential errors and ensure more accurate results.

Narrative context facilitates a more accurate identification of emotions. Arranging information in narrative form helps us better understand and interpret emotional states. Stories create a coherent context that allows people to more easily empathize and identify with the experiences of others. Thus, the use of narrative context not only deepens the perception of emotions, but also makes communication more effective and rich.

Participants in the study were asked to recognize emotions such as anger, surprise, fear, happiness, sadness, relief, and embarrassment based on images containing these emotional manifestations.

• facial expressions, emanation, posture separately;
• facial expressions and emanation;
• facial expressions and posture;
• posture and emanation;
• facial expressions, emanation, posture together.

The experiment demonstrated the following results:

• Emanata turned out to be the most dubious element: respondents used them to identify emotions the least accurately and the least quickly. It's likely that emanate in pictures doesn't improve emotion categorization at all.
• Respondents also didn't accurately interpret specific information about emotions from body posture—but only in the absence of context.

The lack of context in illustrations increases people's cognitive load and reduces the accuracy of emotion interpretation. When visual elements aren't accompanied by necessary information, viewers are forced to expend more effort to understand, which can negatively impact their perception. This underscores the importance of contextualizing images to improve emotion understanding and reduce cognitive effort.

• Depicting facial expressions in the absence of context helped respondents read emotions more accurately.
• When the picture included the face, posture, and emanate, this significantly improved emotion perception. Individually and in pairs, these elements yielded roughly the same lackluster results.
• Indeed, context in any case makes the perception of emotions more accurate.

Why emotions are needed in commercial design: the opinion of the art director

The art director of Dotorg, a company specializing in branding and digital solutions, plays a key role in shaping the visual identity and strategy of brands. He is responsible for the conceptual development and implementation of creative projects that help clients stand out in the marketplace. An important task of the art director is to create a unique and memorable image that reflects the brand's values ​​and mission. With a deep understanding of modern trends in design and digital technologies, the Dotorg art director ensures high-quality visual materials and their alignment with clients' business goals. Working closely with teams of designers, marketers, and developers, the art director contributes to the successful implementation of projects and the maximum impact of advertising campaigns.

As a designer, I value the opportunity to work with illustrations and actively use them in my projects. Illustrations not only visually enrich content but also convey a unique atmosphere, making the design more appealing and memorable.

When discussing commercial illustrations, it is necessary to consider not only individual graphic elements but also many other factors. Key aspects include the company's field of activity, the target audience, and the client's readiness for innovation. It's also important to understand whether the illustration will be presented offline or online. Effective commercial illustrations should be tailored to the specific conditions and needs of the business to maximize the target audience's attention and achieve marketing goals.

Illustration plays a key role in consumer branding, especially in packaging and app design. Using mascots or product-related scenes creates vibrant images that evoke real emotions and reflect clear interaction scenarios. This is especially effective for attracting target audiences such as children and teenagers, as an emotional connection with a brand fosters loyalty and memorability. Properly chosen illustrations can significantly enhance a product's appeal and differentiate it from competitors.

In corporate branding, the time has come to utilize illustration as an important tool. However, instead of traditional mascots, consider metaphorical 2D illustrations and abstract 3D animations that convey the essence of the company rather than specific emotions. The trend for neutral flat illustrations is already outdated, and this is a great opportunity for brands! Modern companies are striving to move away from "mature" corporate styles and return to more playful and creative visual solutions that create an atmosphere and evoke positive associations. Using bright and expressive illustrations helps differentiate a brand from competitors and attract the attention of its target audience. Awkward yet sincere illustrations are currently popular. Children are able to express their feelings with great honesty, and their creativity attracts the attention of major brands. For example, a wolf's pink nose looks very cute, and a wonderful pink poodle with a surprised look makes you want to have a T-shirt like that in your wardrobe. These examples highlight the value of children's art and its influence on contemporary design.

The art director at Dotorg, a company specializing in branding and digital solutions, plays a key role in shaping the company's visual identity and strategy. The art director's responsibilities include not only developing creative concepts but also managing a team of designers and interacting with clients to create effective brands. An important aspect of the job is analyzing current trends in design and digital technologies, which allows them to offer relevant and innovative solutions. Dotorg's specialization in branding and digital solutions makes the Art Director an indispensable team member, contributing to the success and competitiveness of projects.

As a designer, I value the opportunity to work with illustrations and actively use them in my projects. Illustrations not only add uniqueness but also help convey ideas more effectively. They play a vital role in visual communication, making content more engaging and memorable.

When discussing commercial illustration, the key consideration is not only individual graphic elements. Everything matters: the company's industry, the target audience, the client's willingness to embrace innovative solutions, and the chosen medium, whether offline or online. Successful commercial illustration must consider all these factors to achieve maximum impact and attract the attention of the target audience. The right approach to creating illustrations can significantly increase brand recognition and its competitiveness in the market.

Illustration plays a key role in consumer branding, especially in packaging and apps. Using mascots and product-related scenes creates vivid images that evoke real emotions. These visual elements help establish an emotional connection with the brand, which is especially important if the target audience includes children and teenagers. Effective illustrations can make a brand more memorable and appealing, helping to increase consumer loyalty.

In corporate branding, the time has come to actively use illustration as a powerful tool. The emphasis should be shifted from mascots to metaphorical 2D illustrations and abstract 3D animations that help convey a company's unique image rather than specific emotions. The trend for neutral flat illustrations has finally lost its relevance. We are tired of the "mature" corporate world and are striving to return to more playful and vibrant visual solutions that evoke childhood and fresh ideas.

Currently, sincere and imperfect illustrations are popular. Children, with their pure emotions, create graphics that attract the attention of even major brands. For example, the wolf's pink nose looks very cute. Also worth mentioning is the adorable pink poodle with a surprised look—a T-shirt like this would be a great addition to your wardrobe. Sincere children's drawings resonate with many people and are becoming a trend in the world of fashion and design.

Learn more about design by subscribing to our Telegram channel. Here you will find interesting materials, tips, and current trends in the world of design. Don't miss the opportunity to expand your knowledge and be inspired by new ideas.

• Archetype Theory in Design and Marketing: What You Need to Know
• Gestalt Principles in UX. Part 1
• Scientists have discovered how website colors influence user trust
• Scientists have invented a font that makes information more memorable. Is it true?