Emotion in Human-Computer Interaction
Scott Brave and Cliff Nass

1. What are emotions
   • Two defining aspects:
     • Somatic feelings (physiological response)
     • Affective feelings (valence, activation/arousal)
     • The necessity of these two in defining emotion is generally agreed upon, but whether they are sufficient to distinguish among emotions is not.
   • There is no accepted definition of emotion. Researchers often resort to:
     • Physiological definitions
       Emotion is simply Somatic + Affective feelings
     • Cognitive definitions
       Classic James-Lange theory: Emotion is the interpretation of physiological state (e.g. epinephrine experiments) (James 1884)
       Shweder 1994: Emotions are complex narrative structures that give shape and meaning to somatic and affective experiences.
     • Functional definitions
       Emotions prepare an organism to respond to situations that impact its needs/goals/concerns
       Hardwired vs. learned (universal vs. person-specific)
     • Integrated systems / holistic definitions
   • Process of Emotion
     • Regardless of what "defines" emotion, there seems to be a generally accepted "process" relating to emotion
     • Assessment of situation as important to goals/motives/concerns
       These can be precognitive or cognitive evaluations
     • Involuntary reactions (physiological and cognitive preparation for action) - quick onset
     • Behavior
     • Conscious reflection
       Can occur before and/or after behavior to varying degrees
       Can account for individual differences in experience of "basic" emotions
       Explaining misattribution: If the antecedent of the emotion is conscious assessment, the causes are known, consciously. However, If the antecedent of the emotion was preconscious, causes are inferred.
   • Emotions vs. moods
     • Emotions: short-term, object-focused, affect action
     • Moods: longer-term, diffuse, affect cognition
     • Moods are an affective background upon which emotions can occur
     • Emotions vs. sentiment (likes/dislikes)
       Sentiments are judgments of potential emotional reaction
       These judgments can be based on direct experience or inferences based on norms, schemas, etc. (e.g. similarity`liking)
       Interesting theory: We make judgments of like/dislike by bringing object/person to mind and observing our emotional reaction
       [Mention attribution theory?]
2. Causes of emotion
   • By understanding causes, we can better achieve desired/positive emotional states in an HCI context
   • Needs and Goals
     • Basic (survival/reproduction)
     • Social
       Media Equation: if we see computer as a social actor, then we must take social goals into account (e.g. I want to be appreciated, liked, part of the group, etc.)
     • Maslow's hierarchy
     • In HCI:
       Specific to application (e.g. I need to get this done, efficiency)
       Goals for self (e.g. I am computer literate, smart, I can do this, I want to feel comfortable)
     • Emotional Intelligence
   • Abstract Elicitors of Emotion (Ellsworth 1994) and Applications to HCI
     • Novelty/Strangeness
     • Valence
     • Certainty/Uncertainty
       Uncertainty ` (+): curiosity, hope (-): anxiety, fear
       Certainty ` (+): relief (-): despair
     • Obstacles/Control
       Challenge vs. frustration/fear
       Desperation, resignation
     • Agency
       Source = self ` Shame, Pride
       Source = other ` Anger, Love
       Source = fate ` Sorrow, Joy
     • Norm/Self-concept Compatibility
       Social, violation of standards, ostracism
   • Sentiments (like/dislike)
     • Like/dislike is often judged prior to any experiential evidence
     • What heuristics/schemas do people use to make these judgments? (e.g. similarity, relationship structure, in/out group)
     • These initial like/dislike judgments can also bias actual emotional reactions (disposition)
     • Applications to HCI
3. How emotions are manifested/measured
   • Physiological responses
     • Heartrate
     • Respiration
     • Galvanic skin response (GSR)
     • Blood volume pressure (BVP)
     • Muscular tonus
     • Endocrine activity
   • Feelings (affective response)
     • Questionnaire
   • Facial expression
     • Video capture (e.g. Facial Action Coding (FAC) - Ekman)
     • EMG (Electromyography)
   • Voice
   • Brain activity?
   • Behavior (inferences through emotion modeling)
     Gestures
     Actions
   • Should we measure invisible manifestations of emotion?
     • People often intentionally suppress emotional expression, for reasons of:
       Social communication
       Self-regulation of emotional state
     • Psychological consequences of physiological monitoring: invasion vs. trust
4. HCI: what do we do with this information?
   • Change content (functionally or aesthetically (complexity, color))
     • To suit current emotional state
     • To modify current emotional state
     • Use emotion as measure of appropriateness/liking/acceptability of current content
   • Dealing with user frustration
     • Give/remove help
   • Bring in third party agent to diffuse negative emotions / foster positive emotions (requires emotional intelligence)
     • Modification of emotion through social means
5. Can Computers display emotion?
   • Synthesizing facial expression
   • Behavior/gesture
   • TTS - tone of voice
   • Emotional Intelligence
6. Should Computers display emotions?
   • Media Equation
     • We already infer emotional states of computers/agents
     • What are they currently displaying emotionally?
   • How do humans respond to emotions in others?
7. Effects of emotion
   • Alters attention
     • Moves attention toward things relevant to goals and concerns implicated by emotion
     • Attention affects memory
   • Biases judgment
     • Affective bias
     • Arousal transfer
     • Activates relevant associative networks in memory
   • Motivates behavior
     • Shift certain behaviors upward in response hierarchy
     • Activates approach/avoidance behaviors
   • Irrationality
     • People are bad at distinguishing between emotional reactions to a current situation and past emotions that are evoked because the current situation is reminiscent of a past situation
     • Effects of emotions often last even after circumstances change
     • Often emotions tap very basic animalistic needs/desires, so may lead to "irrational" behavior and "loss of perspective"
     • Emotions can short-circuit cognitive processing
   • Affects cognitive style (Clore 1994)
     • Positive affect -->
       More heuristic processing
       More likely to use schemas, scripts, stereotypes
       Less focus on detail
     • Negative affect -->
       Systematic processing
       More analytical
       More focus on detail
   • Affects future interactions
     • Updates sentiment (like/dislike) of objects/people involved in evoking the emotion
     • Classical conditioning: associating emotion with object/situation
   • Signals/Affects other people (Media Equation: so we expect it to affect computers/agents)