Book contents
- Frontmatter
- Contents
- Preface
- Contributors
- Abbreviations
- Introduction
- Part 1 General aspects of visual cues in CMC
- Part 2 Video- and avatar-based communication
- Part 3 Emotions and visual cues in HCI
- 9 Emotions in human–computer interaction
- 10 Embodiment and expressive communication on the internet
- Index
- Studies in Emotion and Social Interaction
- References
10 - Embodiment and expressive communication on the internet
Published online by Cambridge University Press: 05 June 2012
Book contents
- Frontmatter
- Contents
- Preface
- Contributors
- Abbreviations
- Introduction
- Part 1 General aspects of visual cues in CMC
- Part 2 Video- and avatar-based communication
- Part 3 Emotions and visual cues in HCI
- 9 Emotions in human–computer interaction
- 10 Embodiment and expressive communication on the internet
- Index
- Studies in Emotion and Social Interaction
- References
Summary
Overview: Human brains are basically social, and use communication mechanisms that have evolved during our evolutionary past. Thus, we suggest that even in communication with and by machines, humans will tend to react socially and use communication mechanisms that are primarily social and embodied. One of these mechanisms is communicative feedback, which refers to unobtrusive (usually short) vocal or bodily expressions, whereby a recipient of information can inform a contributor of information about whether he or she is able and willing to communicate, perceive the information, and understand the information. We will show how feedback can be modeled in virtual agents on facial expressions of a virtual agent or verbot and thus contribute to human–human communication over the internet. We will present a simple model based on a pleasure, arousal, and dominance space, which allows a complex stimulus generation program to be driven with only a few parameters.
Humans are social – but what about human–machine communication?
Internet communication consists of two major domains: communication with a machine and human–human communication through a machine. Both processes involve different but comparable elements in order to be efficient, as we will outline here.
In its early years, the internet was used by a rather small group of scientists for communication via email and bulletin boards. As compared to phone calls and direct face-to-face communication, it seemed to be missing a social component, thus leading to the introduction of emoticons such as the well-known smiley, which constituted a first attempt to fill this gap.
- Type
- Chapter
- Information
- Face-to-Face Communication over the InternetEmotions in a Web of Culture, Language, and Technology, pp. 237 - 279Publisher: Cambridge University PressPrint publication year: 2011
References
, , and (1992). On the semantics and pragmatics of linguistic feedback. Journal of Semantics, 9(1), 1–26.CrossRefGoogle Scholar
, , , , , and (2008). The analysis of embodied communicative feedback in multimodal corpora – a prerequisite for behavior simulation. Journal of Language Resources and Evaluation (Special Issue on Multimodal Corpora), 41(3–4), 255–272.CrossRefGoogle Scholar
, , and (2004). Simulating the emotion dynamics of a multimodal conversational agent. In (ed.), Proceedings of Affective Dialogue Systems Conference (ADS '04), LNAI 3068 (pp. 154–165). Berlin: Springer.CrossRefGoogle Scholar
and (2004). A cooperative dialogue game for resolving ontological discrepancies. In , (ed.), Advances in Agent Communication (pp. 349–363). Berlin: Springer.CrossRefGoogle Scholar
(2004). Religious costs as adaptations that signal altruistic intention. Evolution and Cognition, 10(1), 19–42.Google Scholar
(2004). Towards a model of technology and literacy development: story listening systems. Journal of Applied Developmental Psychology, 25(1), 75–105.CrossRefGoogle Scholar
and (1999). The power of a nod and a glance: envelope vs. emotional feedback in animated conversational agents. Applied Artificial Intelligence, 13, 519–538.CrossRefGoogle Scholar
, , , , , , and (1999). Embodiment in conversational interfaces: Rea. In CHI '99 Conference Proceedings (pp. 520–527). New York: ACM Press.Google Scholar
, , and (2001). BEAT: the Behavior Expression Animation Toolkit. In Proceedings of SIGGRAPH '01 (pp. 477–486). New York: ACM Press.CrossRefGoogle Scholar
(1979). Robust locally weighted regression and smoothing scatterplots. Journal of the American Statistical Association, 74, 829–836.CrossRefGoogle Scholar
and (1988). Locally weighted regression: an approach to regression analysis by local fitting. Journal of the American Statistical Association, 83, 596–610.CrossRefGoogle Scholar
and (1992). Cognitive adaptations for social exchange. In , , and (eds), The Adapted Mind: Evolutionary Psychology and the Generation of Culture (pp. 163–228). New York: Oxford University Press.Google Scholar
, , and (1992). Evolutionary Psychology and Conceptual Integration. Oxford University Press.Google Scholar
(2004). Attributing emotion to static body postures: recognition accuracy, confusions, and viewpoint dependence. Journal of Nonverbal Behavior, 28, 117–139.CrossRefGoogle Scholar
(1872). The Expression of the Emotions in Man and Animals. New York: D. Appleton and Company.CrossRefGoogle Scholar
and (1978). Animal signals: information or manipulation? In and (eds), Behavioural Ecology: An Evolutionary Approach (pp. 282–309). Sunderland, MA: Sinauer.Google Scholar
, , , , and (2003). From Greta's mind to her face: modeling the dynamics of affective states in a conversational embodied agent. International Journal of Human–Computer Studies, 59, 81–118.CrossRefGoogle Scholar
and (1967). Neue Wege der Humanethologie[New Ways in Human Ethology], Homo, 18, 13–23.Google Scholar
(1969). The repertoire of nonverbal behavior – categories, origins, usage and coding. Semiotica, 1, 49–98.CrossRefGoogle Scholar
(1971). Universals and cultural differences in facial expressions of emotion. Nebraska Symposium on Motivation, 19, 207–283.Google Scholar
(1980). Biological and cultural contributions to body and facial movement in the expression of emotions. In (ed.), Explaining Emotions (pp. 73–101). Berkeley, CA: University of California Press.Google Scholar
(1984). Expression and the nature of emotion. In and (eds), Approaches to Emotion (pp. 319–343). Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
(1994). Strong evidence for universals in facial expressions: a reply to Russell's mistaken critique. Psychological Bulletin, 115, 268–287.CrossRefGoogle ScholarPubMed
and (1978). Investigator's Guide: Facial Action Coding System. Palo Alto, CA: Consulting Psychologists Press.Google Scholar
(2007). Smiley lore :-). www.cs.cmu.edu/~sef/sefSmiley.htm.
(1991). Sociality and solitary smiling: potentiation by an implicit audience. Journal of Personality and Social Psychology, 60, 229–240.CrossRefGoogle Scholar
(2007). Warning sounded over ‘flirting robots.’ www.news.com/beyond-binary/8301-13860_3-9831133-56.html.
, , and (2004). A conversation robot with back-channel feedback function based on linguistic and nonlinguistic information. In Proceedings of the International Conference on Autonomous Robots and Agents (pp. 379–384).Google Scholar
and (1998). Mirror neurons and the simulation theory of mind-reading. Trends in Cognitive Science, 2, 493–501.CrossRefGoogle ScholarPubMed
(1974). The Interactive Generation of Facial Images on a CRT Using a Heuristic Strategy. Ohio State University, Computer Graphics Research Group.Google Scholar
, , , , and (1988). Patterns on the face: the eyebrow flash in crosscultural comparison. Ethology, 77, 279–299.CrossRefGoogle Scholar
and (1997). A neural network approach for the classification of body movements (Abstract). In , , , and (eds), New Aspects of Human Ethology (pp. 202–203). New York: Plenum Press.Google Scholar
, , and (1997). The communication paradox and possible solutions: towards a radical empiricism. In , , , and (eds), New Aspects of Human Ethology (pp. 91–120). New York: Plenum Press.CrossRefGoogle Scholar
, , and (2002). Dynamic systems and inferential information processing in human communication. Neuro Endocrinological Letters (Special Issue), 23(4), 15–22.Google ScholarPubMed
, , , and (2003). Darwinian aesthetics: sexual selection and the biology of beauty. Biological Reviews, 78(3), 385–340.CrossRefGoogle ScholarPubMed
, , , , , and (2004). The representation of self-reported affect in body posture and body posture simulation. Collegium Anthropologicum, 28(2), 159–173.Google ScholarPubMed
and (2006). The reconstruction of facial expressions in embodied systems: new approaches to an old problem. ZIF Mitteilungen, 2, 14–31.Google Scholar
(1990). Modularity and the psychoevolutionary theory of emotion. Biology & Philosophy, 5, 175–196.CrossRefGoogle Scholar
, , and (2004). Facial appearance, gender, and emotion expression. Emotion, 4, 378–388.CrossRefGoogle ScholarPubMed
and (1997). Cross-cultural back channels in English refusals: a source of trouble. In (ed.), Silence – Interdisciplinary Perspectives (pp. 285–308). Berlin: Mouton de Gruyter.Google Scholar
(1973). Visual perception of biological motion and a model of its analysis. Perception & Psychophysics, 14, 201–211.CrossRefGoogle Scholar
(1976). Spatio-temporal differentiation and integration in visual motion perception. Psychological Research, 38, 379–398.CrossRefGoogle ScholarPubMed
(2001). Anvil – a generic annotation tool for multimodal dialogue. In Proceedings of Eurospeech 2001, Aalborg (pp. 1367–1370).Google Scholar
and (1981). A categorized list of emotion definitions with suggestions for a consensual definition. Motivation and Emotion, 5, 345–379.CrossRefGoogle Scholar
, , , and (2005). A conversational agent as a museum guide. Design and evaluation of a real-world application. In et al. (eds), Intelligent Virtual Agents 2005 (pp. 329–343). Hamburg: Springer.CrossRefGoogle Scholar
, , , , and (2008). Modeling embodied feedback with virtual humans. In and (eds), Modeling Communication with Robots and Virtual Humans (pp. 18–37). Berlin: Springer.CrossRefGoogle Scholar
(2001). Bewegende Bewegung. Sozio-emotionale Wirkungen nonverbalen Verhaltens und deren experimentelle Untersuchung mittels Computeranimation. [Moving Movements. Socio-emotional Effects of Nonverbal Behavior and Their Experimental Analysis by Means of Computer Animation] Lengerich: Pabst Science Publishers.Google Scholar
and (1984). Animal signals: mind reading and manipulation. In and (eds), Behavioural Ecology: An Evolutionary Approach (pp. 380–402). Oxford: Blackwell Scientific.Google Scholar
and (2005). Moving smiles: the role of dynamic components for the perception of the genuineness of smiles. Journal of Nonverbal Behavior, 29, 3–24.CrossRefGoogle Scholar
, , , , and (2008). Is a neutral face really evaluated as being emotionally neutral?Psychiatry Research, 157, 77–85.CrossRefGoogle ScholarPubMed
(2003). Microsoft Agent product information. (www.microsoft.com/msagent/prodinfo/).
, , , , , and (2006). Amygdala activation at 3T in response to human and avatar facial expressions of emotions. Journal of Neuroscience Methods, 161(1), 126–133.CrossRefGoogle ScholarPubMed
and (1990). What's basic about basic emotions?Psychological Review, 97, 315–331.CrossRefGoogle ScholarPubMed
(1966). Dimensionality of the semantic space for communication via facial expressions. Scandinavian Journal of Psychology, 7, 1–30.CrossRefGoogle ScholarPubMed
(1992). A critical role for “affective neuroscience” in resolving what is basic about basic emotions. Psychological Review, 99(3), 554–560.CrossRefGoogle ScholarPubMed
(1972). Computer-generated animation of faces. Proceedings of ACM National Conference (vol. I, pp. 451–457).CrossRef
, , , , , , and (1999). Towards engaging full-body interaction. In and (eds), Adjunct Conference Proceedings, HCI International '99, 8th International Conference on Human–Computer Interaction (pp. 280–281). Stuttgart: Fraunhofer IRB Verlag.Google Scholar
, , , , and (2005). GRETA. A believable embodied conversational agent. In and (eds), Multimodal Intelligent Information Presentation. (pp. 3–25). Dordrecht: Kluwer.CrossRefGoogle Scholar
and (1996). The Media Equation. How People Treat Computers, Television, and New Media like Real People and Places. New York: Cambridge University Press.Google Scholar
(1978). Evidence of convergent validity on the dimensions of affect. Journal of Personality and Social Psychology, 36, 1152–1168.CrossRefGoogle Scholar
(1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39, 1161–1178.CrossRefGoogle Scholar
(1991). In defense of a prototype approach to emotion concepts. Journal of Personality and Social Psychology, 60, 425–438.CrossRefGoogle Scholar
(1995). Facial expression of emotion: what lies beyond minimal universality?Psychological Bulletin, 118, 379–391.CrossRefGoogle ScholarPubMed
and (1977). Evidence for a three-factor theory of emotions. Journal of Research in Personality, 11, 273–294.CrossRefGoogle Scholar
(1984). On the nature and function of emotion: a component process approach. In and (eds), Approaches to Emotion (pp. 293–317). Hillsdale, NJ: Erlbaum.Google Scholar
(1994). Affect bursts. In , , and (eds), Emotions: Essays on Emotion Theory (pp. 161–193). Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
(1997). Profiles of emotion-antecedent appraisal: testing theoretical predictions across cultures. Cognition and Emotion, 11, 113–150.CrossRefGoogle Scholar
(1999). Appraisal theory. In and (eds), Handbook of Emotion and Cognition (pp. 637–663). New York: Wiley.Google Scholar
(2001). Appraisal considered as a process of multi-level sequential checking. In , , and (eds), Appraisal Processes in Emotion: Theory, Methods, Research (pp. 92–120). New York: Oxford University Press.Google Scholar
and (2002). Human facial expressions as adaptations: evolutionary questions in facial expression research. Yearbook of Physical Anthropology, 44, 3–24.Google Scholar
(1989). Social and individual impact. Educational Media International, 26(2), 101–106.CrossRefGoogle Scholar
and (1997). Spontaneous facial behavior during intense emotional episodes: artistic truth and optical truth. In and (eds), The Psychology of Facial Expression (pp. 229–254). Cambridge University Press.CrossRefGoogle Scholar
(1992). Judgment of Feeling States from Facial Behavior: A Bottom-up Approach. Unpublished doctoral dissertation, University of British Columbia.Google Scholar
, , , , , , , , and (2001). Making faces: creating three-dimensional parameterized models of facial expression. Behavior Research Methods, Instruments and Computers, 33, 115–123.CrossRefGoogle ScholarPubMed
and (1998). Towards a tractable appraisal-based architecture for situated cognizers. In , , and (eds), Grounding Emotions in Adaptive Systems, Workshop Notes, 5th International Conference of the Society for Adaptive Behavior (pp. 56–61). Zurich: Society for Adaptive Behavior.Google Scholar
, , and (2004). Timing detection for realtime dialog systems using prosodic and linguistic information. In Proceedings of the International Conference on Speech Prosody (pp. 529–532).
(1996). Communicative Humanoids – A Computational Model of Psychological Dialogue Skills. PhD thesis, School of Architecture and Planning, Massachusetts Institute of Technology.Google Scholar
(1962). Affect Imagery Consciousness: Vol. I. The Positive Affects. New York: Springer.Google Scholar
and (2000). Prosodic features which cue back-channel responses in English and Japanese. Journal of Pragmatics, 32(8), 1177–1207.CrossRefGoogle Scholar
, , and (1988). Development and validation of brief measures of positive and negative affect: the PANAS scales. Journal of Personality and Social Psychology, 54(6), 1063–1070.CrossRefGoogle ScholarPubMed
and (2001). Towards computational modeling of appraisal theories. In , , and (eds), Appraisal Processes in Emotion: Theory, Methods, Research (pp. 350–365). New York: Oxford University Press.Google Scholar
(1984). Cybernetics and Control and Communication in the Animal and the Machine. Cambridge, MA: MIT Press.Google Scholar
(2001). Understanding the Use and Utility of Anthropomorphic Interface Agents. Student Poster at the CHI 2001 in Seattle, WA, USA.CrossRefGoogle Scholar
(1970). On getting a word in edgewise. In Papers from the Sixth Regional Meeting of the Chicago Linguistics Society (pp. 567–578). Chicago: Chicago Linguistics Society.Google Scholar
- 1
- Cited by