Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-6bf8c574d5-t27h7 Total loading time: 0 Render date: 2025-02-14T23:54:25.792Z Has data issue: false hasContentIssue false

16 - Behavioral Observation and Coding

from Part III - Deep Dives on Methods and Tools for Testing Your Question of Interest

Published online by Cambridge University Press:  12 December 2024

By
Katherine R. Thorson  and
Tessa West
Edited by
Harry T. Reis ,
Tessa West  and
Charles M. Judd
Harry T. Reis
Affiliation:
University of Rochester, New York
Tessa West
Affiliation:
New York University
Charles M. Judd
Affiliation:
University of Colorado Boulder
Get access

Summary

We provide a practical overview of the most important steps of behavioral observation and coding, with a focus on how these processes are typically executed within social and personality psychology. The chapter has six main sections. We begin by explaining what is meant by behavioral observation and coding, and we outline the strengths and challenges of this method. We then describe two guiding principles that apply throughout observation and coding. Next, we highlight several aspects of observation and coding for researchers to consider, many of which vary along a continuum. We also discuss practical questions regarding coding, such as the number of coders needed. We describe the analysis of behavioral data – from establishing inter-rater agreement to running models with the coded behaviors as outcomes of interest. Lastly, we discuss concerns related to automated processing of videos and text and topics related to the open-science movement.

Keywords

behaviorobservationbehavioral codingvideo codinginterpersonal behaviorinterrater reliability
Type
Chapter
Information
Handbook of Research Methods in Social and Personality Psychology , pp. 378 - 403
Publisher: Cambridge University Press
Print publication year: 2024

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Adolph, K. (2016). Video as data: from transient behavior to tangible recording. APS Observer, 29(3), 23–25.Google ScholarPubMed
Adolph, K., Gilmore, R. O., Staff and Databrary Admin (2013). Databrary sponsored workshops and events. Databrary, http://doi.org/10.17910/B7159Q (accessed March 3, 2023).CrossRefGoogle Scholar
Aron, A., Melinat, E., Aron, E. N., Vallone, R. D., and Bator, R. J. (1997). The experimental generation of interpersonal closeness: A procedure and some preliminary findings. Personality and Social Psychology Bulletin, 23(4), 363–377.CrossRefGoogle Scholar
Back, M. D., Schmukle, S. C., and Egloff, B. (2009). Predicting actual behavior from the explicit and implicit self-concept of personality. Journal of Personality and Social Psychology, 97(3), 533–5418.CrossRefGoogle ScholarPubMed
Barak, M., Lipson, A., and Lerman, S. (2006). Wireless laptops as means for promoting active learning in large lecture halls. Journal of Research on Technology in Education, 38(3), 245–263.CrossRefGoogle Scholar
Bartlett, M. S., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., and Movellan, J. (2005). Recognizing facial expression: Machine learning and application to spontaneous behavior. 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 2, 568–573.CrossRefGoogle Scholar
Baumeister, R. F., Vohs, K. D., and Funder, D. C. (2007). Psychology as the science of self-reports and finger movements: Whatever happened to actual behavior? Perspectives on Psychological Science, 2(4), 396–403.CrossRefGoogle ScholarPubMed
Bergsieker, H. B., Shelton, J. N., and Richeson, J. A. (2010). To be liked versus respected: Divergent goals in interracial interactions. Journal of Personality and Social Psychology, 99(2), 248–264.CrossRefGoogle ScholarPubMed
Black, M. P., Katsamanis, A., Baucom, B. R., Lee, C.-C., Lammert, A. C., Christensen, A., Georgiou, P. G., and Narayanan, S. S. (2013). Toward automating a human behavioral coding system for married couples’ interactions using speech acoustic features. Speech Communication, 55(1), 1–21.CrossRefGoogle Scholar
Bolger, N., and Shrout, P. E. (2007). Accounting for statistical dependency in longitudinal data on dyads. In Little, T. D., Bovaird, J. A., and Card, N. A. (eds.) Modeling Contextual Effects in Longitudinal Studies. Lawrence Erlbaum Associates Publishers.Google Scholar
Boyd, R. L., and Schwarz, H. A. (2021). Natural language analysis and the psychology of verbal behavior: The past, present, and future states of the field. Journal of Language and Social Psychology, 40(1), https://journals.sagepub.com/doi/full/10.1177/0261927X20967028.CrossRefGoogle ScholarPubMed
Brady, W. J., McLoughlin, K., Doan, T. N., and Crockett, M. J. (2021). How social learning amplifies moral outrage expression in online social networks. Science Advances, 7, DOI:10.31234/osf.io/gf7t5.CrossRefGoogle ScholarPubMed
Brunswik, E. (1955). Representative design and probabilistic theory in a functional psychology. Psychological Review, 62, 193–217.CrossRefGoogle Scholar
Capps, K. P., Updegraff, J. A., Foust, J. L., O’Brien, A. G., and Taber, J. M. (2022). Field experiment of signs promoting hand hygiene during the COVID-19 pandemic. Health Psychology, 41, 826–832.CrossRefGoogle ScholarPubMed
Carcone, A. I., Hasan, M., Alexander, G. L., Dong, M., Eggly, S., Brogan Hartlieb, K., Naar, S., MacDonell, K., and Kotov, A. (2019). Developing machine learning models for behavioral coding. Journal of Pediatric Psychology, 44(3), 289–299.CrossRefGoogle Scholar
Carter, N. T., Carter, D. R., and DeChurch, L. A. (2018). Implications of observability for the theory and measurement of emergent team phenomena. Journal of Management, 44(4), 1398–1425.CrossRefGoogle Scholar
Chakravarthula, S. N., Baucom, B. R. W., Narayanan, S., and Georgiou, P. (2021). An analysis of observation length requirements for machine understanding of human behaviors from spoken language. Computer Speech & Language, 66, 101162, https://doi.org/10.1016/j.csl.2020.101162.CrossRefGoogle Scholar
Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290.CrossRefGoogle Scholar
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46.CrossRefGoogle Scholar
Cronbach, L. J., Gleser, G. C., Nanda, H., and Rajaratnam, N. (1972). The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles. John Wiley & Sons Inc.Google Scholar
de Mast, J. (2007). Agreement and kappa-type indices. American Statistician, 61(2), 148–153.CrossRefGoogle Scholar
Dittmann, A. G., Stephens, N. M., and Townsend, S. S. M. (2020). Achievement is not class-neutral: Working together benefits people from working-class contexts. Journal of Personality and Social Psychology, 119, 517–539.CrossRefGoogle Scholar
Dovidio, J. F., Kawakami, K., Johnson, C., Johnson, B., and Howard, A. (1997). On the nature of prejudice: Automatic and controlled processes. Journal of Experimental Social Psychology, 33(5), 510–540.CrossRefGoogle Scholar
Dumitru, O. D., Thorson, K. R., and West, T. V. (2022). Investigating gender differences among tutors and students during STEM peer tutoring: Women are as behaviorally engaged as men but experience more negative affect. Contemporary Educational Psychology, 70, 102088, https://doi.org/10.1016/j.cedpsych.2022.102088.CrossRefGoogle Scholar
Elfenbein, H. A., and Luckman, E. A. (2016). Interpersonal accuracy in relation to culture and ethnicity. In Hall, J. A., Schmid Mast, M., and West, T. V. (eds.) The Social Psychology of Perceiving Others Accurately, 1st ed. Cambridge University Press.Google Scholar
Feldman, R., Gordon, I., and Zagoory-Sharon, O. (2011). Maternal and paternal plasma, salivary, and urinary oxytocin and parent–infant synchrony: Considering stress and affiliation components of human bonding. Developmental Science, 14(4), 752–761.CrossRefGoogle ScholarPubMed
Fisher, P. H., Dobbs-Oates, J., Doctoroff, G. L., and Arnold, D. H. (2012). Early math interest and the development of math skills. Journal of Educational Psychology, 104(3), 673–681.CrossRefGoogle Scholar
Freeman, J. B., Stolier, R. M., and Brooks, J. A. (2020). Dynamic interactive theory as a domain-general account of social perception. Advances in Experimental Social Psychology, 61, 237–287.CrossRefGoogle ScholarPubMed
Funder, D. C. (1995). On the accuracy of personality judgment: A realistic approach. Psychological Review, 10(4), 652–670.CrossRefGoogle Scholar
Furr, R. M. (2009). Personality psychology as a truly behavioural science. European Journal of Personality, 23(5), 369–401.CrossRefGoogle Scholar
Gaither, S. E., Babbitt, L. G., and Sommers, S. R. (2018). Resolving racial ambiguity in social interactions. Journal of Experimental Social Psychology, 76, 259–269.CrossRefGoogle Scholar
Gordon, A. M., and Chen, S. (2016). Do you get where I’m coming from? Perceived understanding buffers against the negative impact of conflict on relationship satisfaction. Journal of Personality and Social Psychology, 110(2), 239–260.CrossRefGoogle Scholar
Gosling, S. D., John, O. P., Craik, K. H., and Robins, R. W. (1998). Do people know how they behave? Self-reported act frequencies compared with on-line codings by observers. Journal of Personality and Social Psychology, 74, 1337–1349.CrossRefGoogle ScholarPubMed
Haghani, P., Narayanan, A., Bacchiani, M., Chuang, G., Gaur, N., Moreno, P., Prabhavalkar, R., Qu, Z., and Waters, A. (2018). From audio to semantics: Approaches to end-to-end spoken language understanding. 2018 IEEE Spoken Language Technology Workshop (SLT), 720–726, https://doi.org/10.1109/SLT.2018.8639043.CrossRefGoogle Scholar
Haines, N., Southward, M. W., Cheavens, J. S., Beauchaine, T., and Ahn, W.-Y. (2019). Using computer-vision and machine learning to automate facial coding of positive and negative affect intensity. PLOS ONE, 14(2), e0211735, https://doi.org/10.1371/journal.pone.0211735.CrossRefGoogle ScholarPubMed
Hall, J. A., Gunnery, S. D., and Horgan, T. G. (2016). Gender differences in interpersonal accuracy. In Hall, J. A., Schmid Mast, M., and West, T. V. (eds.) The Social Psychology of Perceiving Others Accurately, 1st ed. Cambridge University Press.CrossRefGoogle Scholar
Hallgren, K. A. (2012). Computing inter-rater reliability for observational data: An overview and tutorial. Tutorials in Quantitative Methods for Psychology, 8(1), 23–34.CrossRefGoogle ScholarPubMed
Hansen, P. G., Larsen, E. G., and Gundersen, C. D. (2022). Reporting on one’s behavior: A survey experiment on the nonvalidity of self-reported COVID-19 hygiene-relevant routine behaviors. Behavioural Public Policy, 6(1), 34–51.CrossRefGoogle Scholar
Harari, G. M., Lane, N. D., Wang, R., Crosier, B. S., Campbell, A. T., and Gosling, S. D. (2016). Using smartphones to collect behavioral data in psychological science: Opportunities, practical considerations, and challenges. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 11(6), 838–854.CrossRefGoogle Scholar
Härdelin, G., Holding, B. C., Reess, T., Geranmayeh, A., Axelsson, J., and Sundelin, T. (2021). Do mothers have worse sleep than fathers? Sleep imbalance, parental stress, and relationship satisfaction in working parents. Nature and Science of Sleep, 13, 1955–1966.Google Scholar
Heavey, C. L., Christensen, A., and Malamuth, N. M. (1995). The longitudinal impact of demand and withdrawal during marital conflict. Journal of Consulting and Clinical Psychology, 63(5), 797–801.CrossRefGoogle ScholarPubMed
Heyman, R. E., Lorber, M. F., Eddy, J. M., and West, T. V. (2014). Behavioral observation and coding. In Reis, H. T. and Judd, C. M. (eds.) Handbook of Research Methods in Social and Personality Psychology, 2nd ed. Cambridge University Press.Google Scholar
Huckins, J. F., daSilva, A. W., Wang, W., Hedlund, E., Rogers, C., Nepal, S. K., Wu, J., Obuchi, M., Murphy, E. I., Meyer, M. L., Wagner, D. D., Holtzheimer, P. E., and Campbell, A. T. (2020). Mental health and behavior of college students during the early phases of the COVID-19 pandemic: Longitudinal smartphone and ecological momentary assessment study. Journal of Medical Internet Research, 22(6), e20185, https://doi.org/10.2196/20185.CrossRefGoogle ScholarPubMed
Hughes, B. T., Flournoy, J. C., and Srivastava, S. (2021). Is perceived similarity more than assumed similarity? An interpersonal path to seeing similarity between self and others. Journal of Personality and Social Psychology, 121, 184–200.CrossRefGoogle ScholarPubMed
Karremans, J. C., and Verwijmeren, T. (2008). Mimicking attractive opposite-sex others: The role of romantic relationship status. Personality and Social Psychology Bulletin, 34(7), 939–950.CrossRefGoogle ScholarPubMed
Kenny, D. A., Mohr, C. D., and Levesque, M. J. (2001). A social relations variance partitioning of dyadic behavior. Psychological Bulletin, 127, 128–141.CrossRefGoogle ScholarPubMed
Landis, J. R., and Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174.CrossRefGoogle ScholarPubMed
Latu, I. M., and Schmid Mast, M. (2016). Male interviewers’ nonverbal dominance predicts lower evaluations of female applicants in simulated job interviews. Journal of Personnel Psychology, 15(3), 116, https://doi.org/10.1027/1866-5888/a000159CrossRefGoogle Scholar
McGraw, K. O., and Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30–46.CrossRefGoogle Scholar
Maner, J. K. (2016). Into the wild: Field research can increase both replicability and real-world impact. Journal of Experimental Social Psychology, 66, 100–106.CrossRefGoogle Scholar
Mehl, M. R. (2017). The electronically activated recorder (EAR): A method for the naturalistic observation of daily social behavior. Current Directions in Psychological Science, 26(2), 184–190.CrossRefGoogle Scholar
Mendes, W. B., and Koslov, K. (2013). Brittle smiles: Positive biases toward stigmatized and outgroup targets. Journal of Experimental Psychology: General, 142, 923–933.CrossRefGoogle ScholarPubMed
Moskowitz, D. S. (1988). Cross-situational generality in the laboratory: Dominance and friendliness. Journal of Personality and Social Psychology, 54(5), 829–839.CrossRefGoogle Scholar
Murphy, N. A., and Hall, J. A. (2021). Capturing behavior in small doses: A review of comparative research in evaluating thin slices for behavioral measurement. Frontiers in Psychology, 12, www.frontiersin.org/articles/10.3389/fpsyg.2021.667326.CrossRefGoogle ScholarPubMed
Murphy, N. A., Hall, J. A., Ruben, M. A., Frauendorfer, D., Schmid Mast, M., Johnson, K. E., and Nguyen, L. (2019). Predictive validity of thin-slice nonverbal behavior from social interactions. Personality and Social Psychology Bulletin, 45(7), https://journals.sagepub.com/doi/10.1177/0146167218802834.CrossRefGoogle ScholarPubMed
Murphy, N. A., Hall, J. A., Schmid Mast, M., Ruben, M. A., Frauendorfer, D., Blanch-Hartigan, D., Roter, D. L., and Nguyen, L. (2015). Reliability and validity of nonverbal thin slices in social interactions. Personality and Social Psychology Bulletin, 41(2), 199–213.CrossRefGoogle ScholarPubMed
Myaskovsky, L., Unikel, E., and Dew, M. A. (2005). Effects of gender diversity on performance and interpersonal behavior in small work groups. Sex Roles, 52(9), 645–657.CrossRefGoogle Scholar
Nils, F., and Rimé, B. (2012). Beyond the myth of venting: Social sharing modes determine the benefits of emotional disclosure. European Journal of Social Psychology, 42(6), 672–681.CrossRefGoogle Scholar
Paluck, E. L., and Cialdini, R. B. (2014). Field research methods. In Reis, H. T. and Judd, C. M. (eds.) Handbook of Research Methods in Social and Personality Psychology, 2nd ed. Cambridge University Press.Google Scholar
Park, J., Woolley, J., and Mendes, W. B. (2022). The effects of intranasal oxytocin on black participants’ responses to outgroup acceptance and rejection. Frontiers in Psychology, 13, 916305, https://doi.org/10.3389/fpsyg.2022.916305.CrossRefGoogle ScholarPubMed
Patterson, G. R. (1982). Coercive Family Process. Castalia.Google Scholar
Poole, K. L., and Henderson, H. A. (2022). Shyness, self-focused attention, and behavioral mimicry during social interaction. Journal of Research in Personality, 98, 104225, https://doi.org/10.1016/j.jrp.2022.104225.CrossRefGoogle Scholar
Rapuano, M., Sbordone, F. L., Borrelli, L. O., Ruggiero, G., and Iachini, T. (2021). The effect of facial expressions on interpersonal space: A gender study in immersive virtual reality. In Esposito, A., Faundez-Zanuy, M., Morabito, F. C., and Pasero, E. (eds.), Progresses in Artificial Intelligence and Neural Systems. Springer.Google Scholar
Sandstrom, G. M., and Boothby, E. J. (2021). Why do people avoid talking to strangers? A mini meta-analysis of predicted fears and actual experiences talking to a stranger. Self and Identity, 20(1), 47–71.CrossRefGoogle Scholar
Schmid Mast, M., Gatica-Perez, D., Frauendorfer, D., Nguyen, L., and Choudhury, T. (2015). Social sensing for psychology: Automated interpersonal behavior assessment. Current Directions in Psychological Science, 24(2), 154–160.CrossRefGoogle Scholar
Schroeder, J., Risen, J. L., Gino, F., and Norton, M. I. (2019). Handshaking promotes deal-making by signaling cooperative intent. Journal of Personality and Social Psychology, 116(5), 743–768.CrossRefGoogle ScholarPubMed
Shavelson, R. J., and Webb, N. M. (2006). Generalizability theory. In Green, J. L., Camilli, G., and Elmore, P. B. (eds.), Handbook of Complementary Methods in Education Research, 3rd ed. Routledge.Google Scholar
Shrout, P. E., and Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428.CrossRefGoogle ScholarPubMed
Shrout, P. E., and Lane, S. P. (2012). Psychometrics. In Mehl, M. R. and Conner, T. S. (eds.) Handbook of Research Methods for Studying Daily Life. Guilford Press.Google Scholar
Tausczik, Y. R., and Pennebaker, J. W. (2010). The psychological meaning of words: LIWC and computerized text analysis methods. Journal of Language and Social Psychology, 29(1), 24–54.CrossRefGoogle Scholar
ten Hove, D., Jorgensen, T. D., and van der Ark, L. A. (2021). Interrater reliability for multilevel data: A generalizability theory approach. Psychological Methods, 27(4), 650–666.CrossRefGoogle Scholar
Thorson, K. R., Forbes, C. E., Magerman, A. B., and West, T. V. (2019). Under threat but engaged: Stereotype threat leads women to engage with female but not male partners in math. Contemporary Educational Psychology, 58, 243–259.CrossRefGoogle Scholar
Thorson, K. R., Ketay, S., Roy, A. R. K., and Welker, K. M. (2021). Self-disclosure is associated with adrenocortical attunement between new acquaintances. Psychoneuroendocrinology, 132, 105323, https://doi.org/10.1016/j.psyneuen.2021.105323.CrossRefGoogle ScholarPubMed
Thorson, K. R., Mendes, W. B., and West, T. V. (2020). Controlling the uncontrolled: Are there incidental experimenter effects on physiologic responding?Psychophysiology, 57(3), e13500, https://doi.org/10.1111/psyp.13500.CrossRefGoogle ScholarPubMed
Traupman, E. K., Smith, T. W., Florsheim, P., Berg, C. A., and Uchino, B. N. (2011). Appraisals of spouse affiliation and control during marital conflict: Common and specific cognitive correlates among facets of negative affectivity. Cognitive Therapy and Research, 35(3), 187–198.CrossRefGoogle Scholar
Wang, M., Chen, K., and Hall, J. (2021). Predictive validity of thin slices of verbal and nonverbal behaviors: Comparison of slice lengths and rating methodologies. Journal of Nonverbal Behavior, 45, 1–14.CrossRefGoogle Scholar
West, T. V., Koslov, K., Page-Gould, E., Major, B., and Mendes, W. B. (2017). Contagious anxiety: Anxious European Americans can transmit their physiological reactivity to African Americans. Psychological Science, 28(12), 1796–1806.CrossRefGoogle ScholarPubMed
West, T. V., Pearson, A. R., and Stern, C. (2014). Anxiety perseverance in intergroup interaction: When incidental explanations backfire. Journal of Personality and Social Psychology, 107(5), 825–843.CrossRefGoogle ScholarPubMed
Witkower, Z., Tracy, J. L., Cheng, J. T., and Henrich, J. (2020). Two signals of social rank: Prestige and dominance are associated with distinct nonverbal displays. Journal of Personality and Social Psychology, 118(1), 89–120.CrossRefGoogle ScholarPubMed
Xu, S., and Lorber, M. F. (2014). Interrater agreement statistics with skewed data: Evaluation of alternatives to Cohen’s kappa. Journal of Consulting and Clinical Psychology, 82, 1219–1227.CrossRefGoogle ScholarPubMed
Yilmaz, G. (2016). What you do and how you speak matter: Behavioral and linguistic determinants of performance in virtual teams. Journal of Language and Social Psychology, 35(1), 76–97.CrossRefGoogle Scholar
Zee, K. S., and Bolger, N. (2022). Physiological coregulation during social support discussions. Emotion, 23(3), 825–843.CrossRefGoogle ScholarPubMed

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×