Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-cd9895bd7-gvvz8 Total loading time: 0 Render date: 2024-12-27T13:48:01.221Z Has data issue: false hasContentIssue false

32 - Analyzing Nested Data

Multilevel Modeling and Alternative Approaches

from Part VII - General Analytic Considerations

Published online by Cambridge University Press:  23 March 2020

By
Daniel J. Bauer ,
Daniel M. McNeish ,
Scott A. Baldwin  and
Patrick J. Curran
Edited by
Aidan G. C. Wright  and
Michael N. Hallquist
Aidan G. C. Wright
Affiliation:
University of Pittsburgh
Michael N. Hallquist
Affiliation:
Pennsylvania State University
Get access

Summary

Nested data arise frequently in clinical research. The nesting might be hierarchical, such as patients nested within clinicians, or it might be longitudinal, such as repeated assessments over time nested within individuals. As articulated in this chapter, whenever and however nesting occurs, it is necessary to account for the statistical dependence of observations within units when analyzing the data. Further, it is important to determine the level(s) of the data at which predictors exert their effects. Multilevel models are a particularly popular and useful approach for addressing these issues. We thus describe these models in detail, illustrating the application of multilevel models in clinical research via two examples. The first example considers nesting of siblings within families and demonstrates the importance of separating within- versus between-family effects. The second example focuses on the application of multilevel models with repeated measures to evaluate within-person change over time. Additionally, we provide a brief survey of other approaches to the analysis of nested data (e.g., cluster-robust standard errors, generalized estimating equations, fixed-effects models).

Keywords

Nested dataclusteringmultilevel modelingdependencerandom effectsfixed effectshierarchicallongitudinalgrowth
Type
Chapter
Information
The Cambridge Handbook of Research Methods in Clinical Psychology , pp. 426 - 443
Publisher: Cambridge University Press
Print publication year: 2020

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Baldwin, S. A., Bauer, D. J., Stice, E., & Rohde, P. (2011). Evaluating Models for Partially Clustered Designs. Psychological Methods, 16, 149‒165.CrossRefGoogle ScholarPubMed
Baldwin, S. A., & Fellingham, G. W. (2013). Bayesian Methods for the Analysis of Small Sample Multilevel Data with a Complex Variance Structure. Psychological Methods, 18, 151‒164.CrossRefGoogle ScholarPubMed
Baldwin, S. A., Murray, D. M., & Shadish, W. R. (2005). Empirically Supported Treatments or Type I Errors? Problems with the Analysis of Data from Group-Administered Treatments. Journal of Consulting and Clinical Psychology, 73, 924–935.Google Scholar
Baldwin, S. A., Wampold, B. E., & Imel, Z. E. (2007). Untangling the Alliance-Outcome Correlation: Exploring the Relative Importance of Therapist and Patient Variability in the Alliance. Journal of Consulting and Clinical Psychology, 75, 842‒852.CrossRefGoogle ScholarPubMed
Bauer, D. J., & Curran, P. J. (2005). Probing Interactions in Fixed and Multilevel Regression: Inferential and Graphical Techniques. Multivariate Behavioral Research, 40, 373‒400.CrossRefGoogle ScholarPubMed
Bauer, D. J., & Sterba, S. K. (2011). Fitting Multilevel Models with Ordinal Outcomes: Performance of Alternative Specifications and Methods of Estimation. Psychological Methods, 16, 373‒390.CrossRefGoogle ScholarPubMed
Bauer, D. J., Sterba, S. K., & Hallfors, D. D. (2008). Evaluating Group-Based Interventions when Control Participants Are Ungrouped. Multivariate Behavioral Research, 43, 210‒236.CrossRefGoogle ScholarPubMed
Bell, B. A., Morgan, G. B., Schoeneberger, J. A., Kromrey, J. D., & Ferron, J. M. (2014). How Low Can You Go? Methodology, 10, 1‒10.Google Scholar
Biesanz, J. C., Deeb-Sossa, N., Aubrecht, A. M., Bollen, K. A., & Curran, P. J. (2004). The Role of Coding Time in Estimating and Interpreting Growth Curve Models. Psychological Methods, 9, 30‒52.CrossRefGoogle ScholarPubMed
Bliese, P. D. (1998). Group Size, ICC Values, and Group-Level Correlations: A Simulation. Organizational Research Methods, 1, 355‒373.Google Scholar
Cameron, A. C., Gelbach, J. B., & Miller, D. L. (2008). Bootstrap-Based Improvements for Inference with Clustered Errors. Review of Economics and Statistics, 90, 414‒427.CrossRefGoogle Scholar
Crits-Christoph, P., & Mintz, J. (1991). Implications of Therapist Effects for the Design and Analysis of Comparative Studies of Psychotherapies. Journal of Consulting and Clinical Psychology, 59, 20–26.CrossRefGoogle ScholarPubMed
Curran, P. J., & Bauer, D. J. (2011). The Disaggregation of Within-Person and Between-Person Effects in Longitudinal Models of Change. Annual Review of Psychology, 62, 583‒619.Google Scholar
Diez-Roux, A. V. (1998). Bringing Context Back into Epidemiology: Variables and Fallacies in Multilevel Analysis. American Journal of Public Health, 88, 216‒222.Google Scholar
Diggle, P. J., Heagerty, P., Liang, K. Y., & Zeger, S. L. (2002). Analysis of Longitudinal Data. Oxford: Oxford Statistical Science Series.CrossRefGoogle Scholar
Dimidjian, S., Hollon, S. D., Dobson, K. S., Schmaling, K. B., Kohlenberg, R. J., Addis, M. E., … Jacobson, N. S. (2006). Randomized Trial of Behavioral Activation, Cognitive Therapy, and Antidepressant Medication in the Acute Treatment of Adults with Major Depression. Journal of Consulting and Clinical Psychology, 74, 658–670.CrossRefGoogle ScholarPubMed
Enders, C. K., & Tofighi, D. (2007). Centering Predictor Variables in Cross-Sectional Multilevel Models: A New Look at an Old Issue. Psychological Methods, 12, 121–138.Google Scholar
Gottfredson, N. C. (2019). A Straightforward Approach for Coping with Unreliability of Person Means When Parsing Within-Person and Between-Person Effects in Longitudinal Studies. Addictive Behaviors, 94, 156‒161.Google Scholar
Hardin, J., & Hilbe, J. (2003). Generalized Estimating Equations. London: Chapman and Hall/CRC.Google Scholar
Hedges, L. V., & Hedberg, E. C. (2007). Intraclass Correlation Values for Planning Group-Randomized Trials in Education. Educational Evaluation and Policy Analysis, 29, 60‒87.CrossRefGoogle Scholar
Hoffman, L., & Rovine, M. J. (2007). Multilevel Models for the Experimental Psychologist: Foundations and Illustrative Examples. Behavior Research Methods, 39, 101‒117.CrossRefGoogle ScholarPubMed
Hoffman, L., & Stawski, R.S. (2009). Persons as Contexts: Evaluating Between-Person and Within-Person Effects in Longitudinal Analysis. Research in Human Development, 6, 97‒120.Google Scholar
Hox, J. J. (2010). Multilevel Analysis: Techniques and Applications. New York: Routledge.Google Scholar
Hox, J. J., van de Schoot, R., & Matthijsse, S. (2012, July). How Few Countries Will Do? Comparative Survey Analysis from a Bayesian Perspective. Survey Research Methods, 6, 87‒93.Google Scholar
Huang, F. L. (2016). Alternatives to Multilevel Modeling for the Analysis of Clustered Data. Journal of Experimental Education, 84, 175‒196.CrossRefGoogle Scholar
Kenny, D. (1995). The Effect of Nonindependence on Significance Testing in Dyadic Research. Personal Relationships, 2, 67–75.Google Scholar
Kenny, D. A., Kashy, D. A., & Cook, W. L. (2006). Dyadic Data Analysis. New York: Guilford.Google Scholar
Kenny, D. A., Mannetti, L., Pierro, A., Livi, S., & Kashy, D. A. (2002). The Statistical Analysis of Data from Small Groups. Journal of Personality and Social Psychology, 83, 126–137.CrossRefGoogle Scholar
Kievit, R., Frankenhuis, W. E., Waldorp, L., & Borsboom, D. (2013). Simpson’s Paradox in Psychological Science: A Practical Guide. Frontiers in Psychology, 4, 1‒14.Google Scholar
Kutner, M. H., Nachtsheim, C. J., & Neter, J. (2004). Applied Linear Regression Models. Boston, MA: McGraw-HillGoogle Scholar
Kreft, I. G. G. (1996). Are Multilevel Techniques Necessary? An Overview, Including Simulation Studies. California State University at Los Angeles.Google Scholar
Lenzenweger, M. F., Johnson, M. D., & Willett, J. B. (2004). Individual Growth Curve Analysis Illuminates Stability and Change in Personality Disorder Features: The Longitudinal Study of Personality Disorders. Archives of General Psychiatry, 61, 1015–1024.CrossRefGoogle ScholarPubMed
Liang, K. Y., & Zeger, S. L. (1986). Longitudinal Data Analysis Using Generalized Linear Models. Biometrika, 73, 13–22.Google Scholar
Lüdtke, O., Marsh, H. W., Robitzsch, A., Trautwein, U., Asparouhov, T., & Muthén, B. (2008). The Multilevel Latent Covariate Model: A New, More Reliable Approach to Group-Level Effects in Contextual Studies. Psychological Methods, 13, 203‒229.CrossRefGoogle Scholar
Marsh, H. W., Lüdtke, O, Robitzsch, A., Trautwein, U., Asparouhov, T., Muthén, B., & Nagengast, B. (2009). Doubly-Latent Models of School Contextual Effects: Integrating Multilevel and Structural Equation Approaches to Control Measurement and Sampling Error. Multivariate Behavioral Research, 44, 764‒802.Google Scholar
McCulloch, C. E., Searle, S. R., & Neuhaus, J. M. (2008). Generalized, Linear, and Mixed Models. New York: Wiley.Google Scholar
McNeish, D. (2016). On Using Bayesian Methods to Address Small Sample Problems. Structural Equation Modeling, 23, 750‒773.Google Scholar
McNeish, D., & Harring, J. R. (2017). Clustered Data with Small Sample Sizes: Comparing the Performance of Model-Based and Design-Based Approaches. Communications in Statistics: Simulation and Computation, 46, 855‒869.Google Scholar
McNeish, D. M., & Stapleton, L. M. (2016a). The Effect of Small Sample Size on Two-Level Model Estimates: A Review and Illustration. Educational Psychology Review, 28, 295‒314.Google Scholar
McNeish, D., & Stapleton, L. M. (2016b). Modeling Clustered Data with Very Few Clusters. Multivariate Behavioral Research, 51, 495‒518.Google Scholar
McNeish, D., Stapleton, L. M., & Silverman, R. D. (2017). On the Unnecessary Ubiquity of Hierarchical Linear Modeling. Psychological Methods, 22, 114‒140.CrossRefGoogle ScholarPubMed
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical Linear Models. Newbury Park, CA: Sage.Google Scholar
Rausch, J. R., Maxwell, S. E., & Kelley, K. (2003). Analytic Methods for Questions Pertaining to a Randomized Pretest, Posttest, Follow-Up Design. Journal of Clinical and Consulting Psychology, 32, 467‒486.Google Scholar
Rights, J. D., & Sterba, S. K. (2019). Quantifying Explained Variance in Multilevel Models: An Integrative Framework for Defining R-Squared Measures. Psychological Methods, 24, 309‒338.Google Scholar
Roberts, C., & Roberts, S. A. (2005). Design and Analysis of Clinical Trials with Clustering Effects Due to Treatment. Clinical Trials, 2, 152‒162.Google Scholar
Schafer, J. L. (2003). Multiple Imputation: A Primer. Statistical Methods in Medical Research, 8, 3–15.Google Scholar
Schwartz, J. E., & Stone, A. A. (1998). Strategies for Analyzing Ecological Momentary Assessment Data. Health Psychology, 17, 6‒16.Google Scholar
Snijders, T., & Bosker, R. (2012). Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling (2nd edn.). London: Sage.Google Scholar
Stice, E., Shaw, H., Burton, E., & Wade, E. (2006). Dissonance and Healthy Weight Eating Disorder Prevention Programs: A Randomized Efficacy Trial. Journal of Consulting and Clinical Psychology, 74, 263‒275.CrossRefGoogle ScholarPubMed
Sterba, S. K. (2017). Partially Nested Designs in Psychotherapy Trials: A Review of Modeling Developments. Psychotherapy Research, 27, 425‒436.Google Scholar
Taylor, S., Thordarson, D. S., Maxfield, L., Fedoroff, I. C., Lovell, K., & Ogrodniczuk, J. (2003). Comparative Efficacy, Speed, and Adverse Effects of Three PTSD Treatments: Exposure Therapy, EMDR, and Relaxation Training. Journal of Consulting and Clinical Psychology, 71, 330–338.Google Scholar
Wang, L., & Maxwell, S. E. (2015). On Disaggregating Between-Person and Within-Person Effects with Longitudinal Data Using Multilevel Models. Psychological Methods, 20, 63‒83.Google Scholar
West, S. G., Ryu, E., Kwok, O. M., & Cham, H. (2011). Multilevel Modeling: Current and Future Applications in Personality Research. Journal of Personality, 79, 2‒50.Google Scholar
van de Schoot, R., Broere, J. J., Perryck, K. H., Zondervan-Zwijnenburg, M., & Van Loey, N. E. (2015). Analyzing Small Data Sets Using Bayesian Estimation: The Case of Posttraumatic Stress Symptoms Following Mechanical Ventilation in Burn Survivors. European Journal of Psychotraumatology, 6(1), 25216.Google Scholar
Zeger, S. L., Liang, K. Y., & Albert, P. S. (1988). Models for Longitudinal Data: A Generalized Estimating Equation Approach. Biometrics, 44, 1049‒1060.Google Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×