Hostname: page-component-78c5997874-8bhkd Total loading time: 0 Render date: 2024-11-04T19:43:27.682Z Has data issue: false hasContentIssue false
  • English
  • Français

Measurement validity in cross-cultural comparative research

Published online by Cambridge University Press:  11 April 2011

Martin Prince
Martin Prince*
Affiliation:
Institute of Psychiatry, King's College London
*
Professor M. Prince, Institute of Psychiatry, King's College London P060, De Crespigny Park, London SE5 8AF (United Kingdom). Fax: +44-20-78480137 E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Summary

Background – The purpose of this article is to review the procedures to establish measurement validity in crosscultural comparative research, including recent developments in the quantitative assessment of cross-cultural construct validity. Methods – A narrative review, illustrated by selected examples, of methods in four areas – formative conceptual research, translation and adaptation, criterion validity and construct validity. Results – Valid assessment across cultures requires qualitative research to investigate the cultural relevance of the construct, a careful translation and adaptation of a common measure, followed by pre-testing and cognitive interviews on the populations to be tested. Full criterion validation across diverse cultures may be a chimera given the difficulty in establishing a universally applicable ‘gold standard'. Quantitative analyses can, however, have a part to play in establishing construct validity across cultures. Scale internal consistency, inter-item and item-total correlations and test-retest reliability provide basic support for the viability of a measure in a new cultural setting. Exploratory factor analysis can be used to compare factors and factor loadings. The hypothesis of ‘measurement invariance’ across countries and cultures can be tested explicitly using confirmatory factor analysis (common underlying factors and factor loadings) and Rasch models (common hierarchality of items). Despite measurement invariance, threshold effects arising from cultural differences in norms, or expectations, or expressions of mental distress may still be a problem. Conclusions – There are few examples in the cross-cultural mental health literature of demonstrably valid culture-fair comparison. Much more, could, in principle, be done either to demonstrate measurement invariance, or to identify and explore sources of heterogeneity.

Declaration of Interest: None.

Keywords

cross cultural comparisonpsychometricsmeasurementvalidity
Type
Special Articles
Information
Epidemiology and Psychiatric Sciences , Volume 17 , Issue 3 , September 2008 , pp. 211 - 220
Copyright
Copyright © Cambridge University Press 2008

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Akaike, H. (1987). Factor analysis and AIC. Psychometrika 52, 317–332.Google Scholar
Alexopoulos, G.S., Meyers, B.S. & Young, R.C. (19979. Vascular depression hypothesis. Archives General Psychiatry 54, 915–922.Google Scholar
Bentler, P.M. & Bonett, D.G. (1980). Significance tests and goodness of fit in the analysis of covariances structures. Psychological Bulletin 88, 588–606.Google Scholar
Borsch-Supan, A., Brugiavini, A., Jurges, H., Mackenbach, J., Siegrist, J. & Weber, G. (2005). Health, Ageing and Retirement in Europe – First Results from the Survey of Health, Ageing and Retirement in Europe. MEA: Manheim.Google Scholar
Browne, M.W. (1990). MUTMUM PC: User's Guide Ohio State University: Columbus.Google Scholar
Burnham, K.P. & Anderson, D.R. (1998). Model Selection and Inference: A Practical Information-Theoretic Approach. Springer-Verlag: New York.Google Scholar
Castro-Costa, E., Dewey, M., Stewart, R., Banerjee, S., Huppert, F., Mendonca-Lima, C., Bula, C., Reisches, F., Wancata, J., Ritchie, K., Tsolaki, M., Mateos, R. & Prince, M. (2007). Prevalence of depressive symptoms and syndromes in later life in ten European countries: the SHARE study. British Journal of Psychiatry 191, 393–401.CrossRefGoogle ScholarPubMed
Castro-Costa, E., Dewey, M., Stewart, R., Banerjee, S., Huppert, F., Mendonca-Lima, C., Bula, C., Reisches, F., Wancata, J., Ritchie, K., Tsolaki, M., Mateos, R. & Prince, M. (2008). Ascertaining late-life depressive symptoms in Europe: an evaluation of the survey version of the EURO-D scale in 10 nations. The SHARE project. International Journal of Methods in Psychiatric Research 17(1), 12–29.Google Scholar
Cohen, L. (1995). Toward an anthropology of senility: anger, weakness, and Alzheimer's in Banaras, India. Medical Anthropology Quarterly 9(3), 314–334.CrossRefGoogle ScholarPubMed
Copeland, J.R., Dewey, M.E. & Griffiths-Jones, H.M. (1986).Computerised psychiatric diagnostic system and case nomenclature for elderly subjects:GMS and AGECAT. Psychological Medicine 16, 89–99.CrossRefGoogle ScholarPubMed
Cronbach, L.E. & Meehl, P.E. (1955). Construct validity in psychological tests. Psychological Bulletin 52, 281–302.CrossRefGoogle ScholarPubMed
Demyttenaere, K., Bruffaerts, R., Posada-Villa, J., Gasquet, I., Kovess, V., Lepine, J.P., Angermeyer, M.C., Bernert, S., de Girolamo, G., Morosini, P., Polidori, G., Kikkawa, T., Kawakami, N., Ono, Y., Takeshima, T., Uda, H., Karam, E.G., Fayyad, J.A., Karam, A.N., Mneimneh, Z.N., Medina-Mora, M.E., Borges, G., Lara, C., de Graaf, R., Ormel, J., Gureje, O., Shen, Y., Huang, Y., Zhang, M., Alonso, J., Haro, J.M., Vilagut, G., Bromet, E.J., Gluzman, S., Webb, C., Kessler, R.C., Merikangas, K.R., Anthony, J.C., von Korff, M.R., Wang, P.S., Brugha, T.S., Guilar-Gaxiola, S., Lee, S., Heeringa, S., Pennell, B.E., Zaslavsky, A.M., Ustun, T.B. & Chatterji, S. (2004). Prevalence, severity, and unmet need for treatment of mental disorders in the World Health Organization World Mental Health Surveys. Journal of American Medical Association 291(21), 2581–2590.Google ScholarPubMed
Dunn, G., Everitt, B. & Pickles, A. (1993). Modelling Covariances and Latent Variables using EQS, 1a ed. Chapman & Hall: London.Google Scholar
Flaherty, J.A., Gaviria, F.M., Pathak, D., Mitchell, T., Wintrob, R., Richman, J.A. & Birz, S. (1988). Developing instruments for crosscultural psychiatric research. Journal of Nervous and Mental Disease 176(5), 257–263.CrossRefGoogle ScholarPubMed
Hanlon, C., Medhin, G., Alem, A., Araya, M., Abdulahi, A., Hughes, M., Tesfaye, M., Wondimagegn, D., Patel, V. & Prince, M. (2007). Detecting perinatal common mental disorders in Ethiopia: Validation of the self-reporting questionnaire and Edinburgh Postnatal Depression Scale. Journal of Affective Disorders 108, 251–262.Google Scholar
Harding, T.W., Climent, C.E., Diop, M., Giel, R., Ibrahim, H.H., Murthy, R.S., Suleiman, M.A., & Wig, N.N. (1983). The WHO collaborative study on strategies for extending mental health care, II: The development of new research methods. American Journal of Psychiatry 140(11), 1474–1480.Google Scholar
Haro, J.M., Arbabzadeh-Bouchez, S., Brugha, T.S., de Girolamo, G., Guyer, M.E., Jin, R., Lepine, J.P., Mazzi, F., Reneses, B., Vilagut, G., Sampson, N.A. & Kessler, R.C. (2006). Concordance of the Composite International Diagnostic Interview Version 3.0 (CIDI 3.0) with standardized clinical assessments in the WHO World Mental Health surveys. International Journal of Methods in Psychiatric Research 15(4), 167–180.CrossRefGoogle ScholarPubMed
Heider, D., Matschinger, H., Bernert, S., Vilagut, G., Martinez-Alonso, M., Dietrich, S. & Angermeyer, M.C. (2005). Empirical evidence for an invariant three-factor structure of the Parental Bonding Instrument in six European countries. Psychiatry Research 135(3), 237–247.CrossRefGoogle ScholarPubMed
Kessler, R.C., Abelson, J., Demler, O., Escobar, J.I., Gibbon, M., Guyer, M.E., Howes, M.J., Jin, R., Vega, W.A., Walters, E.E., Wang, P., Zaslavsky, A. & Zheng, H. (2004). Clinical calibration of DSM-IV diagnoses in the World Mental Health (WMH) version of the World Health Organization (WHO) Composite International Diagnostic Interview (WMHCIDI). International Journal of Methods in Psychiatric Research 13(2), 122–139.CrossRefGoogle ScholarPubMed
Kessler, R.C., Haro, J.M., Heeringa, S.G., Pennell, B.E. & Ustun, T.B. (2006). The World Health Organization World Mental Health Survey Initiative. Epidemiologia e Psichiatria Sociale 15(3), 161–166.Google Scholar
Kleinman, A. (1987). Anthropology and psychiatry. The role of culture in cross-cultural research on illness. British Journal of Psychiatry 151, 447–454.CrossRefGoogle ScholarPubMed
Larraga, L., Saz, P., Dewey, M.E., Marcos, G. & Lobo, A. (2006). Validation of the Spanish version of the EURO-D scale: an instrument for detecting depression in older people. International Journal of Geriatric Psychiatry 21(12), 1199–1205.CrossRefGoogle ScholarPubMed
Manson, S.M., Shore, J.H. & Bloom, J.D. (1985). The depressive experience in American Indian communities. A challenge for psychiatric theory and diagnosis. In Culture and Depression (ed. Kleinman, A. and Good, B.), pp. 331–368. University of California Press: Berkeley.CrossRefGoogle Scholar
Marsh, H.W., Balla, J.R. & Hau, K.T. (1996). An evaluation of incremental fit indices: a clarification of mathematical and empiracal properties. In Advanced Structural Equation Modelling: Issues and Techniques (ed. Marcoulides, G.A. and Schumacker, R.E.), pp. 315–355. Lawrence Erlbaum Associates: Mahwah.Google Scholar
Mezzich, J.E., Kirmayer, L.J., Kleinman, A., Fabrega, H. Jr., Parron, D.L., Good, B.J., Lin, K.M. & Manson, S.M. (1999). The place of culture in DSM-IV. Journal of Nervous and Mental Disease 187(8), 457–464.CrossRefGoogle ScholarPubMed
Mumford, D.B., Bavington, J.T., Bhatnagar, K.S., Hussain, Y., Mirza, S. & Naraghi, M.M. (1991a). The Bradford Somatic Inventory. A multi-ethnic inventory of somatic symptoms reported by anxious and depressed patients in Britain and the Indo-Pakistan subcontinent. British Journal of Psychiatry 158, 379–386.CrossRefGoogle ScholarPubMed
Mumford, D.B., Tareen, I.A., Bajwa, M.A., Bhatti, M.R. & Karim, R. (1991b). The translation and evaluation of an Urdu version of the Hospital Anxiety and Depression Scale. Acta Psychiatrica Scandinaviac 83(2), 81–85.CrossRefGoogle ScholarPubMed
Nunnally, J.C. & Bernstein, I.H. (1994). Psychometric Theory, 3rd ed. McGraw-Hill: New York.Google Scholar
Olsson, U. (1979. Maximum likelihood estimation of the polychoric correlation coefficient. Psychometrika 44, 443–460.CrossRefGoogle Scholar
Patel, V. (2003). Cultural issues in measurement and research. In Practical Psychiatric Epidemiology (ed. Prince, M. et al.). Oxford University Press: Oxford.Google Scholar
Patel, V. & Prince, M. (2001). Ageing and mental health in a developing country: who cares? Qualitative studies from Goa, India. Psychological Medicine 31(1), 29–38.Google Scholar
Patel, V., Musara, T., Butau, T., Maramba, P. & Fuyane, S. (1995). Concepts of mental illness and medical pluralism in Harare. Psychological Medicine 25, 485–493.Google Scholar
Patel, V., Simunyu, E., Gwanzura, F., Lewis, G. & Mann, A. (1997). The Shona Symptom Questionnaire: the development of an indigenous measure of common mental disorders in Harare: Acta Psychiatrica Scandinavica 95(6), 469–475.CrossRefGoogle ScholarPubMed
Prince, M., Beekman, A., Fuhrer, R., Hooijer, C., Kivela, S., Lawlor, B., Lobo, A., Magnusson, H., Meller, I., Oyen Reischies, F., Skoog, I., Turrina, C. & Copeland, J.R.M. (1999a). Depression symptoms in late-life assessed using the EURO-D scale. Effect of age, gender and marital status in 14 European centres. British Journal of Psychiatry 174, 339–345.CrossRefGoogle ScholarPubMed
Prince, M., Reischies, F., Beekman, A.T.F., Fuhrer, R., Jonker, C., Kivela, S.L., Lawlor, B., Lobo, A., Magnusson, H., Fichter, M.M., Van Oyen, H., Roelands, M., Skoog, I., Turrina, C. & Copeland, J.R. (1999b). Development of the EURO-D scale – a European Union initiative to compare symptoms of depression in 14 European centres. British Journal of Psychiatry 174, 330–338.Google Scholar
Prince, M., Acosta, D., Chiu, H., Scazufca, M. & Varghese, M. (2003). Dementia diagnosis in developing countries: a cross-cultural validation study. Lancet 361, 909–917.Google Scholar
Prince, M., Acosta, D., Chiu, H., Copeland, J., Dewey, M., Scazufca, M. & Varghese, M. (2004). Effects of education and culture on the validity of the Geriatric Mental State and its AGECAT algorithm. British Journal of Psychiatry 185, 429–436.CrossRefGoogle ScholarPubMed
Prince, M., Ferri, C.P., Acosta, D., Albanese, E., Arizaga, R., Dewey, M., Gavrilova, S.I., Guerra, M., Huang, Y., Jacob, K.S., Krishnamoorthy, E.S., McKeigue, P., Rodrigues, J.L., Salas, A., Sosa, A.L., Sousa, R., Stewart, R. & Uwakwe, R. (2007). The protocols for the 10/66 Dementia Research Group population-based research programme. BMC Public Health 7, 165.CrossRefGoogle Scholar
Rasch, G. (1993). Probabilistic Models for Some Intelligence and Attainment Tests. Mesa Press: Chicago, IL.Google Scholar
Reise, S.P., Widaman, K.F. & Pugh, R.H. (1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin 114(3), 552–566.CrossRefGoogle ScholarPubMed
Salomon, J.A., Tandon, A., Murray, C.J.L. & World Health Survey Pilot Study Collaborating Group (2004). Comparability of self rated health: cross sectional multi-country survey using anchoring vignettes. British Medical Journal 328(7434), 258.CrossRefGoogle ScholarPubMed
Sartorius, N., Shapiro, R., Kimura, M. & Barrett, K. (1972). WHO international pilot study of schizophrenia. Psychological Medicine 2(4), 422–425.CrossRefGoogle ScholarPubMed
Sartorius, N., Jablensky, A. & Shapiro, R. (1977). Two-year follow-up of patients included in the WHO International Pilot Study of Schizophrenia. Psychological Medicine 7, 529–541.Google Scholar
Sartorius, N., Ustun, T.B., Costa, E.S.J., Goldberg, D., Lecrubier, Y., Ormel, J., Von Korff, M. & Wittchen, H.-U. (1993). An international study of psychological problems in primary care. Preliminary report from the World Health Organization Collaborative Project on Psychological Problems in General Health Care. Archives of General Psychiatry 50(10), 819–824.Google Scholar
Shaji, K.S., Smitha, K., Praveen, Lal K. & Prince, M. (2002). Caregivers of patients with Alzheimer's Disease : A qualitative study from the Indian 10/66 Dementia Research Network. International Journal of Geriatric Psychiatry 18, 1–6.Google Scholar
Simon, G.E., Goldberg, D.P., Von Korff, M. & Ustun, T.B. (2002).Understanding cross-national differences in depression prevalence. Psychological Medicine 32(4), 585–594.Google Scholar
Sorbom, D. (1974). A general method for studying differences in factor means and factor structure between groups. British Journal of Mathematical and Statistical Psychology 27, 229–239.CrossRefGoogle Scholar
Sumathipala, A. & Murray, J. (2001). New approach to translating instruments for cross-cultural research: a combined qualitative and quantitative approach for translation and consensus generation. International Journal of Methods in Psychiatric Research 9(2), 85–97.Google Scholar
Tucker, L. & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika 38, 1–10.Google Scholar
Wickramasinghe, S.C., Rajapakse, L., Abeysinghe, R. & Prince, M. (2002). The Clinical Interview Schedule-Revised (CIS-R): modification and validation in Sri Lanka. International Journal of Methods in Psychiatric Research 11(4), 169–177.CrossRefGoogle Scholar
Wig, N.N., Suleiman, M.A., Routledge, R., Murthy, R.S., Ladrido-Ignacio, L., Ibrahim, H.H. & Harding, T.W. (1980). Community reactions to mental disorders. A key informant study in three developing countries. Acta Psychiatrica Scandinavica 61(2), 111–126.Google Scholar
World Health Organisation (2008). Process of translation and adaptation of instruments. Retrieved May 15, 2008, from http://www. who.int/substance_abuse/research_tools/translation/en/print.htmlGoogle Scholar