Hostname: page-component-cd9895bd7-jn8rn Total loading time: 0 Render date: 2024-12-27T13:30:19.097Z Has data issue: false hasContentIssue false
  • English
  • Français

Cross-Validating Measures of Global Religious Intolerance: Comparing Coded State Department Reports with Survey Data and Expert Opinion

Published online by Cambridge University Press:  18 November 2009

Brian J. Grim  and
Richard Wike
Brian J. Grim*
Affiliation:
Pew Research Center's Forum on Religion & Public Life
Richard Wike*
Affiliation:
Pew Research Center's Global Attitudes Project
*
Address correspondence and reprint requests to: Brian J. Grim, Pew Research Center's Forum on Religion & Public Life. E-mail: [email protected]
Richard Wike, Pew Research Center's Global Attitudes Project. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

We address a pressing substantive issue as well as evaluate several methodologies in this article. Substantively, we ask whether the U.S. State Department has a clear understanding of the level of cross-national religious intolerance that triggers daily headlines around the globe. Methodologically, we ask whether data on social attitudes coded from systematic qualitative reports can reliably represent cross-national public opinion. We empirically address these questions by comparing cross-national levels of religious intolerance coded from the State Department's annual international religious freedom reports with relevant population survey data from the World Values Survey and the Pew Research Center, as well as with data from written interviews of country experts conducted by the Hudson Institute. The results indicate that the understanding of social religious intolerance embodied in the State Department reports is comparable with the results of population surveys and individual expert opinion. Methodologically, this suggests that cross-national public opinion survey data can be cross-validated with coded data from systematic qualitative analysis as well as with expert opinion.

Type
Articles
Information
Politics and Religion , Volume 3 , Issue 1 , April 2010 , pp. 102 - 129
Copyright
Copyright © Religion and Politics Section of the American Political Science Association 2009

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Bahr, Howard M., Caplow, Theodore, and Chadwick, Bruce A.. 1983. “Middletown III: Problems of Replication, Longitudinal Measurement, and Triangulation.” Annual Review of Sociology 9:243–264.CrossRefGoogle Scholar
Barro, Robert J., and McCleary, Rachel M.. 2003. “Religion and Economic Growth Across Nations.” American Sociological Review 68:760–781.CrossRefGoogle Scholar
Berger, Peter. 2006. “Religion in a Globalizing World.” http://pewforum.org/events/?EventID=136 (Accessed on January 7, 2007).Google Scholar
Bratton, Michael. 2002. “Wide but Shallow: Popular Support for Democracy in Africa.” Afrobarometer Paper No. 19.Google Scholar
Crittendon, Kathleen S., and Hill, Richard J.. 1971. “Coding Reliability and Validity of Interview Data.” American Sociological Review 36:1073–1080.CrossRefGoogle Scholar
Curtin, Richard, Presser, Stanley, and Singer, Eleanor. 2000. “The Effect of Response Rate Changes on the Index of Consumer Sentiment.” Public Opinion Quarterly 64:413–428.CrossRefGoogle ScholarPubMed
Danan, Liora, Hunt, Alice, Barton, Rick, and von Hippel, Karin. 2007. “Mixed Blessings: U.S. Government Engagement with Religion in Conflict-prone Settings.” http://www.csis.org/media/csis/pubs/070820_religion.pdf (Accessed on January 4, 2006).Google Scholar
Daves, Robert P., and Newport, Frank. 2005. “Pollsters under Attack: 2004 Election Incivility and Its Consequences.” Public Opinion Quarterly 69:670–681.CrossRefGoogle Scholar
Denzin, Norman K. 1970. The Research Act: A Theoretical Introduction to Sociological Methods. Chicago, IL: Aldine.Google Scholar
Dilulio, John J. Jr. 2007. “Spiritualpolitique: Religion Matters More than Ever in Global Affairs. But Don't Count on the Experts—or the State Department—to know that.”http://www.weeklystandard.com/Content/Public/Articles/000/000/013/603ukfsh.asp.Google Scholar
Eisenstein, Marie A. 2006. “Rethinking the Relationship between Religion and Political Tolerance in the US.” Political Behavior 28:327–348.CrossRefGoogle Scholar
Farr, Thomas F. 2008. “Diplomacy in an Age of Faith: Religious Freedom and National Security.” Foreign Affairs 87:110–124.Google Scholar
Fetzer, Joel S., and Soper, J. Christopher. 2003. “The Roots of Public Attitudes Toward State Accommodation of European Muslims' Religious Practices Before and After September 11.” Journal for the Scientific Study of Religion 42:247–258.CrossRefGoogle Scholar
Finke, Roger, and Stark, Rodney. 2005. The Churching of America 1776–2005: Winners and Losers in Our Religious Economy, 2d ed.New Brunswick, NJ: Rutgers University Press.Google Scholar
Fox, Jonathan. 2008. A World Survey of Religion and the State. New York, NY: Cambridge University Press.CrossRefGoogle Scholar
Frankovic, Kathleen A. 2005. “Reporting ‘the Polls’ in 2004.” Public Opinion Quarterly 69:682–697.CrossRefGoogle Scholar
Funkhouser, , Ray, G., and Parker, Edwin B.. 1968. “Analyzing Coding Reliability: The Random Systematic-Error Coefficient.” The Public Opinion Quarterly 32:122–128.CrossRefGoogle Scholar
Grim, Brian J. 2004. “The Cities of God versus the Countries of Earth: The Regulation of Religious Freedom (RRF).” Presented at the annual meeting of the Association for Study of Religion, Economics, and Culture, October 22, Kansas City, KS.Google Scholar
Grim, Brian J. 2005. “Religious Regulation's Impact on Religious Persecution: The Effects of De Facto and De Jure Religious Regulation.” Ph.D. dissertation, Department of Sociology, The Pennsylvania State University, University Park, PA.Google Scholar
Grim, Brian J. 2008. “Dimensions of Restricted Religious Freedom.” In Religious Freedom in the World 2007, ed. Marshall, P.Lanham, MD: Rowman and Littlefield Publishers. 496–498.Google Scholar
Grim, Brian J., and Finke, Roger. 2005. “Documenting Religion Worldwide: Decreasing the Data Deficit.” IASSIST Quarterly 29:11–16.CrossRefGoogle Scholar
Grim, Brian J., and Finke, Roger. 2006. “International Religion Indexes: Government Regulation, Government Favoritism, and Social Regulation of Religion.” http://www.ReligJournal.com (Accessed on September 3, 2006).Google Scholar
Grim, Brian J., and Finke, Roger. 2007. “Religious Persecution in Cross-National Context: Clashing Civilizations or Regulated Economies?” American Sociological Review 72:633–658.CrossRefGoogle Scholar
Grim, Brian J., Finke, Roger, Harris, Jaime, Meyers, Catherine, and VanEerden, Julie. 2006. “Measuring International Socio-Religious Values and Conflict by Coding U.S. State Department Reports.” JMS Proceedings, AAPOR-Section on Survey Research Methods [CD-ROM]. Alexandria, VA: American Statistical Association: 4120–27.Google Scholar
Grim, Brian J., Harmon, Alison H., and Gromis, Judy C.. 2006. “Focused Group Interviews as an Innovative Quanti-Qualitative Methodology (QQM): Integrating Quantitative Elements into a Qualitative Methodology.” The Qualitative Report 11:516–537.Google Scholar
Groves, Robert M., Cialdini, Robert B., and Couper, Mick P.. 1992. “Understanding the Decision to Participate in a Survey.” Public Opinion Quarterly 56:475–495.CrossRefGoogle Scholar
Groves, Robert M., Fowler, Floyd J. Jr.Couper, Mick P., Lepowski, James M., Singer, Eleanor, and Tourangeau, Roger. 2004. Survey Methodology. Hoboken, NJ: John Wiley & Sons.Google Scholar
Harkness, Janet A., van de Vijver, Fons J. R., and Mohler, Peter Ph.. 2002. Cross-Cultural Survey Methods. Hoboken, N.J: John Wiley-Interscience.Google Scholar
Health, Anthony, Fisher, Stephen, and Smith, Shawna. 2005. “The Globalization of Public Opinion.” Annual Review of Political Science 8:297–333.CrossRefGoogle Scholar
Holbrook, Allyson L., Green, Melanie C., and Krosnick, Jon A.. 2003. “Telephone versus face-to-face interviewing of national probability samples with long questionnaires: Comparisons of respondent satisficing and social desirability response bias.” Public Opinion Quarterly 67:79–125.CrossRefGoogle Scholar
Huffington, Arianna. 1998. “Hang It Up.” http://ariannaonline.huffingtonpost.com/columns/column.php?id=445 (Accessed on December 13, 2007).Google Scholar
Hyman, H.H., and Sheatsley, P.B.. 1950. “The Current Status of American Public Opinion.” In The Teaching of Contemporary Affairs, Twenty-first Yearbook of the National Council of Social Studies, ed. Payne, J.C. Washington, DC: National Council of Social Studies. 11–34.Google Scholar
Igo, Sahar. 2007. The Averaged American: Surveys, Citizens, and the Making of a Mass Public. Cambridge, MA: Harvard University Press.CrossRefGoogle Scholar
Inglehart, Ronald. 2003. “How Solid is Mass Support for Democracy – And How Can We Measure It?” PS: Political Science and Politics 36:51–57.Google Scholar
Inglehart, Ronald, and Welzel, Christian. 2005. Modernization, Cultural Change, and Democracy: The Human Development Sequence. New York, NY: Cambridge University Press.Google Scholar
Jenkins, Philip. 2007. God's Continent: Christianity, Islam, and Europe's Religious Crisis. Oxford, UK: Oxford University Press.Google Scholar
Karpov, Vyacheslav. 2002. “Religiosity and Tolerance in the United States and Poland.” Journal for the Scientific Study of Religion 41:267–288.CrossRefGoogle Scholar
Keeter, Scott, Miller, Carolyn, Kohut, Andrew, Groves, Robert M., and Presser, Stanley. 2000. “Consequences of Reducing Nonresponse in a National Telephone Survey.” Public Opinion Quarterly 64:125–148.CrossRefGoogle Scholar
Keeter, Scott, Kennedy, Courtney, Clark, April, Tompson, Trevor, and Mokrzycki, Mike. 2007. “What's Missing from National Landline RDD Surveys? The Impact of The Growing Cell-Only Population.” Public Opinion Quarterly 71:772–792.CrossRefGoogle Scholar
Kohut, Andrew, and Stokes, Bruce. 2006. America Against the World: How We Are Different and Why We Are Disliked. New York, NY: Henry Holt and Co.Google Scholar
Lang, Gladys Engel, and Lang, Kurt. 1987. “Comments in: The Future Study of Public Opinion: A Symposium.” Public Opinion Quarterly 51:S181–S182.Google Scholar
Lang, Kurt. 2008. “What Polls Can and Cannot Tell us About Public Opinion: Keynote Speech at the 60th Annual Conference of WAPOR.” International Journal of Public Opinion Research 20:3–22.CrossRefGoogle Scholar
Lazarsfeld, Paul F., and Rosenberg, Morris. 1955. The Language of Social Research: A Reader in the Methodology of Social Research. New York, NY: The Free Press.Google Scholar
Lin, I-Fen, and Schaeffer, Nora Cate. 1995. “Using Survey Participants to Estimate the Impact of Nonparticipation.” Public Opinion Quarterly 59:236–258.CrossRefGoogle Scholar
Lynch, Marc. 2003. “Taking Arabs Seriously.” Foreign Affairs 82:81–94.CrossRefGoogle Scholar
Marshall, Paul. 2008. Religious Freedom in the World 2007. Lanham, MD: Rowman and Littlefield Publishers.Google Scholar
Maruyama, Geoffrey M. 1998. Basics of Structural Equation Modeling. Thousand Oaks, CA: Sage.CrossRefGoogle Scholar
Moaddel, Mansoor. 2002. Jordanian Exceptionalism: A Comparative Analysis of Religion and State Relationships in Egypt, Iran, Jordan, and Syria. New York, NY: Palgrave.Google Scholar
Montgomery, Andrew, and Crittendon, Kathleen S.. 1977. “Improving Coding Reliability for Open-ended Questions.” The Public Opinion Quarterly 41:235–243.CrossRefGoogle Scholar
Netjes, Catherine E., and Binnema, Harmen A.. 2007. “The Salience of the European Integration Issue: Three Data Sources Compared.” Electoral Studies 26:39–49.CrossRefGoogle Scholar
Neuman, W. Russell. 1990. “The Threshold of Public Attention.” Public Opinion Quarterly 54: 159–176.CrossRefGoogle Scholar
Norris, Pipa. 1999. “Institutional Explanations for Political Support.” In Critical Citizens: Global Support for Democratic Governance, ed. Norris, Pipa.Oxford, UK: Oxford University Press. 217–235.CrossRefGoogle Scholar
Norris, Pippa, and Inglehart, Ronald. 2008. “Polls for the Public Good: Mass and Elite Evaluations of the Health of Democratic Governance.” Paper presented at the annual meeting of the World Assosication of Public Opinion Researchers, New Orleans, May 14.Google Scholar
North, Charles M., and Gwin, Carl R.. 2004. “Religious Freedom and the Unintended Consequences of the Establishment of Religion.” Southern Economic Journal, 71:103–117.Google Scholar
Pew Forum. 2006. “Spirit and Power: A Ten-Nation Study of Pentecostals.” http://pewforum.org/surveys/pentecostal (Accessed on October 5, 2006).Google Scholar
Pew Global Attitudes Project. 2006. “The Great Divide: How Westerners and Muslims View Each Other.” http://pewglobal.org (Accessed on September 3, 2006).Google Scholar
Putnam, Robert D., Leonardi, Robert, and Nanetti, Rffaella Y.. 1993. Making Democracy Work: Civic Traditions in Modern Italy. Princeton, NJ: Princeton University Press.Google Scholar
Riley, John W. Jr. 1962. “Reflections on Data Sources in Opinion Research.” Public Opinion Quarterly 26:313–322.CrossRefGoogle Scholar
Schuman, Howard, Kalton, Graham, and Ludwig, Jacob. 1983. “Context and Contiguity in Survey Questionnaires.” The Public Opinion Quarterly 47:112–115.CrossRefGoogle Scholar
Shah, Timothy Samuel, and Toft, Monica Duffy. 2006. “Why God is Winning.” Foreign Policy, July/August:39–43.Google Scholar
Smith, Pamela Hyde. 2007. “Politics and Diplomacy: The Hard Road Back to Soft Diplomacy.” Georgetown Journal of International Affairs. http://www.12.georgetown.edu/sfs/publications/journal/81/highres/smith.cfm (Accessed August 2008).Google Scholar
Teitler, Julien O., Reichman, Nancy E., and Sprachman, Susan. 2003. “Costs and Benefits of Improving Response Rates for a Hard-to-Reach Population.” Public Opinion Quarterly 67:126–138.CrossRefGoogle Scholar
Tetlock, Philip. 2006. Expert Political Judgment: How Good Is It? How Can We Know? Princeton, NJ: Princeton University Press.Google Scholar
Johan D, Van der Vyver. 2006. “Prospects for Converting the Declaration into a Binding Convention.” Presented at the U.S. State Department commemoration of the 25th anniversary of the United Nations' Declaration on the Elimination of All Forms of Intolerance and of Discrimination Based on Religion or Belief, October 30, Washington, DC.Google Scholar
Webb, Eugene J., Campbell, Donald T., Schwartz, Richard D., Sechrest, Lee, and Grove, Janet Belew. 1981. Nonreactive Measures in the Social Sciences, 2nd Ed.Boston, MA: Houghton Mifflin.Google Scholar
Webb, Eugene J., Campbell, Donald T., Schwartz, Richard D., and Sechrest, Lee. 1966. Nonreactive Measures in the Social Sciences. Chicago, IL: Rand McNally.Google Scholar
Wike, Richard, and Grim, Brian J.. 2007. “Levels of Negativity: How Muslim and Western Publics See One Another.” http://www.publicopinionpros.com/features/2007/oct/wike.asp (Accessed on October 2007).Google Scholar
Wiktorowicz, Quintan. 2001. The Management of Islamic Activism: Salafis, the Muslim Brotherhood, and State Power in Jordan. Albany, NY: State University of New York.Google Scholar
Willis, Gordon B., Caspar, Rachel A., and Lessler, Judith T.. 1999. “Cognitive Interviewing: A ‘How to’ Guide,”http://appliedresearch.cancer.gov/areas/cognitive/interview.pdf (Accessed October 2003).Google Scholar
Yamagishi, T., and Yamagishi, M.. 1994. “Trust and Commitment in the United States and Japan.” Motivation and Emotion 18:129–66.CrossRefGoogle Scholar