Hostname: page-component-586b7cd67f-t7czq Total loading time: 0 Render date: 2024-11-27T23:07:30.937Z Has data issue: false hasContentIssue false
  • English
  • Français

Capturing Rationalization Bias and Differential Item Functioning: A Unified Bayesian Scaling Approach

Published online by Cambridge University Press:  07 January 2020

Jørgen Bølstad Open the ORCID record for Jørgen Bølstad [Opens in a new window]
Jørgen Bølstad*
Affiliation:
Associate Professor of Political Science, University of Stavanger, Norway. Email: [email protected]
*
*Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Information about the ideological positions of different political actors is crucial in answering questions regarding political representation, polarization, and voting behavior. One way to obtain such information is to ask survey respondents to place actors on a common ideological scale, but, unfortunately, respondents typically display a set of biases when performing such placements. Key among these are rationalization bias and differential item functioning (DIF). While Aldrich–McKelvey (AM) scaling offers a useful solution to DIF, it ignores the issue of rationalization bias, and this study presents Monte Carlo simulations demonstrating that AM-type models thus can give inaccurate results. As a response to this challenge, this study develops an alternative Bayesian scaling approach, which simultaneously estimates DIF and rationalization bias, and therefore performs better when the latter bias is present.

Keywords

ideal point estimationBayesian estimationcognitive biasspatial voting modelsurvey research
Type
Articles
Information
Political Analysis , Volume 28 , Issue 3 , July 2020 , pp. 340 - 355
Copyright
Copyright © The Author(s) 2020. Published by Cambridge University Press on behalf of the Society for Political Methodology.

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Contributing Editor: Jeff Gill

References

Ahler, D. J., and Broockman, D. E.. 2018. “The Delegate Paradox: Why Polarized Politicians Can Represent Citizens Best.” The Journal of Politics 80(4):1117–1133.CrossRefGoogle Scholar
Aldrich, J. H., and McKelvey, R. D.. 1977. “A Method of Scaling With Applications to the 1968 and 1972 Presidential Elections.” The American Political Science Review 71(1):111–130.CrossRefGoogle Scholar
Aronson, E. 1997. “Back to the Future: Retrospective Review of Leon Festinger’s ‘A Theory of Cognitive Dissonance’.” The American Journal of Psychology 110(1):127–137.CrossRefGoogle Scholar
Bafumi, J., Gelman, A., Park, D. K., and Kaplan, N.. 2005. “Practical Issues in Implementing and Understanding Bayesian Ideal Point Estimation.” Political Analysis 13(2):171–187.CrossRefGoogle Scholar
Bafumi, J., and Herron, M. C.. 2010. “Leapfrog Representation and Extremism: A Study of American Voters and Their Members in Congress.” American Political Science Review 104(03):519–542.CrossRefGoogle Scholar
Bakker, R., De Vries, C., Edwards, E., Hooghe, L., Jolly, S., Marks, G., Polk, J., Rovny, J., Steenbergen, M., and Vachudova, M. A.. 2015. “Measuring Party Positions in Europe: The Chapel Hill Expert Survey Trend File, 1999–2010.” Party Politics 21(1):143–152.CrossRefGoogle Scholar
Bakker, R., Jolly, S., Polk, J., and Poole, K.. 2014. “The European Common Space: Extending the Use of Anchoring Vignettes.” The Journal of Politics 76(4):1089–1101.CrossRefGoogle Scholar
Barber, M., and Pope, J. C.. 2019. “Does Party Trump Ideology? Disentangling Party and Ideology in America.” American Political Science Review 113(1):38–54.CrossRefGoogle Scholar
Barberá, P. 2015. “Birds of the Same Feather Tweet Together: Bayesian Ideal Point Estimation Using Twitter Data.” Political Analysis 23(1):76–91.CrossRefGoogle Scholar
Bølstad, J.2019. “Replication Materials for: Capturing Rationalization Bias and Differential Item Functioning: A Unified Bayesian Scaling Approach.” https://doi.org/10.7910/DVN/HXAKGJ, Harvard Dataverse, V1.CrossRefGoogle Scholar
Bølstad, J., and Dinas, E.. 2017. “A Categorization Theory of Spatial Voting: How the Center Divides the Political Space.” British Journal of Political Science 47(4):829–850.CrossRefGoogle Scholar
Brady, H. E. 1985. “The Perils of Survey Research: Inter-Personally Incomparable Responses.” Political Methodology 11(3/4):269–291.Google Scholar
Broockman, D. E. 2016. “Approaches to Studying Policy Representation.” Legislative Studies Quarterly 41(1):181–215.CrossRefGoogle Scholar
van der Brug, W. 2001. “Perceptions, Opinions and Party Preferences in the Face of a Real World Event Chernobyl as a Natural Experiment in Political Psychology.” Journal of Theoretical Politics 13(1):53–80.CrossRefGoogle Scholar
Carpenter, B., Gelman, A., Hoffman, M., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., and Riddell, A.. 2017. “Stan: A Probabilistic Programming Language.” Journal of Statistical Software 76(1):1–32.CrossRefGoogle Scholar
Claassen, R. L. 2007. “Ideology and Evaluation in an Experimental Setting.” Political Research Quarterly 60(2):263–273.CrossRefGoogle Scholar
Clinton, J., Jackman, S., and Rivers, D.. 2004. “The Statistical Analysis of Roll Call Data.” American Political Science Review 98(2):355–370.CrossRefGoogle Scholar
Conover, P. J., and Feldman, S.. 1982. “Projection and the Perception of Candidates’ Issue Positions.” Western Political Quarterly 35(2):228–244.CrossRefGoogle Scholar
EES. 2011. “European parliament election study 2009, voter study, advance release, 23/06/2011.” http://www.piredeu.eu.Google Scholar
van der Eijk, C., van der Brug, W., Kroh, M., and Franklin, M.. 2006. “Rethinking the Dependent Variable in Voting Behavior: On the Measurement and Analysis of Electoral Utilities.” Electoral Studies 25(3):424–447.CrossRefGoogle Scholar
van der Eijk, C., and Franklin, M. N.. 1996. Choosing Europe? The European Electorate and National Politics in the Face of Union. Ann Arbor: The University of Michigan Press.CrossRefGoogle Scholar
Festinger, L. 1957. A Theory of Cognitive Dissonance. Evanston, IL: Row, Peterson.Google Scholar
Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., and Rubin, D. B.. 2014. Bayesian Data Analysis. 3rd edn.London: Chapman & Hall/CRC Press.Google Scholar
Gelman, A., Jakulin, A., Pittau, M. G., and Su, Y.-S.. 2008. “A Weakly Informative Default Prior Distribution for Logistic and Other Regression Models.” The Annals of Applied Statistics 2(4):1360–1383.CrossRefGoogle Scholar
Hare, C., Armstrong, D. A., Bakker, R., Carroll, R., and Poole, K. T.. 2015. “Using Bayesian Aldrich-McKelvey Scaling to Study Citizens’ Ideological Preferences and Perceptions.” American Journal of Political Science 59(3):759–774.CrossRefGoogle Scholar
Hoffman, M. D., and Gelman, A.. 2014. “The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo.” Journal of Machine Learning Research 15(1):1593–1623.Google Scholar
Imai, K., Lo, J., and Olmsted, J.. 2016. “Fast Estimation of Ideal Points With Massive Data.” American Political Science Review 110(4):631–656.CrossRefGoogle Scholar
Jackman, S. 2001. “Multidimensional Analysis of Roll Call Data via Bayesian Simulation: Identification, Estimation, Inference, and Model Checking.” Political Analysis 9(3):227–241.CrossRefGoogle Scholar
Jessee, S. 2016. “(How) Can We Estimate the Ideology of Citizens and Political Elites on the Same Scale? ” American Journal of Political Science 60(4):1108–1124.CrossRefGoogle Scholar
Jessee, S. A. 2009. “Spatial Voting in the 2004 Presidential Election.” American Political Science Review 103(1):59–81.CrossRefGoogle Scholar
Johnston, R., Fournier, P., and Jenkins, R.. 2000. “Party Location and Party Support: Unpacking Competing Models.” The Journal of Politics 62(4):1145–1160.CrossRefGoogle Scholar
King, G., Murray, C. J., Salomon, J. A., and Tandon, A.. 2004. “Enhancing the Validity and Cross-cultural Comparability of Measurement in Survey Research.” American Political Science Review 98(1):191–207.CrossRefGoogle Scholar
Kroh, M. 2007. “Measuring Left–Right Political Orientation: The Choice of Response Format.” Public Opinion Quarterly 71(2):204–220.CrossRefGoogle Scholar
Krosnick, J. A. 1990. “Americans’ Perceptions of Presidential Candidates: A Test of the Projection Hypothesis.” Journal of Social Issues 46(2):159–182.CrossRefGoogle Scholar
Lacy, D., and Paolino, P.. 2010. “Testing Proximity Versus Directional Voting Using Experiments.” Electoral Studies 29(3):460–471.CrossRefGoogle Scholar
Lewis, J. B., and King, G.. 1999. “No Evidence on Directional vs. Proximity Voting.” Political analysis 8(1):21–33.CrossRefGoogle Scholar
Lo, J., Proksch, S.-O., and Gschwend, T.. 2014. “A Common Left-Right Scale for Voters and Parties in Europe.” Political Analysis 22(2):205–223.CrossRefGoogle Scholar
Markus, G. B., and Converse, P. E.. 1979. “A Dynamic Simultaneous Equation Model of Electoral Choice.” American Political Science Review 73(4):1055–1070.CrossRefGoogle Scholar
Merrill, S. I. 1995. “Discriminating Between the Directional and Proximity Spatial Models of Electoral Competition.” Electoral Studies 14(3):273–287.CrossRefGoogle Scholar
Merrill, S., Grofman, B., and Adams, J.. 2001. “Assimilation and Contrast Effects in Voter Projections of Party Locations: Evidence From Norway, France, and the USA.” European Journal of Political Research 40(2):199–221.CrossRefGoogle Scholar
Palfrey, T. R., and Poole, K. T.. 1987. “The Relationship Between Information, Ideology, and Voting Behavior.” American Journal of Political Science 31(3):511–530.CrossRefGoogle Scholar
Poole, K. T. 1998. “Recovering a Basic Space From a Set of Issue Scales.” American Journal of Political Science 42(3):954–993.CrossRefGoogle Scholar
Poole, K. T., and Rosenthal, H.. 1985. “A Spatial Model for Legislative Roll Call Analysis.” American Journal of Political Science 29(2):357–384.CrossRefGoogle Scholar
Poole, K., Rosenthal, H., Lewis, J., Lo, J., and Carroll, R.. 2013. “Basicspace: A package to recover a basic space from issue scales.” http://CRAN.R-project.org/package=basicspace.Google Scholar
Rabinowitz, G., and Macdonald, S. E.. 1989. “A Directional Theory of Issue Voting.” American Political Science Review 83(1):93–121.CrossRefGoogle Scholar
Rubin, D. B. et al. . 1984. “Bayesianly Justifiable and Relevant Frequency Calculations for the Applied Statistician.” The Annals of Statistics 12(4):1151–1172.CrossRefGoogle Scholar
Saiegh, S. M. 2009. “Recovering a Basic Space from Elite Surveys: Evidence from Latin America.” Legislative Studies Quarterly 34(1):117–145.CrossRefGoogle Scholar
Stan Development Team. 2018. “RStan: The R interface to Stan.” http://mc-stan.org. R package version 2.17.3.Google Scholar
Stan Development Team. 2019. “Stan user’s guide.” http://mc-stan.org. Version 2.19.Google Scholar
Tomz, M., and Van Houweling, R. P.. 2008. “Candidate Positioning and Voter Choice.” American Political Science Review 102(3):303–318.CrossRefGoogle Scholar
Watanabe, S. 2010. “Asymptotic Equivalence of Bayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory.” Journal of Machine Learning Research 11:3571–3594.Google Scholar
Westholm, A. 1997. “Distance Versus Direction: The Illusory Defeat of the Proximity Theory of Electoral Choice.” American Political Science Review 91(4):865–883.CrossRefGoogle Scholar
Supplementary material: Link

Bølstad Dataset

Dataset

https://doi.org/10.7910/DVN/HXAKGJ
Link
Supplementary material: File

Bølstad supplementary material

Bølstad supplementary material

Download Bølstad supplementary material(File)
File 1.1 MB