Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-cd9895bd7-q99xh Total loading time: 0 Render date: 2024-12-25T04:16:45.812Z Has data issue: false hasContentIssue false
Series:   Elements in Quantitative and Computational Methods for the Social Sciences

Adaptive Inventories

A Practical Guide for Applied Researchers

Published online by Cambridge University Press:  08 July 2022

Jacob M. Montgomery  and
Erin L. Rossiter
Jacob M. Montgomery
Affiliation:
Washington University, St Louis
Erin L. Rossiter
Affiliation:
University of Notre Dame, Indiana

Summary

The goal of this Element is to provide a detailed introduction to adaptive inventories, an approach to making surveys adjust to respondents' answers dynamically. This method can help survey researchers measure important latent traits or attitudes accurately while minimizing the number of questions respondents must answer. The Element provides both a theoretical overview of the method and a suite of tools and tricks for integrating it into the normal survey process. It also provides practical advice and direction on how to calibrate, evaluate, and field adaptive batteries using example batteries that measure variety of latent traits of interest to survey researchers across the social sciences.
Get access

Keywords

survey methodscomputerized adaptive testinglatent variableactive learningmeasurement
Type
Element
Information
Series: Elements in Quantitative and Computational Methods for the Social Sciences
Online ISBN: 9781108862516
Publisher: Cambridge University Press
Print publication: 28 July 2022

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Altemeyer, Bob. 1988. Enemies of freedom: Understanding right-wing authoritarianism. San Fransisco, CA: Jossey-Bass.Google Scholar
Ames, Daniel R., Rose, Paul, and Anderson, Cameron P.. 2006. “The NPI-16 as a short measure of narcissism.” Journal of Research in Personality 40(4): 440–450.CrossRefGoogle Scholar
Baker, Frank B., and Kim, Seock-Ho. 2004. Item response theory: Parameter estimation techniques. New York: Marcel Dekker.CrossRefGoogle Scholar
Bakker, Bert N., and Lelkes, Yphtach. 2018. “Selling ourselves short? How abbreviated measures of personality change the way we think about personality and politics.” The Journal of Politics 80(4): 1311–1325.Google Scholar
Berinsky, Adam J., Margolis, Michele F., Sances, Michael W., and Warshaw, Christopher. 2019. “Using screeners to measure respondent attention on self-administered surveys: Which items and how many?” Political Science Research and Methods, 9(2),430–437.Google Scholar
Birnbaum, Allan. 1968. “Some latent train models and their use in inferring an examinee’s ability.” In Lord, F. M. and Novick, M. R. (eds.), Statistical theories of mental test scores (395–479). Reading, MA: Addison-Wesley.Google Scholar
Bizer, George Y. Krosnick, Jon A., Petty, Richard E., Rucker, Derek D., and Christian Wheeler, S.. 2000. National Election Studies Report. “Need for cognition and need to evaluate in the 1998 national election survey pilot study.”Google Scholar
Cacioppo, John T., and Petty, Richard E.. 1982. “The need for cognition.” Journal of Personality & Social Psychology 42(1): 116–131.Google Scholar
Chang, Hua-Hua, and Ying, Zhiliang. 1996. “A global information approach to computerized adaptive testing.” Applied Psychological Measurement 20(3): 213–229.Google Scholar
Chang, Hua-Hua, and Ying, Zhiliang. 1999. “A-stratified multistage computerized adaptive testing.” Applied Psychological Measurement 23(3): 211–222.Google Scholar
Chen, Shu-Ying, Ankenmann, Robert D., and Chang, Hua-Hua. 2000. “A comparison of item selection rules at the early stages of computerized adaptive testing.” Applied Psychological Measurement 24(3): 241–255.Google Scholar
Choi, Seung W., and Swartz, Richard J.. 2009. “Comparison of CAT item selection criteria for polytomous items.” Applied Psychological Measurement 33(6): 419–440.CrossRefGoogle ScholarPubMed
Costa, Paul T., and McCrae, Robert R.. 2008. “The revised neo personality inventory (neo-pi-r).” In Boyle, G. J., Matthews, G., and Saklofske, D. H. (eds.), The SAGE handbook of personality theory and assessment, Vol. 2. Personality measurement and testing (179–198). SAGE Publications.Google Scholar
Carpini, Delli, Michael, X., and Keeter, Scott. 1993. “Measuring political knowledge: Putting first things first.” American Journal of Political Science 37(4): 1179–1206.Google Scholar
Carpini, Delli, Michael, X., and Keeter, Scott. 1996. What Americans know about politics and why it matters. New Haven, CT: Yale University Press.Google Scholar
Dodd, Barbara G., De Ayala, R. J., and Koch, William R.. 1995. “Computerized adaptive testing with polytomous items.” Applied Psychological Measurement 19(1): 5–22.Google Scholar
Donnellan, M. Brent, Oswald, Frederick L. Baird, Brendan M. Lucas, Richard E. 2006. “The mini-IPIP scales: Tiny-yet-effective measures of the big five factors of personality.” Psychological Assessment 18(2): 192–203.Google Scholar
Druckman, James N. 2004. “Political preference formation: Competition, deliberation, and the (ir)relevance of framing effects.” American Political Science Review 98(4): 671–686.CrossRefGoogle Scholar
Embretson, Susan E. 1996. “The new rules of measurement.” Psychological Assessment 8(4): 341–349.CrossRefGoogle Scholar
Enamorado, Ted. 2018. “Active learning for probabilistic record linkage.” Social Science Research Network (SSRN). URL: https://ssrn.com/abstract53257638.CrossRefGoogle Scholar
Groves, Robert M., and Heeringa, Steven G.. 2006. “Responsive design for household surveys: Tools for actively controlling survey errors and costs.” Journal of the Royal Statistical Society: Series A (Statistics in Society) 169(3): 439–457.Google Scholar
Herzog, A. Regula, and Bachman, Jerald G.. 1981. “Effects of questionnaire length on response quality.” Public Opinion Quarterly 45(4): 549–559.Google Scholar
Hetherington, Marc J., and Weiler, Jonathan D.. 2009. Authoritarianism and polarization in American politics. Cambridge: Cambridge University Press.Google Scholar
Hetherington, Marc, and Suhay, Elizabeth. 2011. “Authoritarianism, threat, and Americans’ support for the War on Terror.” American Journal of Political Science 55(3): 546–560.CrossRefGoogle Scholar
Jarvis, W. Blair G., and Petty, Richard E.. 1996. “The need to evaluate.” Journal of Personality and Social Psychology 70(1): 172–194.Google Scholar
Kaufman, Aaron R. 2020. “Implementing novel, flexible, and powerful survey designs in r shiny.” PloS one 15(4): e0232424.Google Scholar
Kingsbury, G. Gage, and Weiss, David J.. 1983. “A comparison of IRT-based adaptive mastery testing and a sequential mastery testing procedure.” In Weiss, David J., (ed.), New horizons in testing: Latent trait test theory and computerized adaptive testing, (257–283). New York: Academic Press.Google Scholar
Miller, Blake, Linder, Fridolin, and Mebane, Walter R.. 2019. “Active learning approaches for labeling text: Review and assessment of the performance of active learning approaches.” Political Analysis 28(4),532–551.CrossRefGoogle Scholar
Montgomery, Jacob M., and Cutler, Josh. 2013. “Computerized adaptive testing for public opinion surveys.” Political Analysis 21(2): 172–192.CrossRefGoogle Scholar
Montgomery, Jacob M., and Rossiter, Erin L.. 2017. “CatSurv: Computerized adaptive testing for survey research.” https://CRAN.R-project.org/package=catSurv.Google Scholar
Montgomery, Jacob M., and Rossiter, Erin L.. 2020. “So many questions, so little time: Integrating adaptive inventories into public opinion research.” Journal of Survey Statistics and Methodology, 8(4),667–690.Google Scholar
Moore, Ryan T., and Moore, Sally A.. 2013. “Blocking for sequential political experiments.” Political Analysis 21(4): 507.CrossRefGoogle ScholarPubMed
Muraki, Eiji. 1992. “A generalized partial credit model: Application of an em algorithm.” ETS Research Report Series 1992(1): i–30.Google Scholar
Muraki, Eiji, and Muraki, Mari. 2016. “Generalized partial credit model.” In van der Linden, Wim J. (ed.), Handbook of item response theory, vol.1 (155–166). Boca Raton, FL: Chapman & Hall/CRC, p. 155–166.Google Scholar
Offer-Westort, Molly, Coppock, Alexander, and Green, Donald P.. 2021. “Adaptive experimental design: Prospects and applications in political science.” American Journal of Political Science 65(4): 826–844.Google Scholar
Ooms, Jeroen. 2014. “The OpenCPU system: Towards a universal interface for scientific computing through separation of concerns.” https://arxiv.org/pdf/1406.4806.pdf.Google Scholar
Pawitan, Yudi. 2001. In all likelihood: Statistical modelling and inference using likelihood. Oxford: Oxford University Press.Google Scholar
Raskin, Robert, and Terry, Howard. 1988. “A principal-components analysis of the narcissistic personality inventory and further evidence of its construct validity.” Journal of Personality and Social Psychology 54(5): 890–902.Google Scholar
Rizopoulos, Dimitris. 2006. “Ltm: An rR package for latent variable modeling and item response theory analyses.” Journal of Statistical Software 17(5): 1–25.CrossRefGoogle Scholar
Salganik, Matthew J., and Karen, E. C. Levy. 2015. “Wiki surveys: Open and quantifiable social data collection.” PloS one 10(5): e0123483.Google Scholar
Samejima, Fumiko. 1969. “Estimation of latent ability using a response pattern of graded scores.” Psychometrika Monograph Supplement 34(4): 100, Number 17.Google Scholar
Schwartz, Shalom H. 1992. “Universals in the content and structure of values: Theoretical advances and empirical tests in 20 countries.” In Zanna, Mark P. (ed.), Advances in experimental social psychology, vol. 25 (1–65). New York: Academic Press.Google Scholar
Segall, Daniel O. 1996. “Multidimensional adaptive testing.” Psychometrika 61(2): 331–354.Google Scholar
Segall, Daniel O. 2005. “Computerized adaptive testing.” In Kempf-Leonard, Kimberly (ed.), Encylopedia of social measurement, (429–438). Oxford: Elsevier.CrossRefGoogle Scholar
Sheatsley, Paul B. 1983. “Questionnaire construction and item writing.” In Rossi, P. H., Wright, J. D., and Anderson, A. B. (eds.), Handbook of survey research (195–230). San Diego, CA: Academic Press.Google Scholar
Sidanius, Jim Pratto, Felicia, Colette, Van Laar, Levin, Shana. 2004. “Social dominance theory: Its agenda and method.” Political Psychology 25(6): 845–880.Google Scholar
Stillwell, David J., and Kosinski, Michal. 2004. “MyPersonality project: Example of successful utilization of online social networks for large-scale social research.” American Psychologist 59(2): 93–104.Google Scholar
van der Linden, Wim J. 1998. “Bayesian item selection criteria for adaptive testing.” Psychometrika 63(2): 201–216.CrossRefGoogle Scholar
van der Linden, Wim J., and Pashley, Peter J.. 2010. Elements of adaptive testing. New York: Springer.Google Scholar
Velez, Yamil Ricardo, and Lavine, Howard. 2017. “Racial diversity and the dynamics of authoritarianism.” The Journal of Politics 79(2): 519–533.Google Scholar
Wang, Wei, Tay, Louis, and Drasgow, Fritz. 2013. “Detecting differential item functioning of polytomous items for an ideal point response process.” Applied Psychological Measurement 37(4): 316–335.Google Scholar
Warm, Thomas A. 1989. “Weighted likelihood estimation of ability in item response theory.” Psychometrika 54(3): 427–450.CrossRefGoogle Scholar
Weiss, David J. 1982. “Improving measurement quality and efficiency with adaptive testing.” Applied Psychological Measurement 6(4): 473–492.CrossRefGoogle Scholar
Weiss, David J., and Gage Kingsbury, G.. 1984. “Application of computerized adaptive testing to educational problems.” Journal of Educational Measurement 21(4): 361–375.Google Scholar

Save element to Kindle

To save this element to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Adaptive Inventories
Available formats
×

Save element to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Adaptive Inventories
Available formats
×

Save element to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Adaptive Inventories
Available formats
×