Hostname: page-component-cd9895bd7-lnqnp Total loading time: 0 Render date: 2025-01-05T02:02:10.891Z Has data issue: false hasContentIssue false
  • English
  • Français

Using response times to measure ability on a cognitive task

Published online by Cambridge University Press:  01 January 2025

Aleksandr Alekseev Open the ORCID record for Aleksandr Alekseev [Opens in a new window]
Aleksandr Alekseev*
Affiliation:
Economic Science Institute, Chapman University, One University Drive, 92866 Orange, CA, USA
*
e-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

I show how using response times as a proxy for effort can address a long-standing issue of how to separate the effect of cognitive ability on performance from the effect of motivation. My method is based on a dynamic stochastic model of optimal effort choice in which ability and motivation are the structural parameters. I show how to estimate these parameters from the data on outcomes and response times in a cognitive task. In a laboratory experiment, I find that performance on a digit-symbol test is a noisy and biased measure of cognitive ability. Ranking subjects by their performance leads to an incorrect ranking by their ability in a substantial number of cases. These results suggest that interpreting performance on a cognitive task as ability may be misleading.

Keywords

Cognitive abilityTest scoresResponse timesDrift-diffusion modelChoice-process data

JEL classification

C41: Duration Analysis • Optimal Timing Strategies C24: Truncated and Censored Models • Switching Regression Models • Threshold Regression Models C91: Laboratory, Individual Behavior D91: Role and Effects of Psychological, Emotional, Social, and Cognitive Factors on Decision Making‡ J24: Human Capital • Skills • Occupational Choice • Labor Productivity
Type
Original Paper
Information
Journal of the Economic Science Association , Volume 5 , Issue 1: Special Issue: Choice-Process Data in Experimental Economics , August 2019 , pp. 65 - 75
Copyright
Copyright © Economic Science Association 2019

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Electronic supplementary material The online version of this article (https://doi.org/10.1007/s40881-019-00064-2) contains supplementary material, which is available to authorized users.

References

Agarwal, S., Mazumder, B. (2013). Cognitive abilities and household financial decision making. American Economic Journal: Applied Economics, 5(1), 193–207.Google Scholar
Benndorf, V., Rau, H. A., Sölch, C. (2018). Minimizing learning behavior in repeated real-effort tasks. Working Paper 343, Center for European, Governance and Economic Development Research, Georg-August-Universität Göttingen.Google Scholar
Borghans, L., Duckworth, A. L., Heckman, J. J., Ter Weel, B. (2008). The economics and psychology of personality traits. Journal of Human Resources, 43(4), 972–1059. 10.1353/jhr.2008.0017CrossRefGoogle Scholar
Cattell, R. B. (1971). Abilities: Their structure, growth, and action, Boston: Houghton Mifflin.Google Scholar
Clithero, J. A. (2018). Improving out-of-sample predictions using response times and a model of the decision process. Journal of Economic Behavior & Organization, 148, 344–375. 10.1016/j.jebo.2018.02.007CrossRefGoogle Scholar
Dohmen, T., Falk, A., Huffman, D., Sunde, U. (2010). Are risk aversion and impatience related to cognitive ability? American Economic Review, 100(3), 1238–1260. 10.1257/aer.100.3.1238CrossRefGoogle Scholar
Duckworth, A. L., Quinn, P. D., Lynam, D. R., Loeber, R., Stouthamer-Loeber, M. (2011). Role of test motivation in intelligence testing. Proceedings of the National Academy of Sciences, 108(19), 7716–7720. 10.1073/pnas.1018601108CrossRefGoogle ScholarPubMed
Gill, D., Prowse, V. (2016). Cognitive ability, character skills, and learning to play equilibrium: A level-k analysis. Journal of Political Economy, 124(6), 1619–1676. 10.1086/688849CrossRefGoogle Scholar
Heckman, J., Pinto, R., Savelyev, P. (2013). Understanding the mechanisms through which an influential early childhood program boosted adult outcomes. American Economic Review, 103(6), 2052–2086. 10.1257/aer.103.6.2052CrossRefGoogle ScholarPubMed
Heckman, J. J., Stixrud, J., Urzua, S. (2006). The effects of cognitive and noncognitive abilities on labor market outcomes and social behavior. Journal of Labor Economics, 24(3), 411–482. 10.1086/504455CrossRefGoogle Scholar
Krajbich, I., Lu, D., Camerer, C., Rangel, A. (2012). The attentional drift-diffusion model extends to simple purchasing decisions. Frontiers in psychology, 3, 193 10.3389/fpsyg.2012.00193CrossRefGoogle ScholarPubMed
Lee, M.-L. T., Whitmore, G. (2006). Threshold regression for survival analysis: Modeling event times by a stochastic process reaching a boundary. Statistical Science, 21(4), 501–513. 10.1214/088342306000000330CrossRefGoogle Scholar
Murnane, R., Willett, J. B., Levy, F. (1995). The growing importance of cognitive skills in wage determination. The Review of Economics and Statistics, 77(2), 251–66. 10.2307/2109863CrossRefGoogle Scholar
Ofek, E., Yildiz, M., Haruvy, E. (2007). The impact of prior decisions on subsequent valuations in a costly contemplation model. Management Science, 53(8), 1217–1233. 10.1287/mnsc.1060.0689CrossRefGoogle Scholar
Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85(2), 59 10.1037/0033-295X.85.2.59CrossRefGoogle Scholar
Ratcliff, R., Van Dongen, H. P. A. (2011). Diffusion model for one-choice reaction-time tasks and the cognitive effects of sleep deprivation. Proceedings of the National Academy of Sciences, 108(27), 11285–11290. 10.1073/pnas.1100483108CrossRefGoogle ScholarPubMed
Segal, C. (2012). Working when no one is watching: Motivation, test scores, and economic success. Management Science, 58(8), 1438–1457. 10.1287/mnsc.1110.1509CrossRefGoogle Scholar
Spiliopoulos, L., Ortmann, A. (2018). The BCD of response time analysis in experimental economics. Experimental Economics, 21(2), 383–433. 10.1007/s10683-017-9528-1CrossRefGoogle ScholarPubMed
Vernon, P. A. (1983). Speed of information processing and general intelligence. Intelligence, 7(1), 53–70. 10.1016/0160-2896(83)90006-5CrossRefGoogle Scholar
Webb, R. (2019). The (neural) dynamics of stochastic choice. Management Science, 65(1), 230–255. 10.1287/mnsc.2017.2931CrossRefGoogle Scholar
Weiss, L. G., Saklofske, D. H., Coalson, D. L., Raiford, S. E. (2010). WAIS-IV clinical use and interpretation: Scientist-practitioner perspectives, San Diego: Academic Press.Google Scholar
Wilcox, N. T. (1993). Lottery choice: Incentives, complexity and decision time. The Economic Journal, 103(421), 1397–1417. 10.2307/2234473CrossRefGoogle Scholar
Woodford, M. (2014). Stochastic choice: An optimizing neuroeconomic model. American Economic Review, 104(5), 495–500. 10.1257/aer.104.5.495CrossRefGoogle Scholar
Supplementary material: File

Alekseev supplementary material

Appendices
Download Alekseev supplementary material(File)
File 359.4 KB