Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-6bf8c574d5-9f2xs Total loading time: 0 Render date: 2025-03-04T01:03:32.229Z Has data issue: false hasContentIssue false

Part I - Common Assessment Methods Deployed with Internet Technology

Published online by Cambridge University Press:  18 December 2017

Edited by
John C. Scott ,
Dave Bartram  and
Douglas H. Reynolds
John C. Scott
Affiliation:
APT Metrics
Dave Bartram
Affiliation:
CEB-SHL
Douglas H. Reynolds
Affiliation:
Development Dimensions International
Get access

Summary

A summary is not available for this content so a preview has been provided. Please use the Get access link above for information on how to access this content.
Image of the first page of this content. For PDF version, please use the ‘Save PDF’ preceeding this image.'
Type
Chapter
Information
Next Generation Technology-Enhanced Assessment
Global Perspectives on Occupational and Workplace Testing
 , pp. 1 - 68
Publisher: Cambridge University Press
Print publication year: 2017

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

References

Aggarwal, R., Grantcharov, T., Krishna Moorthy, K., Hance, J., & Darzi, A. (2006). A competency-based virtual reality training curriculum for the acquisition of laparoscopic psychomotor skill. The American Journal of Surgery, 191(1), 128–133.CrossRefGoogle ScholarPubMed
Aguinis, H., Henle, C. A., & Beaty, J. C., Jr. (2001). Virtual reality technology: A new tool for personal selection. International Journal of Selection and Assessment, 9, 70–83.CrossRefGoogle Scholar
American Educational Research Association, American Psychological Association, National Council on Measurement in Education, & Joint Committee on Standards for Educational and Psychological Testing. (2014). Standards for educational and psychological testing. Washington, DC: AERA.Google Scholar
Arthur, W., Glaze, R. M., Villado, A. J., & Taylor, J. E. (2010). The magnitude and extent of cheating and response distortion effects on unproctored Internet-based tests of cognitive ability and personality. International Journal of Selection and Assessment, 18, 1–16.CrossRefGoogle Scholar
Back, M. D., Stopfer, J. M., Vazire, S., Gaddis, S., Schmukle, S. C., Egloff, B., & Gosling, S. D. (2010). Facebook profiles reflect actual personality, not self-idealization. Psychological Science, 21, 372–374.CrossRefGoogle Scholar
Barrick, M. R., & Mount, M. K. (1991). The big five personality dimensions and job performance: A meta‐analysis. Personnel Psychology, 44, 1–26.CrossRefGoogle Scholar
Bartram, D. (2011a). Scalar equivalence of OPQ32: Big Five profiles of 31 countries. Journal of Cross-Cultural Psychology, 44(1), 61–83.CrossRefGoogle Scholar
Bartram, D. (2011b). Contributions of the EFPA standing committee on tests and testing to standards and good practice. European Psychologist, 16, 149–159.CrossRefGoogle Scholar
Benartzi, S. (2015). The smarter screen. New York: Penguin Random House.Google Scholar
Bennett, G. K., Seashore, H. G., & Wesman, A. G. (1974). DAT: differential aptitude test. New York: The Psychological Corporation.Google Scholar
Bergman, M. E., & Jean, V. A. (2016). Where have all the “workers” gone? A critical analysis of the unrepresentativeness of our samples relative to the labor market in the industrial-organizational psychology literature. Industrial and Organizational Psychology, 9, 84–113.CrossRefGoogle Scholar
Bideau, B., Kulpa, R., Vignais, N., Brault, S., Multon, F., & Craig, C. (2010). Using virtual reality to analyze sports performance. IEEE Computer Graphics and Applications, 30(2), 14–21.Google ScholarPubMed
Bono, J. E., & Judge, T. A. (2004). Personality and transformational and transactional leadership: A meta-analysis. Journal of Applied Psychology, 89(5), 901–910.CrossRefGoogle ScholarPubMed
Boyce, T. S., Conway, J. S., & Caputo, P. M. (2014). ADEPT-15 technical documentation: Development and validation of Aon Hewitt’s Personality Model and Adaptive Employee Test (ADEPT-15). New York: Aon Hewitt.Google Scholar
Boyce, A. S., Corbet, C. E., & Adler, S. (2013). Simulations in the selection context: Considerations, challenges, and opportunities. In Fetzer, M. S. & Tuzinski, K. A. (Eds.), Simulations for employee selection. New York: Springer.Google Scholar
Bruk-Lee, V., Drew, E. N., & Hawkes, B. (2013). Candidate reactions to simulations and media-rich assessments in personnel selection. In Fetzer, M. S. & Tuzinski, K. A. (Eds.), Simulations for employee selection. New York: Springer.Google Scholar
Buhrmester, M., Kwang, T., & Gosling, S.D. (2011). Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6(1), 3–5.CrossRefGoogle Scholar
Chamorro-Premuzik, T., Winsborough, D., Sherman, R. A., & Hogan, R. (2016). New talent signals: Shiny new objects or brave new world? Industrial-Organizational Psychology Perspectives, 9, 629–640.Google Scholar
Cianciolo, A. T., & Sternberg, R. J. (2004). Intelligence – a brief history. Malden, MA: Blackwell Publishin.CrossRefGoogle Scholar
Cole, M. S., Field, H. S., & Stafford, J. O. (2005). Validity of résumé reviewers’ inferences concerning applicant personality based on résumé evaluation. International Journal of Selection and Assessment, 13(4), 321–324.CrossRefGoogle Scholar
Cottrell, J. M., Newman, D. A., & Roisman, G. I. (2015). Explaining the black–white gap in cognitive test scores: Toward a theory of adverse impact. Journal of Applied Psychology, 100(6), 1713–1736.CrossRefGoogle Scholar
Cucina, J. M., Caputo, P. M., Thibodeaux, H. F., & MacLane, C. N. (2012). Unlocking the key to biodata scoring: A comparison of empirical, rational and hybrid approaches at different sample sizes. Personnel Psychology, 65, 385–428.CrossRefGoogle Scholar
Cucina, J. M., Caputo, P. M., Thibodeaux, H. F., MacLane, C. N., & Bayless, J. M. (2013). Scoring biodata: Is it rational to be quasi-rational? International Journal of Selection and Assessment, 21, 226–232.CrossRefGoogle Scholar
Dalal, R. S., Meyer, R. D., Bradshaw, R. P., Green, J. P., Kelly, E. D., & Zhu, M. (2015). Personality strength and situational influences on behavior: A conceptual review and research agenda. Journal of Management, 41, 261–287.CrossRefGoogle Scholar
Dunnette, M. D. (1962). Personnel management. Annual Review of Psychology, 13, 285–314.CrossRefGoogle Scholar
Fallaw, S. S., Kantrowitz, T. M., & Dawson, C. R. (2012). 2012 Global Assessment Trends Report. SHL.Google Scholar
Fetzer, M., & Tuzinski, K. (2013). Simulations for personnel selection. New York: Springer.CrossRefGoogle Scholar
Gatewood, R. D., Field, H. S., & Barrick, M. (2011). Human resources selection: 7th edition. Mason, OH: South-Western.Google Scholar
Ghiselli, E. E. (1973). The validity of aptitude tests in personnel selection. Personnel Psychology, 26, 461–477.CrossRefGoogle Scholar
Gneezy, U., & List, J. A. (2013). The why axis: Hidden motives and the undiscovered economics of everyday life. New York: Public Affairs.Google Scholar
Goldberg, L. R. (1993). The structure of phenotypic personality traits. American Psychologist, 48, 26–34.CrossRefGoogle Scholar
Gowing, M. K., Morris, D. M., Adler, S., & Gold, M. (2008). The next generation of leadership assessments: Some case studies. Public Personnel Management, 37, 435–455.CrossRefGoogle Scholar
Greenwald, A. G., Banaji, M. R., & Nosek, B. A. (2015). Statistically small effects of the Implicit Association Test can have societally large effects. Journal of Personality and Social Psychology, 108, 553–561.CrossRefGoogle ScholarPubMed
Grimmer, J., & Stewart, B. M. (2013). Text as data: The promise and pitfalls of automatic content analysis methods for political texts. Political Analysis, 21, 267–297.CrossRefGoogle Scholar
Guion, R. M. (1997). Assessment, measurement, and prediction for personnel decisions. Mahwah, NJ: Lawrence Erlbaum Associates, Inc.CrossRefGoogle Scholar
Hoffman, B. J., Kennedy, C. L., LoPilato, A. C., Monahan, E. L., & Lance, C. E. (2015). A review of the content, criterion-related, and construct-related validity of assessment center exercises. Journal of Applied Psychology, 100, 1143.CrossRefGoogle Scholar
Holland, J. L. (1973). Making vocational choices: A theory of careers. Englewood Cliffs, NJ: Prentice Hall.Google Scholar
James, L. R., & McIntyre, M. D. (2000). Conditional reasoning test of aggression test manual. Knoxville, TN: Innovative Assessment Technology.Google Scholar
Johnson, T., & Boyce, A. S. (2015). Selection testing: An updated look at trends in mobile device usage. Presented at the 30th Annual Meeting of the Society for Industrial and Organizational Psychology, Philadelphia, PA.Google Scholar
Judge, T. A., & Bono, J. E. (2001). Relationship of core self-evaluations traits – self-esteem, generalized self-efficacy, locus of control, and emotional stability – with job satisfaction and job performance: A meta-analysis. Journal of Applied Psychology, 86, 80.CrossRefGoogle Scholar
Kosinski, M., Matz, S. C., Gosling, S. D., Popov, V., & Stillwell, D. (2015). Facebook as a research tool for the social sciences: Opportunities, challenges, ethical considerations, and practical guidelines. American Psychologist, 70(6), 543–556.CrossRefGoogle ScholarPubMed
Kosinski, M., Stillwell, D. J., & Graepel, T. (2013). Private traits and attributes are predictable from digital records of human behavior. Proceedings of the National Academy of Sciences of the United States of America, 110(15), 5802–5805.CrossRefGoogle ScholarPubMed
Kristof-Brown, A., & Ryan, A. M. (2003). Focusing on personality in person-organization fit research: Unaddressed issues. In Barrick, M. R. & Ryan, A. M. (Eds.), Personality and work: Reconsidering the role of personality in organizations (pp. 262–288). San Francisco: Jossey-Bass.Google Scholar
Lievens, F., & Burke, E. (2011). Dealing with the threats inherent in unproctored Internet testing of cognitive ability: Results from a large-scale operational test program. Journal of Occupational and Organizational Psychology, 84, 817–824.CrossRefGoogle Scholar
Lilienfeld, S. O., Wood, J. M., & Garb, H. N. (2000). The scientific status of projective techniques. Psychological Science in the Public Interest, 1, 27–66.CrossRefGoogle ScholarPubMed
McDaniel, M. A., Whetzel, D. L., Schmidt, F. L. & Maurer, S. D. (1994). The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79(4), 599–616.CrossRefGoogle Scholar
McNelly, T., Ruggeberg, B. J., & Hall, C. R. (2011). Web‐based management simulations: Technology‐enhanced assessment for executive‐level selection and development. In N. T. Tippins & S. Adler (Eds.), Technology-enhanced assessment of talent. Hoboken, NJ: John Wiley & Sons.Google Scholar
Motowidlo, S., Dunnette, M. D., & Carter, G. W. (1990). An alternative selection procedure: The low-fidelity simulation. Journal of Applied Psychology, 75, 640–647.CrossRefGoogle Scholar
Nye, C. D., Do, B., Drasgow, F., & Fine, S. (2008). Two-step testing in employee selection: Is score inflation a problem? International Journal of Selection and Assessment, 16, 112–120.CrossRefGoogle Scholar
Park, G., Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Kosinski, M., Stillwell, D. J., Ungar, L. H., & Seligman, M. E. P. (2015). Automatic personality assessment through social media language. Journal of Personality and Social Psychology, 108, 934–952.CrossRefGoogle ScholarPubMed
Prinsloo, M., & Prinsloo, R. (2001). The Cognitive Process Profile (CPP) user’s manual. Johannesburg, South Africa: Cognadev.Google Scholar
Provost, F., & Fawcett, T. (2013). Data science for business. Sebastopol, CA: O’Reilly Media.Google Scholar
Ree, M. J., & Earles, J. A. (1994). Predicting training success: Not much more than g. Personnel Psychology, 44, 321–332.CrossRefGoogle Scholar
Ree, M. J., Carretta, T. R., & Teachout, M. S. (2015). Pervasiveness of dominant general factors in organizational measurement. Industrial and Organizational Psychology 8(3), 409–427.CrossRefGoogle Scholar
Ruggs, E. N., Walker, S. S., Blanchard, A., & Gur, S. (2016). Online exclusion: Biases that may arise when using social media in talent acquisition. In Landers, R. N. & Schmidt, G. B. (Eds.), Social media in employee selection and recruitment. Switzerland: Springer.Google Scholar
Ryan, A. M., & Tippins, N. (2009). Designing and implementing global selection systems. Chichester, UK: John Wiley.CrossRefGoogle Scholar
Sackett, P. R., & Lievens, F. (2008). Personnel selection. Annual Review of Psychology, 59, 419–445.CrossRefGoogle ScholarPubMed
Salgado, J. F. (1997). The five factor model of personality and job performance in the European community. Journal of Applied Psychology, 82, 30–43.CrossRefGoogle Scholar
Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection procedures in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2), 262–274.CrossRefGoogle Scholar
Seibert, M., Williams, K., & Raymark, P. (2010). Résumé screening: A policy capturing study of recruiter judgments. Paper presented at 25th Annual Conference of the Society for Industrial & Organizational Psychology, Atlanta, GA.Google Scholar
Seligman, M. E., & Schulman, P. (1986). Explanatory style as a predictor of productivity and quitting among life insurance sales agents. Journal of Personality and Social psychology, 50, 832–838.CrossRefGoogle Scholar
Seymour, N. E., Gallagher, A. G., Roman, S. A., O’Brien, M. K., Bansal, V. K., Andersen, D. K., & Satava, R. M. (2002). Virtual reality training improves operating room performance: Results of a randomized, double-blinded study. Annuals of Surgery, 236(4): 458–464.CrossRefGoogle ScholarPubMed
Smith, D. B. (Ed.). (2008). The people make the place: Dynamic linkages between individuals and organizations. New York: Lawrence Erlbaum Associates.CrossRefGoogle Scholar
Stahl, M. J. (1983). Achievement, power and managerial motivation: Selecting managerial talent with the job choice exercise. Personnel Psychology, 36(4), 775–789.CrossRefGoogle Scholar
Stark, S., Chernyshenko, O. S., Drasgow, F., & White, L. A. (2012). Adaptive testing with multidimensional pairwise preference items: Improving the efficiency of personality and other noncognitive assessments. Organizational Research Methods, 15(3), 463–487.CrossRefGoogle Scholar
Stoughton, J. W. (2016). Applicant reactions to social media in selection: Early returns and future directions. In Landers, R. N. & Schmidt, G. B. (Eds.), Social media in employee selection and recruitment. Switzerland: Springer.Google Scholar
Tippins, N. T., Beatty, J., Drasgow, F., Gibson, W. M., Pearlman, K., Segall, D. O., & Shepherd, W. (2006). Unproctored Internet testing in employment settings. Personnel Psychology, 59, 189–225.CrossRefGoogle Scholar
Van Iddekinge, C. H., Putka, D. J., & Campbell, J. P. (2011). Reconsidering vocational interests for personnel selection: The validity of an interest-based selection test in relation to job knowledge, job performance, and continuance intentions. Journal of Applied Psychology, 96(1), 13–33.CrossRefGoogle ScholarPubMed
Weekley, J. A., & Ployhart, R. E. (Eds.). (2006). Situational judgment tests: Theory, measurement and application. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
Weiner, J. (2010). Verification testing models, values, and considerations. Presented at the 25th Annual Meeting of the Society for Industrial and Organizational Psychology, Atlanta, GA.Google Scholar
Werbach, K., & Hunter, D. (2015). The gamification toolkit: Dynamics, mechanics, and components for the win. Philadelphia: Wharton Digital Press.Google Scholar
Zeidner, M., Matthews, G., & Roberts, R. D. (2004). Emotional intelligence in the workplace: A critical review. Applied Psychology: An International Journal, 53, 371–399.CrossRefGoogle Scholar

References

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (2014). Standards for educational and psychological testing. Washington, DC: AERA.Google Scholar
Allen, M. J., & Yen, W. M. (1979). Introduction to measurement theory. Long Grove, IL: Waveland Press.Google Scholar
Americans with Disabilities Act of 1990. Pub. L. 101–336. 26 July 1990. 104 Stat. 328.Google Scholar
Baker, & McKenzie, (2006). Worldwide guide to termination, employment discrimination, and workplace harassment laws. Ithaca, NY: Cornell ILR School Digital Commons.Google Scholar
Bennett, R. E., & Zhang, M. (2016). Validity and automated scoring. In Drasgow, F. (Ed.), Technology and testing. New York: Routledge.Google Scholar
Bloom, B. S. (Ed.). Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives, handbook I: The cognitive domain. New York: David McKay Co.Google Scholar
Cizek, G. J. (Ed.). (2001). Setting performance standards: Concepts, methods and perspectives. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
Conradt, C. (2012). The game of work. Layton, UT: Gibbs Smith.Google Scholar
Davey, T., & Nering, M. (2002). Controlling item exposure and maintaining item security. In Mills, C. N., Potenza, M. T., Fremer, J. J., & Ward, W. C. (Eds.), Computer-based testing: Building the foundations for future assessments (pp. 165–191). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
Feinberg, R. A., Raymond, M. R., & Haist, S. A. (2015). Repeat testing effects on credentialing exams: Are repeaters misinformed or uninformed? Educational Measurement Issues and Practice, 34(1), 34–39.CrossRefGoogle Scholar
Foster, D., & Miller, H. L., Jr. (2009). A new format for multiple-choice testing: Discrete-Option Multiple-Choice. Results from early studies. Psychology Science Quarterly, 51(4), 355–369.Google Scholar
Foster, D. F., & Zervos, C. (2006). The big Internet heist. Poster presented at the Association of Test Publishers annual conference, Orlando, FL.Google Scholar
Gibson, W. M., & Weiner, J. A. (1998). Generating random parallel test forms using CTT in a computer-based environment. Journal of Educational Measurement, 25, 4.Google Scholar
Gierl, M., & Lai, H. (2013). Educational measurement: Issues and practice: Instructional topics in educational measurement (ITEMS). Module: Using automated processes to generate test items.Google Scholar
Gulino v. The Board of Education of the City of New York and the New York State Education.Google Scholar
The International Standards Organization (ISO) (2012). International Standard ISO/IEC Standard 17024:2012 ISO.Google Scholar
Kolen, M. J., & Brennan, R. L. (1995). Test equating. New York: Springer.CrossRefGoogle Scholar
Krogue, K. (2012, September 18). 5 gamification rules from the grandfather of gamification. Forbes.Google Scholar
Lord, F., & Novick, M. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.Google Scholar
Mehrens, W. A. (1995). Licensure testing: Purposes, procedures, and practices. Paper 6.Google Scholar
Moore, G. (circa 1970). Retrieved from https://en.wikipedia.org/wiki/Moore%27s_law (accessed May 21, 2016).Google Scholar
National Commission for Certifying Agencies (2016). Standards for the accreditation of certification programs. Washington DC: Institute for Credentialling Excellence.Google Scholar
Peng, C. J., & Subkoviak, M. J. (1980). A note on Huynh’s normal approximation procedure for estimating criterion-referenced reliability. Journal of Educational Measurement, 17(4), 359–368.CrossRefGoogle Scholar
Segall, D. O., & Moreno, K. E. (1999). Development of the computerized adaptive testing version of the Armed Services Vocational Batter. In Drasgow, F. & Olson-Buchanan, J. B. (Eds.), Innovations in cauterized assessments. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
Sergeant, (2012). Verification and validation of simulation models. Journal of Simulation, 7, 12–14.CrossRefGoogle Scholar
Shapiro, J. (2015). How a law to protect disabled Americans became imitated around the world. Retrieved from www.npr.org/sections/goatsandsoda/2015/07/24/425607389/how-a-law-to-protect-disabled-americans-became-imitated-around-the-world.Google Scholar
US Copyright Office, Library of Congress (2000). Copyright basics. Washington, DC: Author.Google Scholar
US Copyright Office, Library of Congress (2015). Circular 38a.0915. Washington, DC: Author.Google Scholar
US Equal Employment Opportunity Commission, Department of Labor, Department of Justice, & Civil Service Commission (1978). Section 60–3, Uniform guidelines on employee selection procedures. 43 FR 38295.Google Scholar
Weller, J., Henderson, R., Webster, C., Shulruf, B., Torrie, J., Davies, E., Frampton, C., & Merry, A. (2014). Building the evidence on simulation validity: Comparison of anesthesiologists’ communication patterns in real and simulated cases. Anesthesiology, 120, 1.CrossRefGoogle ScholarPubMed
Werbach, K., & Hunter, D. (2015). For the win: How game thinking can revolutionize your business. Wharton Digital Press.Google Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×