Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-745bb68f8f-f46jp Total loading time: 0 Render date: 2025-01-27T04:33:37.924Z Has data issue: false hasContentIssue false

46 - Principles for Educational Assessment with Multimedia

from Part VIII - Multimedia Learning with Media

Published online by Cambridge University Press:  19 November 2021

By
Marlit Annalena Lindner
Edited by
Richard E. Mayer  and
Logan Fiorella
Richard E. Mayer
Affiliation:
University of California, Santa Barbara
Logan Fiorella
Affiliation:
University of Georgia
Get access

Summary

This chapter gives an overview of the main findings regarding the multimedia principle in testing, which states that testing material that combines text and adequate pictures is more comprehensible and more efficient to process for test takers than text alone. Even though learning and testing with multimedia have much in common, substantial differences exist that are discussed in this chapter. Overall, the findings indicate that adding pictures to verbal test material is mainly associated with facilitative effects. However, the findings differ with respect to different outcome measures and depending on the investigated picture type (i.e., decorative, representational, organizational, and informational). Furthermore, effects also appear to be moderated by item and test-taker characteristics that need to be subjected to future research.

Keywords

multimedia principle in testingassessmenttestingcomputer-based assessmentsmultimedia testing
Type
Chapter
Information
The Cambridge Handbook of Multimedia Learning , pp. 552 - 565
Publisher: Cambridge University Press
Print publication year: 2021

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Agathangelou, S., Papakosta, V., & Gagatsis, A. (2008). The impact of iconic representations in solving mathematical one-step problems of the additive structure by primary second grade pupils. In Proceedings of the 11th International Congress of Mathematical Education. July 2008, Monterrey, Mexico.Google Scholar
Ainsworth, S. (2006). DeFT: A conceptual framework for considering learning with multiple representations. Learning and Instruction, 16, 183–198.CrossRefGoogle Scholar
Beitzel, B. D. (2018). Creating diagrams for problem-solving in mathematics: Is it worth the effort? People: International Journal of Social Sciences, 4(1), 690–699.Google Scholar
Beitzel, B. D., Staley, R. K., & DuBois, N. F. (2011a). The (in)effectiveness of visual representations as an aid to solving probability word problems. Effective Education, 3(1), 11–22.CrossRefGoogle Scholar
Beitzel, B. D., Staley, R. K., & DuBois, N. F. (2011b). When best intentions go awry: The failures of concrete representations to help solve probability word problems. Educational Research Quarterly, 34, 3–14.Google Scholar
Berends, I. E., & van Lieshout, E. C. D. M. (2009). The effect of illustrations in arithmetic problem-solving: Effects of increased cognitive load. Learning and Instruction, 19(4), 345–353.CrossRefGoogle Scholar
Beveridge, M., & Parkins, E. (1987). Visual representation in analogical problem solving. Memory & Cognition, 15(3), 230–237.Google Scholar
Boonen, A. J., van Wesel, F., Jolles, J., & van der Schoot, M. (2014). The role of visual representation type, spatial ability, and reading comprehension in word problem solving: An item-level analysis in elementary school children. International Journal of Educational Research, 68, 15–26.Google Scholar
Brase, G. L. (2009). Pictorial representations in statistical reasoning. Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition, 23(3), 369–381.Google Scholar
Carney, R. N., & Levin, J. R. (2002). Pictorial illustrations still improve students’ learning from text. Educational Psychology Review, 14, 5–26.Google Scholar
Chu, J., Rittle‐Johnson, B., & Fyfe, E. R. (2017). Diagrams benefit symbolic problem‐solving. British Journal of Educational Psychology, 87(2), 273–287.Google Scholar
Clinton, V., & Walkington, C. (2019). Interest-enhancing approaches to mathematics curriculum design: Illustrations and personalization. The Journal of Educational Research, 112(4), 495–511.Google Scholar
Cooper, J. L., Sidney, P. G., & Alibali, M. W. (2018). Who benefits from diagrams and illustrations in math problems? Ability and attitudes matter. Applied Cognitive Psychology, 32(1), 24–38.Google Scholar
Crisp, V., & Sweiry, E. (2006). Can a picture ruin a thousand words? The effects of visual resources in exam questions. Educational Research, 48(2), 139–154.Google Scholar
Dewolf, T., van Dooren, W., Ev Cimen, E., & Verschaffel, L. (2014). The impact of illustrations and warnings on solving mathematical word problems realistically. The Journal of Experimental Education, 82(1), 103–120.Google Scholar
Dewolf, T., van Dooren, W., Hermens, F., & Verschaffel, L. (2015). Do students attend to representational illustrations of non-standard mathematical word problems, and, if so, how helpful are they? Instructional Science, 43(1), 147–171.Google Scholar
Dewolf, T., van Dooren, W., & Verschaffel, L. (2017). Can visual aids in representational illustrations help pupils to solve mathematical word problems more realistically? European Journal of Psychology of Education, 32(3), 335–351.Google Scholar
Dindar, M., Kabakçı Yurdakul, I., & Dönmez, F. I. (2015). Measuring cognitive load in test items: Static graphics versus animated graphics. Journal of Computer Assisted Learning, 31(2), 148–161.Google Scholar
Elia, I., Gagatsis, A., & Demetriou, A. (2007). The effects of different modes of representation on the solution of one-step additive problems. Learning and Instruction, 17(6), 658–672.Google Scholar
Elia, I., & Philippou, G. (2004). The functions of pictures in problem solving. In Hoines, M. J., & Fuglestad, A. B. (eds.), Proceedings of the 28th Conference of the International Group for the Psychology of Mathematics Education (Vol. 2., pp. 327–334). Bergen, Norway: Bergen University College.Google Scholar
Gagatsis, A., Agathangelou, S., & Papakosta, V. (2010). Conceptualizing the role of pictures in problem solving by using the implicative statistical analysis. Acta Didactica Universitatis Comenianae Mathematics, 10, 19–34.Google Scholar
Garcia-Retamero, R., Galesic, M., & Gigerenzer, G. (2010). Do icon arrays help reduce denominator neglect? Medical Decision Making, 30(6), 672–684.Google Scholar
Garcia-Retamero, R., & Hoffrage, U. (2013). Visual representation of statistical information improves diagnostic inferences in doctors and their patients. Social Science & Medicine, 83, 27–33.Google Scholar
Ginther, A. (2001). Effects of the presence and absence of visuals on performance on Toefl ® CBT Listening-Comprehension stimuli. ETS Research Report Series, 2001(2), 1–43.Google Scholar
Goldhammer, F., Naumann, J., Stelter, A., Tóth, K., Rölke, H., & Klieme, E. (2014). The time on task effect in reading and problem solving is moderated by task difficulty and skill: Insights from a computer-based large-scale assessment. Journal of Educational Psychology, 106, 608–626.Google Scholar
Goolkasian, P. (1996). Picture-word differences in a sentence verification task. Memory & Cognition, 24, 584–594.Google Scholar
Goolkasian, P. (2000). Pictures, words, and sounds: From which format are we best able to reason?. The Journal of General Psychology, 127(4), 439–459.Google Scholar
Grant, E. R., & Spivey, M. J. (2003). Eye movements and problem solving: Guiding attention guides thought. Psychological Science, 14(5), 462–466.CrossRefGoogle ScholarPubMed
Hao, Y. (2010). Does multimedia help students answer test items? Computers in Human Behavior, 26(5), 1149–1157.Google Scholar
Hoogland, K., de Koning, J., Bakker, A., Pepin, B. E., & Gravemeijer, K. (2018). Changing representation in contextual mathematical problems from descriptive to depictive: The effect on students’ performance. Studies in Educational Evaluation, 58, 122–131.Google Scholar
Hoogland, K., Pepin, B. E., de Koning, J., Bakker, A., & Gravemeijer, K. (2018). Word problems versus image-rich problems: An analysis of effects of task characteristics on students’ performance on contextual mathematics problems. Research in Mathematics Education, 20(1), 37–52.Google Scholar
Hughes, E. M., Riccomini, P. J., & Witzel, B. (2018). Using concrete-representational-abstract sequence to teach fractions to middle school students with mathematics difficulties. Journal of Evidence-Based Practices for Schools, 16, 171–190.Google Scholar
Jarodzka, H., Janssen, N., Kirschner, P. A., & Erkens, G. (2015). Avoiding split attention in computer‐based testing: Is neglecting additional information facilitative?. British Journal of Educational Technology, 46(4), 803–817.Google Scholar
Kirschner, P., Park, B., Malone, S., & Jarodzka, H. (2017). Towards a cognitive theory of multimedia assessment (CTMMA). In Spector, J. M., Lockee, B. B., & Childress, M. D. (eds.), Learning, Design, and Technology: An International Compendium of Theory, Research, Practice, and Policy (pp. 1–23). Cham: Springer.Google Scholar
Koedinger, K. R., Alibali, M. W., & Nathan, M. J. (2008). Trade‐offs between grounded and abstract representations: Evidence from algebra problem solving. Cognitive Science, 32, 366–397.Google Scholar
Lin, Y. H., Wilson, M., & Cheng, C. L. (2013). An investigation of the nature of the influences of item stem and option representation on student responses to a mathematics test. European Journal of Psychology of Education, 28(4), 1141–1161.Google Scholar
Lindner, M. A. (2020). Representational and decorative pictures in science and mathematics tests: Do they make a difference? Learning and Instruction, 68, 101345.Google Scholar
Lindner, M. A., Eitel, A., Barenthien, J., & Köller, O. (2021). An integrative study on learning and testing with multimedia: Effects on students’ performance and metacognition. Learning and Instruction, 71, 1–9.Google Scholar
Lindner, M. A., Eitel, A., Strobel, B., & Köller, O. (2017). Identifying processes underlying the multimedia effect in testing: An eye-movement analysis. Learning and Instruction, 47, 91–102.Google Scholar
Lindner, M. A., Eitel, A., Thoma, G.-B., Dalehefte, I. M., Ihme, J. M., & Köller, O. (2014). Tracking the decision-making process in multiple-choice assessment: Evidence from eye movements. Applied Cognitive Psychology, 28, 738–752.CrossRefGoogle Scholar
Lindner, M. A., Ihme, J. M., Saß, S., & Köller, O. (2018). How representational pictures enhance students’ performance and test-taking pleasure in low-stakes assessment. European Journal of Psychological Assessment, 34, 376–385.Google Scholar
Lindner, M. A., Lüdtke, O., Grund, S., & Köller, O. (2017). The merits of representational pictures in educational assessment: Evidence for cognitive and motivational effects in a time-on-task analysis. Contemporary Educational Psychology, 51, 482–492.Google Scholar
Lindner, M. A., Lüdtke, O., & Nagy, G. (2019). The onset of rapid-guessing behavior over the course of testing time: A matter of motivation and cognitive resources. Frontiers in Psychology, 10, Article 1533, 1–15.CrossRefGoogle Scholar
Lindner, M. A., Schult, S., & Mayer, R. E. (2020). A multimedia effect for multiple-choice and constructed-response test items. Journal of Educational Psychology. Advance online publication. https://doi.org/10.1037/edu0000646Google Scholar
Malone, S., Altmeyer, K., Vogel, M., & Brünken, R. (2020). Homogeneous and heterogeneous multiple representations in equation‐solving problems: An eye‐tracking study. Journal of Computer Assisted Learning, 36, 781–798.Google Scholar
Malone, S., & Brünken, R. (2013). Assessment of driving expertise using multiple choice questions including static vs. animated presentation of driving scenarios. Accident Analysis & Prevention, 51, 112–119.Google Scholar
Mayer, R. E. (2013). Problem solving. In Reisberg, D. (ed.), Oxford Handbook of Cognitive Psychology (pp. 769–778). Oxford: Oxford University Press.Google Scholar
Mayer, R. E. (2019). Problem solving. In McCrudden, M. (ed.), Oxford Research Encyclopedia of Education. Oxford: Oxford University Press.Google Scholar
Mayer, R. E. (2020). Multimedia Learning (3rd ed.). Cambridge: Cambridge University Press.Google Scholar
Múñez, D., Orrantia, J., & Rosales, J. (2013). The effect of external representations on compare word problems: Supporting mental model construction. The Journal of Experimental Education, 81(3), 337–355.Google Scholar
Ögren, M., Nyström, M., & Jarodzka, H. (2017). There’s more to the multimedia effect than meets the eye: Is seeing pictures believing? Instructional Science, 45, 263–287.Google Scholar
Organization for Economic Co-operation and Development [OECD]. (2007). PISA 2006: Science Competencies for Tomorrow’s World: Volume 1: Analysis. Paris: PISA, OECD Publishing.Google Scholar
Ott, N., Brünken, R., Vogel, M., & Malone, S. (2018). Multiple symbolic representations: The combination of formula and text supports problem solving in the mathematical field of propositional logic. Learning and Instruction, 58, 88–105.Google Scholar
Padilla, L., Creem-Regehr, S., Hegarty, M., & Stefanucci, J. (2018). Decision making with visualizations: A cognitive framework across disciplines. Cognitive Research: Principles and Implications, 3, 3–29.Google Scholar
Pinker, S. (1990). A theory of graph comprehension. In Freedle, R. (ed.), Artificial Intelligence and the Future of Testing (pp. 73–126). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.Google Scholar
Prangsma, M. E., van Boxtel, C. A., Kanselaar, G., & Kirschner, P. A. (2009). Concrete and abstract visualizations in history learning tasks. British Journal of Educational Psychology, 79(2), 371–387.Google Scholar
Ramjan, L. M. (2011). Contextualism adds realism: Nursing students’ perceptions of and performance in numeracy skills tests. Nurse Education Today, 31(8), 16–21.Google Scholar
Rowland, C. A. (2014). The effect of testing versus restudy on retention: A meta-analytic review of the testing effect. Psychological Bulletin, 140, 1432–1463.Google Scholar
Saß, S., & Schütte, K. (2016). Helping poor readers demonstrate their science competence: Item characteristics supporting text–picture integration. Journal of Psychoeducational Assessment, 34(1), 91–96.Google Scholar
Saß, S., Schütte, K., & Lindner, M. A. (2017). Test-takers’ eye movements: Effects of integration aids and types of graphical representations. Computers and Education, 109, 85–97.Google Scholar
Saß, S., Wittwer, J., Senkbeil, M., & Köller, O. (2012). Pictures in test items: Effects on response time and response correctness. Applied Cognitive Psychology, 26, 70–81.CrossRefGoogle Scholar
Schnotz, W., & Bannert, M. (2003). Construction and interference in learning from multiple representation. Learning and Instruction, 13, 141–156.Google Scholar
Solano-Flores, G., Wang, C., & Shade, C. (2016). International semiotics: Item difficulty and the complexity of science item illustrations in the PISA-2009 international test comparison. International Journal of Testing, 16(3), 205–219.Google Scholar
Strobel, B., Lindner, M. A., Saß, S., & Köller, O. (2018). Task-irrelevant data impair processing of graph reading tasks: An eye tracking study. Learning and Instruction, 55, 139–147.Google Scholar
Strobel, B., Saß, S., Lindner, M. A., & Köller, O. (2016). Do graph readers prefer the graph type most suited to a given task? Insights from eye tracking. Journal of Eye Movement Research, 9(4), 1–15.Google Scholar
Ullrich, M., Schnotz, W., Horz, H., McElvany, N., Schroeder, S., & Baumert, J. (2012). Kognitionspsychologische Aspekte eines Kompetenzmodells zur Bild-Text-Integration. Psychologische Rundschau, 63, 11–17.Google Scholar
Verschaffel, L., Schukajlow, S., Star, J., & van Dooren, W. (2020). Word problems in mathematics education: A survey. ZDM Mathematics Education, 52, 1–16.Google Scholar
Whitley, K. N., Novick, L. R., & Fisher, D. (2006). Evidence in favor of visual representation for the dataflow paradigm: An experiment testing LabVIEW’s comprehensibility. International Journal of Human–Computer Studies, 64(4), 281–303.Google Scholar
Wise, S. L., Pastor, D. A., & Kong, X. J. (2009). Correlates of rapid-guessing behavior in low-stakes testing: Implications for test development and measurement practice. Applied Measurement in Education, 22, 185–205.Google Scholar
Wu, H. K., Kuo, C. Y., Jen, T. H., & Hsu, Y. S. (2015). What makes an item more difficult? Effects of modality and type of visual information in a computer-based assessment of scientific inquiry abilities. Computers & Education, 85, 35–48.Google Scholar
Yang, D. C., & Huang, F. Y. (2004). Relationships among computational performance, pictorial representation, symbolic representation and number sense of sixth‐grade students in Taiwan. Educational Studies, 30(4), 373–389.Google Scholar
Zhao, F., Schnotz, W., Wagner, I., & Gaschler, R. (2020). Texts and pictures serve different functions in conjoint mental model construction and adaptation. Memory & Cognition, 48(1), 69–82.Google Scholar
Zheng, R., & Cook, A. (2012). Solving complex problems: A convergent approach to cognitive load measurement. British Journal of Educational Technology, 43(2), 233–246.CrossRefGoogle Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×