Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-745bb68f8f-hvd4g Total loading time: 0 Render date: 2025-01-27T04:15:36.477Z Has data issue: false hasContentIssue false

37 - Multimedia Learning with Cognitive Tutors

from Part VIII - Multimedia Learning with Media

Published online by Cambridge University Press:  19 November 2021

By
Kenneth Koedinger  and
Vincent Aleven
Edited by
Richard E. Mayer  and
Logan Fiorella
Richard E. Mayer
Affiliation:
University of California, Santa Barbara
Logan Fiorella
Affiliation:
University of Georgia
Get access

Summary

Cognitive Tutors are effective AI-based learning environments. The following statement summarizes three core instructional features from past summaries of deliberate practice (shown in italics) along with three elaborations (shown in bold): Good learning-by-doing instruction requires repeated practice on well-tailored tasks in varied contexts with explanatory feedback and as-needed instruction. This chapter describes these six learning-by-doing principles and how they are achieved in Cognitive Tutors. We correct some key misconceptions about cognitive tutors, including that knowledge component (KC) decomposition does not exclude, but includes conceptual connections, teachers are not replaced but valued, and up-front-telling is not the instructional focus whereas learning-by-doing guidance is. We also point to a need for more experimentation on the benefits of as-needed versus up-front instruction, better integration of supports for enhancing student motivation, and better pathways for teachers to participate in co-design (during the inevitable need for Learning Engineering beyond Learning Science) and customization.

Keywords

cognitive tutorsartificial intelligencelearning-by-doingdeliberate practice
Type
Chapter
Information
The Cambridge Handbook of Multimedia Learning , pp. 439 - 449
Publisher: Cambridge University Press
Print publication year: 2021

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Aleven, V., & Koedinger, K. R. (2002). An effective metacognitive strategy: Learning by doing and explaining with a computer-based Cognitive Tutor. Cognitive Science, 26(2), 147–179.Google Scholar
Aleven, V., & Koedinger, K. R. (2013). Knowledge component approaches to learner modeling. In Sottilare, R., Graesser, A., Hu, X., & Holden, H. (eds.), Design Recommendations for Adaptive Intelligent Tutoring Systems (Learner Modeling, 1, pp. 165–182). Orlando, FL: US Army Research Laboratory.Google Scholar
Aleven, V., McLaren, B. M., Roll, I., & Koedinger, K. R. (2016). Help helps, but only so much: Research on help seeking with intelligent tutoring systems. International Journal of Artificial Intelligence in Education, 26(1), 205–223.Google Scholar
Aleven, V., McLaren, B. M., Sewall, J., van Velsen, M., Popescu, O., Demi, S., Ringenberg, M., & Koedinger, K. R. (2016). Example-tracing tutors: Intelligent tutor development for non-programmers. International Journal of Artificial Intelligence in Education, 26(1), 224–269.Google Scholar
Aleven, V., McLaughlin, E. A., Glenn, R. A., & Koedinger, K. R. (2017). Instruction based on adaptive learning technologies. In Mayer, R. E., & Alexander, P. (eds.), Handbook of Research on Learning and Instruction (2nd ed., pp. 522–560). New York: Routledge.Google Scholar
Anderson, J. R., Conrad, F. G., & Corbett, A. T. (1989). Skill acquisition and the LISP tutor. Cognitive Science, 13(4), 467–505.Google Scholar
Anderson, J. R., Corbett, A. T., Koedinger, K., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. The Journal of Learning Sciences, 4, 167–207.Google Scholar
Baker, R. S. J. D., Corbett, A. T., & Koedinger, K. R. (2007). The difficulty factors approach to the design of lessons in intelligent tutor curricula. International Journal of Artificial Intelligence in Education, 17(4), 341–369.Google Scholar
Butcher, K., & Aleven, V. (2013). Using student interactions to foster rule-diagram mapping during problem solving in an intelligent tutoring system. Journal of Educational Psychology, 105(4), 988–1009.Google Scholar
Carvalho, P. F., & Goldstone, R. L. (2014). Putting category learning in order: Category structure and temporal arrangement affect the benefit of interleaved over blocked study. Memory & Cognition, 42, 481–495.Google Scholar
Carvalho, P. F., & Goldstone, R. L. (2019). When does interleaving practice improve learning?. In Dunlosky, J., & Rawson, K. (eds.), The Cambridge Handbook of Cognition and Education (pp. 411–436). Cambridge: Cambridge University Press.Google Scholar
Chi, M. T., & Wylie, R. (2014). The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist, 49(4), 219–243.Google Scholar
Clark, R. E., Feldon, D., van Merriënboer, J., Yates, K., & Early, S. (2007). Cognitive task analysis. In Spector, J. M., Merrill, M. D., van Merrie¨nboer, J. J. G., & Driscoll, M. P. (eds.), Handbook of Research on Educational Communications and Technology (3rd ed., pp. 577–593). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
Corbett, A. T., & Anderson, J. R. (1995). Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4, 253–278.Google Scholar
Corbett, A. T., & Anderson, J. R. (2001). Locus of feedback control in computer-based tutoring: Impact on learning rate, achievement and attitudes. In CHI ‘01: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 245–252). New York: ACM Press.Google Scholar
Corbett, A. T., McLaughlin, M., & Scarpinatto, K. C. (2000). Modeling student knowledge: Cognitive Tutors in high school and college. User Modeling and User-Adapted Interaction, 10, 81–108.Google Scholar
Deslauriers, L., McCarty, L. S., Miller, K., Callaghan, K., & Kestin, G. (2019). Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proceedings of the National Academy of Sciences, 116(39), 19251–19257.Google Scholar
Deslauriers, L., Schelew, E., & Wieman, C. (2011). Improved learning in a large-enrollment physics class. Science, 332(603), 862–864.Google Scholar
Doroudi, S., Holstein, K., Aleven, V., & Brunskill, E. (2015). Towards understanding how to leverage sense-making, induction/refinement and fluency to improve robust learning. In Santos, O. C., Boticario, J. G., Romero, C., Pechenizkiy, M., Merceron, A., Mitros, P., Luna, J. M., Mihaescu, C., Moreno, P., Hershkovitz, A., Ventura, S., & Desmarais, M. (eds.), Proceedings of the 8th International Conference on Educational Data Mining, EDM 2015 (pp. 376–379). Worcester, MA: International Educational Data Mining Society.Google Scholar
Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100(3), 363–406.Google Scholar
Ericsson, K. A. & Simon, H. A. (1985). Protocol analysis. In van Dijk, T. A. (ed.), Handbook of Discourse Analysis (Vol. 2; pp. 259–268). London: Academic Press.Google Scholar
Fries, L., Son, J. Y., Givvin, K. B. & Stigler, J. W. (2020). Practicing connections: A framework to guide instructional design for developing understanding in complex domains. Educational Psychology Review, 33, 739–762.Google Scholar
Hattie, J. (2008). Visible Learning: A Synthesis of over 800 Meta-Analyses Relating to Achievement. New York: Routledge.Google Scholar
Holstein, K., Aleven, V., & Rummel, N. (2020). A conceptual framework for human–AI hybrid adaptivity in education. In Bittencourt, I., Cukurova, M., Muldner, K., Luckin, R., & Millán, E. (eds.), Proceedings, 21th International Conference on Artificial Intelligence in Education, AIED 2020 (pp. 240–254). Cham: Springer.Google Scholar
Holstein, K., McLaren, B. M., & Aleven, V. (2018). Student learning benefits of a mixed-reality teacher awareness tool in AI-enhanced classrooms. In Rosé, C. P., Martínez-Maldonado, R., Hoppe, H. U., Luckin, R., Mavrikis, M., Porayska-Pomsta, K., McLaren, B., & du Boulay, B. (eds.), Proceedings, 19th International Conference on Artificial Intelligence in Education, AIED 2018 (Part 1, pp. 154–168). Cham: Springer.Google Scholar
Holstein, K., McLaren, B. M., & Aleven, V. (2019). Co-designing a real-time classroom orchestration tool to support teacher–AI complementarity. Journal of Learning Analytics, 6(2), 27–52.Google Scholar
Huang, Y., Aleven, V., McLaughlin, E., & Koedinger, K. (2020). A general multi-method approach to design-loop adaptivity in intelligent tutoring systems. Artificial Intelligence in Education: 21st International Conference, AIED 2020, Ifrane, Morocco, July 6–10, 2020, Proceedings, Part II, 12164, 124–129.Google Scholar
Kalyuga, S., Chandler, P., Tuovinen, J., & Sweller, J. (2001). When problem solving is superior to studying worked examples. Journal of Educational Psychology, 93, 579–588.Google Scholar
Kellogg, R. T., & Whiteford, A. P. (2009). Training advanced writing skills: The case for deliberate practice. Educational Psychologist, 44, 250–266.CrossRefGoogle Scholar
Kellman, P. J., & Krasne, S. (2018). Accelerating expertise: Perceptual and adaptive learning technology in medical learning. Medical Teacher, 40(8), 797–802.Google Scholar
Koedinger, K. R. (2002). Toward evidence for instructional design principles: Examples from Cognitive Tutor Math 6. Invited paper. In Mewborn, D., Sztajn, P., White, D. Y., Wiegel, H. G., Bryant, R. L., & Nooney, K. (eds.), Proceedings of the 24th Annual Meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education (pp. 21–49). Columbus, OH: ERIC Clearinghouse for Science, Mathematics, and Environmental Education.Google Scholar
Koedinger, K. R. & Aleven, V. (2007). Exploring the assistance dilemma in experiments with Cognitive Tutors. Educational Psychology Review, 19(3), 239–264.Google Scholar
Koedinger, K. R., & Aleven, V. (2016). An interview reflection on “intelligent tutoring goes to school in the big city.” International Journal of Artificial Intelligence in Education, 26(1), 13–24.Google Scholar
Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8(1), 30–43.Google Scholar
Koedinger, K. R., Booth, J. L., & Klahr, D. (2013). Instructional complexity and the science to constrain it. Science, 342, 935–937.Google Scholar
Koedinger, K. R., & Corbett, A. T. (2006). Cognitive Tutors: Technology bringing learning sciences to the classroom. In Sawyer, R. K. (ed.), The Cambridge Handbook of the Learning Sciences (pp. 61–78). New York: Cambridge University Press.Google Scholar
Koedinger, K. R., Corbett, A. C., & Perfetti, C. (2012). The Knowledge-Learning-Instruction (KLI) framework: Bridging the science-practice chasm to enhance robust student learning. Cognitive Science, 36(5), 757–798.Google Scholar
Koedinger, K. R. & McLaughlin, E. A. (2010). Seeing language learning inside the math: Cognitive analysis yields transfer. In Ohlsson, S., & Catrambone, R. (eds.), Proceedings of the 32nd Annual Conference of the Cognitive Science Society (pp. 471–476). Austin, TX: Cognitive Science Society.Google Scholar
Koedinger, K. R., & McLaughlin, E. A. (2016). Closing the loop with quantitative cognitive task analysis. In Barnes, T., Chi, M., and Feng, M. (eds.), Proceedings of the 9th International Conference on Educational Data Mining (pp. 412–417). Raleigh, NC: International Conference on Educational Data Mining (EDM).Google Scholar
Koedinger, K. R., McLaughlin, E. A., & Stamper, J. C. (2012). Automated student model improvement. In Yacef, K., Zaïane, O., Hershkovitz, H., Yudelson, M., & Stamper, J. (eds.), Proceedings of the 5th International Conference on Educational Data Mining (pp. 17–24). Greece: Chania.Google Scholar
Li, N., Cohen, W. W., & Koedinger, K. R. (2013). Problem order implications for learning. International Journal of Artificial Intelligence in Education, 23(1–4), 71–93.Google Scholar
Li, N., Matsuda, N., Cohen, W. W., and Koedinger, K. R. (2015). Integrating representation learning and skill learning in a human-like intelligent agent. Artificial Intelligence, 219, 67–91.Google Scholar
Liu, R., & Koedinger, K. R. (2017). Closing the loop: Automated data-driven cognitive model discoveries lead to improved instruction and learning gains. Journal of Educational Data Mining, 9(1), 25–41.Google Scholar
Liu, R., Koedinger, K. R., & McLaughlin, E. A. (2014). Interpreting model discovery and testing generalization to a new dataset. In Stamper, J., Pardos, Z., Mavrikis, M., & McLaren, B. M. (eds.), Proceedings of the 7th International Conference on Educational Data Mining (pp. 107–113). Worcester, MA: International Conference on Educational Data Mining.Google Scholar
Lovett, M. C. (1998). Cognitive task analysis in service of intelligent tutoring system design: A case study in statistics. In International Conference on Intelligent Tutoring Systems (pp. 234–243). Berlin: Springer.Google Scholar
Lovett, M. C., Meyer, O., & Thille, C. (2008). JIME-The open learning initiative: Measuring the effectiveness of the OLI statistics course in accelerating student learning. Journal of Interactive Media in Education, 2008(1), Art. 13.Google Scholar
MacLellan, C. J. (2017). Computational Models of Human Learning: Applications for Tutor Development, Behavior Prediction, and Theory Testing [Doctoral Dissertation]. Human-Computer Interaction Institute, Carnegie Mellon University, Pittsburgh, PA.Google Scholar
MacLellan, C. J., Harpstead, E., Patel, R., & Koedinger, K. (2016). The apprentice learner architecture: Closing the loop between learning theory and educational data. In Proceedings of the 9th International Conference in Educational Data Mining (pp. 151–158). Worcester, MA: International Educational Data Mining Society.Google Scholar
Martin, B., Mitrovic, T., Mathan, S., & Koedinger, K.R. (2011). Evaluating and improving adaptive educational systems with learning curves. User Modeling and User-Adapted Interaction: The Journal of Personalization Research (UMUAI), 21(3), 249–283.Google Scholar
Mathan, S. A., & Koedinger, K. R. (2005). Fostering the intelligent novice: Learning from errors with metacognitive tutoring. Educational Psychologist, 40(4), 257–265.Google Scholar
Mayer, R. E. (2020). Multimedia Learning (3rd ed.), New York: Cambridge University Press.Google Scholar
McDaniel, M. A., Agarwal, P. K., Huelser, B. J., McDermott, K. B., & Roediger, H. L. III (2011). Test-enhanced learning in a middle school science classroom: The effects of quiz frequency and placement. Journal of Educational Psychology, 103(2), 399.Google Scholar
Mitrovic, A., Ohlsson, S., & Barrow, D. K. (2013). The effect of positive feedback in a constraint-based intelligent tutoring system. Computers & Education, 60(1), 264–272.Google Scholar
Moreno, R. (2004). Decreasing cognitive load for novice students: Effects of explanatory versus corrective feedback in discovery‐based multimedia. Instructional Science, 32, 99–113.Google Scholar
Moreno, R., & Mayer, R. E. (2005). Role of guidance, reflection, and interactivity in an agent‐based multimedia game. Journal of Educational Psychology, 97, 117–128.Google Scholar
Nagashima, T., Bartel, A. N., Silla, E., Vest, N., Alibali, M. W., & Aleven, V. (2020). Enhancing conceptual knowledge in early algebra through scaffolding diagrammatic self-explanation. In Proceedings of International Conference of the Learning Sciences, ICLS 2020 (Part 1, pp. 35–42). Nashville, TN: International Society of the Learning Sciences.Google Scholar
Ohlsson, S. (1994). Constraint based student modeling. In Greer, J. E., & McCalla, G. (eds.), Student Modelling: The Key to Individualized Knowledge-Based Instruction. NATO ASI Series (Series F: Computer and Systems Sciences) (vol. 125, pp. 167–189). Berlin: Springer.Google Scholar
Paas, F. G. W. C., & van Merriënboer, J. J. G. (1994). Variability of worked examples and transfer of geometrical problem-solving skills: A cognitive-load approach. Journal of Educational Psychology, 86(1), 122–133.Google Scholar
Pane, J. F., Griffin, B. A., McCaffrey, D. F., & Karam, R. (2014). Effectiveness of cognitive tutor algebra I at scale. Educational Evaluation and Policy Analysis, 36(2), 127–144.Google Scholar
Patel, R. (2017). Addressing Interference in Fraction Learning: What Difficulties are Desirable? [Doctoral Dissertation]. Human-Computer Interaction Institute, Carnegie Mellon University, Pittsburgh, PA.Google Scholar
Patel, R., Liu, R., & Koedinger, K. (2016). When to block versus interleave practice? Evidence against teaching fraction addition before fraction multiplication. In Papafragou, A., Grodner, D., Mirman, D., & Trueswell, J. C. (eds.), Proceedings of the 38th Annual Conference of the Cognitive Science Society (pp. 2069–2074). Austin, TX: Cognitive Science Society.Google Scholar
Rau, M. A. (2017). Conditions for the effectiveness of multiple visual representations in enhancing STEM learning. Educational Psychology Review, 29(4), 717–761.Google Scholar
Rau, M. A., Aleven, V., & Rummel, N. (2015). Successful learning with multiple graphical representations and self-explanation prompts. Journal of Educational Psychology, 107(1), 30–46.Google Scholar
Ritter, S., Anderson, J. R., Koedinger, K. R., & Corbett, A. (2007). Cognitive tutor: Applied research in mathematics education. Psychonomic Bulletin & Review, 14(2), 249–255.Google Scholar
Roediger, H. L., and Karpicke, J. D. (2006a). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17, 249–255.Google Scholar
Roediger, H. L., and Karpicke, J. D. (2006b). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1, 181–210.Google Scholar
Rohrer, D. (2012). Interleaving helps students distinguish among similar concepts. Educational Psychology Review, 24, 355–367.Google Scholar
Rohrer, D., & Taylor, K. (2007). The shuffling of mathematics practice problems boosts learning. Instructional Science, 35, 481–498.Google Scholar
Roll, I., Baker, R. S. J. D., Aleven, V., & Koedinger, K. R. (2014) On the benefits of seeking (and avoiding) help in online problem- solving environments. Journal of the Learning Sciences, 23(4), 537–560.Google Scholar
Schnackenberg, H. L., Sullivan, H. J., Leader, L. F., & Jones, E. E. K. (1998). Learner preferences and achievement under differing amounts of learner practice. Educational Technology Research and Development, 46, 5–16.Google Scholar
Schofield, J. W. (1995). Computers and Classroom Culture. New York: Cambridge University Press.Google Scholar
Schooler, L. J., & Anderson, J. R. (1990). The disruptive potential of immediate feedback. In Piattelli-Palmarini, M. (ed.), Proceedings of the Twelfth Annual Conference of the Cognitive Science Society (pp. 702–708). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.Google Scholar
Shih, B., Koedinger, K. R., & Scheines, R. (2008). A response time model for bottom-out hints as worked examples. In Proceedings of the First International Conference on Educational Data Mining, 2008, Montreal, QC, pp. 117–126.Google Scholar
Tofel-Grehl, C., & Feldon, D. F. (2013). Cognitive task analysis-based training: A meta-analysis of studies. Journal of Cognitive Engineering and Decision Making, 7(2), 293–304.Google Scholar
van der Kleij, F. M., Feskens, C. W. R., & Eggen, T. J. H. M. (2015). Effects of feedback in a computer‐based learning environment on students’ learning outcomes: A meta‐analysis. Review of Educational Research, 85(4), 475–511.Google Scholar
VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197–221.Google Scholar
VanLehn, K. (2016). Regulative loops, step loops and task loops. International Journal of Artificial Intelligence in Education, 26(1), 107–112.Google Scholar
Weitekamp, D., Harpstead, E., & Koedinger, K. R. (2020). An interaction design for machine teaching to develop AI tutors. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, April 2020, Honolulu, HI, pp. 1–11.Google Scholar
Wylie, R., Sheng, M., Mitamura, T., & Koedinger, K. R. (2011). Effects of adaptive prompted self-explanation on robust learning of second language grammar. In International Conference on Artificial Intelligence in Education (pp. 588–590). Berlin: Springer.Google Scholar
Yannier, N., Hudson, S. E., & Koedinger, K. R. (2020). Active learning is about more than hands-on: A mixed-reality AI system to support STEM education. International Journal of Artificial Intelligence in Education, 30(1), 74–96.Google Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×