Hostname: page-component-586b7cd67f-2brh9 Total loading time: 0 Render date: 2024-11-28T18:14:22.586Z Has data issue: false hasContentIssue false
  • English
  • Français

MEASURING COGNITIVE TASK DEMANDS USING DUAL-TASK METHODOLOGY, SUBJECTIVE SELF-RATINGS, AND EXPERT JUDGMENTS

A Validation Study

Published online by Cambridge University Press:  14 September 2015

Andrea Révész ,
Marije Michel  and
Roger Gilabert
Andrea Révész
Affiliation:
University College London
Marije Michel
Affiliation:
Lancaster University
Roger Gilabert
Affiliation:
University of Barcelona
Get access Rights & Permissions [Opens in a new window]

Abstract

This study explored the usefulness of dual-task methodology, self-ratings, and expert judgments in assessing task-generated cognitive demands as a way to provide validity evidence for manipulations of task complexity. The participants were 96 students and 61 English as a second language (ESL) teachers. The students, 48 English native speakers and 48 ESL speakers, carried out simple and complex versions of three oral tasks—a picture narrative, a map task, and a decision-making task. Half of the students completed the tasks under a dual-task condition. The remaining half performed the tasks under a single-task condition without a secondary task. Participants in the single condition were asked to rate their perceived mental effort and task difficulty. The ESL teachers provided expert judgments of anticipated mental effort and task difficulty along with explanations for their ratings via an online questionnaire. As predicted, the more complex task versions were found and judged to pose greater cognitive effort on most measures.

Type
Articles
Information
Studies in Second Language Acquisition , Volume 38 , Issue 4 , December 2016 , pp. 703 - 737
Check for updates
Copyright
Copyright © Cambridge University Press 2015 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Bachman, L. F. (2002). Some reflections on task-based language performance assessment. Language Testing, 19, 453–476.Google Scholar
Baralt, M. (2013). The impact of cognitive complexity on feedback efficacy during online versus face-to-face interactive tasks. Studies in Second Language Acquisition, 35, 689–725.Google Scholar
Brown, J. D., Hudson, T., Norris, J. M., & Bonk, W. (2002). An investigation of second language task-based performance assessments (Technical Report No. 24). University of Hawaii, Second Language Teaching & Curriculum Center.Google Scholar
Brünken, R., Plass, J. L., & Leutner, D. (2004). Assessment of cognitive load in multimedia learning with dual-task methodology: Auditory load and modality effects. Instructional Science, 32, 115–132.CrossRefGoogle Scholar
Brünken, R., Seufert, T., & Paas, F. (2010). Measuring cognitive load. In Plass, J. L., Moreno, R., & Brünken, R. (Eds.), Cognitive load theory (pp. 181–202). Cambridge: Cambridge University Press.Google Scholar
Chandler, P., & Sweller, J. (1996). Cognitive load while learning to use a computer program. Applied Cognitive Psychology, 10, 1–20.Google Scholar
Cierniak, G., Scheiter, K., & Gerjets, P. (2009). Explaining the split-attention effect: Is the reduction of extraneous cognitive load accompanied by an increase in germane cognitive load? Computers in Human Behavior, 25, 315–324.Google Scholar
Cohen, J. (1988). Statistical power analysis for the behavioural sciences (2nd ed.). New York: Academic Press.Google Scholar
Dave, A. (2004). Oxford Placement Test 1. Oxford: Oxford University Press.Google Scholar
Declerck, M., & Kormos, J. (2012). The effect of dual task demands and proficiency on second language speech production. Bilingualism: Language and Cognition, 15, 782–796.Google Scholar
DeKeyser, R. (1997). Beyond explicit rule learning: Automatizing second language morphosyntax. Studies in Second Language Acquisition, 19, 195–221.CrossRefGoogle Scholar
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191.Google Scholar
Fredericks, T. K., Choi, S. D., Hart, J., Butt, S. E., & Mital, A. (2005). An investigation of myocardial aerobic capacity as a measure of both physical and cognitive workloads. International Journal of Industrial Ergonomics, 35, 1097–1107.Google Scholar
Gilabert, R. (2006). The simultaneous manipulation of task complexity along planning time and (+/− here-and-now): Effects on L2 oral production. In Mayo, M. P. García (Ed.), Investigating tasks in formal language learning (pp. 44–68). Clevedon, England: Multilingual Matters.Google Scholar
Gilabert, R. (2007). Effects of manipulating task complexity on self-repairs during L2 oral production. International Review of Applied Linguistics in Language Teaching, 45, 215–240.CrossRefGoogle Scholar
Gilabert, R., & Barón, J. (2013). The impact of increasing task complexity on L2 pragmatic moves. In Mackey, A. & McDonough, K. (Eds.), Second language interaction in diverse educational settings (pp. 45–69). Amsterdam: John Benjamins.Google Scholar
Gilabert, R., Barón, J., & Levkina, M. (2011). Manipulating task complexity across task types and modes. In Robinson, P. (Ed.), Second language task complexity: Researching the cognition hypothesis of language learning and performance (pp. 105–140). Amsterdam: John Benjamins.Google Scholar
Gilabert, R., Barón, J., & Llanes, À. (2009). Manipulating cognitive complexity across task types and its impact on learners’ interaction during oral performance. International Review of Applied Linguistics in Language Teaching, 47, 367–395.Google Scholar
Housen, A., & Kuiken, F. (Eds.). (2009). Complexity, accuracy and fluency in second language acquisition research [Special issue]. Applied Linguistics, 30, 461–601.Google Scholar
Kim, Y. (2009). The effects of task complexity on learner-learner interaction. System, 37, 254–268.Google Scholar
Kim, Y., & Tracy-Ventura, N. (2011). Task complexity, language anxiety, and the development of the simple past. In Robinson, P. (Ed.), Second language task complexity: Researching the cognition hypothesis of language learning and performance (pp. 287–306). Amsterdam: Benjamins.CrossRefGoogle Scholar
Levelt, W. J. M. (1989). Speaking: From intention to articulation. Cambridge, MA: MIT Press.Google Scholar
Levkina, M., & Gilabert, R. (2012). The effects of cognitive task complexity on L2 oral production. In Housen, A., Vedder, I., & Kuiken, F. (Eds.), Dimensions of L2 performance and proficiency investigating complexity, accuracy, and fluency in SLA (pp. 171–198). Amsterdam: John Benjamins.Google Scholar
Levkina, M., & Gilabert, R. (2014). Task sequencing in the L2 development of spatial expressions. In Baralt, M., Gilabert, R., & Robinson, P. (Eds.), Task sequencing and instructed second language learning. New York: Bloomsbury.Google Scholar
Malicka, A., & Levkina, M. (2012). Measuring task complexity: Does L2 proficiency matter? In Shehadeh, A. & Coombe, C. (Eds.), Task-based language teaching in foreign language contexts: Research and implementation (pp. 43–66). Amsterdam: John Benjamins.Google Scholar
Michel, M. (2011). Effects of task complexity and interaction on L2 performance. In Robinson, P. (Ed.), Second language task complexity: Researching the Cognition Hypothesis of language learning and performance (pp. 141–174). Amsterdam: John Benjamins.Google Scholar
Murata, A. (2005). An attempt to evaluate mental workload using wavelet transform of EEG. Human Factors, 47, 498–508.Google Scholar
Norris, J. M. (2010, September). Understanding instructed SLA: Constructs, contexts, and consequences. Plenary address delivered at the annual conference of the European Second Language Association (EUROSLA), Reggio Emilia, Italy.Google Scholar
Norris, J., & Ortega, L. (2003). Defining and measuring L2 acquisition. In Doughty, C. J. & Long, M. H. (Eds.), The handbook of second language acquisition (pp. 717–761). Malden, MA: Blackwell.Google Scholar
Norris, J. M., & Ortega, L. (2009). Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics, 30, 555–578.Google Scholar
Paas, F. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of Educational Psychology, 84, 429–434.Google Scholar
Paas, F., Tuovinen, J., Tabbers, H., & van Gerven, P. W. M. (2003). Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist, 38, 63–71.Google Scholar
Paas, F., & van Merriënboer, J. J. G. (1994). Instructional control of cognitive load in the training of complex cognitive tasks. Educational Psychology Review, 6, 351–372.CrossRefGoogle Scholar
Révész, A. (2009). Task complexity, focus on form, and second language development. Studies in Second Language Acquisition, 31, 437–470.Google Scholar
Révész, A. (2011). Task complexity, focus on L2 constructions, and individual differences: A classroom-based study. Modern Language Journal, 95, 168–181.CrossRefGoogle Scholar
Révész, A. (2014). Towards a fuller assessment of cognitive models of task-based learning: Investigating task-generated cognitive demands and processes. Applied Linguistics, 35, 87–92.Google Scholar
Révész, A., Sachs, R., & Hama, M. (2014). The effects of task complexity and input frequency on the acquisition of the past counterfactual construction through recasts. Language Learning, 64, 615–650.CrossRefGoogle Scholar
Robinson, P. (2001a). Task complexity, cognitive resources, and syllabus design: A triadic framework for investigating task influences on SLA. In Robinson, P. (Ed.), Cognition and second language instruction (pp. 287–318). New York: Cambridge University Press.Google Scholar
Robinson, P. (2001b). Task complexity, task difficulty, and task production: Exploring interactions in a componential framework. Applied Linguistics, 22, 27–57.Google Scholar
Robinson, P. (2003). Attention and memory during SLA. In Doughty, C. J. & Long, M. H. (Eds.), Handbook of second language acquisition (pp. 631–678). Malden, MA: Blackwell.Google Scholar
Robinson, P. (2007). Task complexity, theory of mind, and intentional reasoning: Effects on L2 speech production, interaction, uptake and perceptions of task difficulty. International Review of Applied Linguistics, 45, 193–213.Google Scholar
Robinson, P. (2011). Task-based language learning: A review of issues. Language Learning, 61, 1–36.Google Scholar
Sasayama, S. (2013, October). Is a “complex” task really complex? Measuring task complexity independently from linguistic production. Paper presented at the 5th Biennial International Conference on Task-Based Language Teaching, Banff, Alberta, Canada.Google Scholar
Schneider, W., Eschman, A., & Zuccolotto, A. (2002). E-Prime user’s guide. Pittsburgh, PA: Psychology Software Tools.Google Scholar
Skehan, P. (1998). A cognitive approach to language learning. Oxford: Oxford University Press.Google Scholar
Skehan, P. (2009). Modelling second language performance: Integrating complexity, accuracy, fluency and lexis. Applied Linguistics, 30, 510–532.Google Scholar
Wickens, C. (2007). Attention to the second language. International Review of Applied Linguistics, 45, 177–191.Google Scholar