Hostname: page-component-cd9895bd7-dzt6s Total loading time: 0 Render date: 2024-12-26T18:15:23.938Z Has data issue: false hasContentIssue false
  • English
  • Français

An editable learner model for text recommendation for language learning

Published online by Cambridge University Press:  30 June 2021

John S. Y. Lee Open the ORCID record for John S. Y. Lee [Opens in a new window]
John S. Y. Lee*
Affiliation:
Department of Linguistics and Translation, City University of Hong Kong, Hong Kong SAR, China ([email protected])
Get access Rights & Permissions [Opens in a new window]

Abstract

Extracurricular reading is important for learning foreign languages. Text recommendation systems typically classify users and documents into levels, and then match users with documents at the same level. Although this approach can be effective, it has two significant shortcomings. First, the levels assume a standard order of language acquisition and cannot be personalized to the users’ learning patterns. Second, recommendation decisions are not transparent because the leveling algorithms can be difficult for users to interpret. We propose a novel method for text recommendation that addresses these two issues. To enhance personalization, an open, editable learner model estimates user knowledge of each word in the foreign language. The documents are ranked by new-word density (NWD) – that is, the percentage of words that are new to the user in the document. The system then recommends documents according to a user-specified target NWD. This design offers complete transparency as users can scrutinize recommendations by reviewing the NWD estimation of the learner model. This article describes an implementation of this method in a mobile app for learners of Chinese as a foreign language. Evaluation results show that users were able to manipulate the learner model and NWD parameters to adjust the difficulty of the recommended documents. In a survey, users reported satisfaction with both the concept and implementation of this text recommendation method.

Keywords

text recommendationtext difficultyopen learner modelpersonalizationnew-word density
Type
Research Article
Information
ReCALL , Volume 34 , Issue 1 , January 2022 , pp. 51 - 65
Check for updates
Copyright
© The Author(s), 2021. Published by Cambridge University Press on behalf of European Association for Computer Assisted Language Learning

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Anderson, J. (1983) Lix and rix: Variations on a little-known readability index. Journal of Reading, 26(6): 490–496. http://www.jstor.org/stable/40031755 Google Scholar
Brisbois, J. E. (1995) Connections between first- and second-language reading. Journal of Reading Behavior, 27(4): 565–584. https://doi.org/10.1080/10862969509547899 CrossRefGoogle Scholar
Brown, J. & Eskenazi, M. (2004) Retrieval of authentic documents for reader-specific lexical practice. In Proceedings of the InSTIL/ICALL 2004 Symposium on Computer Assisted Learning. Venice, Italy, 17–19 June.Google Scholar
Brusilovsky, P. (2012) Adaptive hypermedia for education and training. In Durlach, P. J. & Lesgold, A. M. (eds.), Adaptive technologies for training and education. New York: Cambridge University Press, 46–66. https://doi.org/10.1017/CBO9781139049580.006 CrossRefGoogle Scholar
Buendgens-Kosten, J. (2013) Authenticity in CALL: Three domains of ‘realness’. ReCALL, 25(2): 272–285. https://doi.org/10.1017/S0958344013000037 CrossRefGoogle Scholar
Bull, S. & Kay, J. (2007) Student models that invite the learner in: The SMILI Open Learner Modelling Framework. International Journal of Artificial Intelligence in Education, 17(2): 89–120.Google Scholar
Bull, S. & Kay, J. (2010) Open learner models. In Nkambou, R., Bourdeau, J. & Mizoguchi, R. (eds.), Advances in intelligent tutoring systems: Studies in computational intelligence, Vol. 308. Berlin: Springer, 301–322. https://doi.org/10.1007/978-3-642-14363-2_15 CrossRefGoogle Scholar
Bull, S., Pain, H. & Brna, P. (1995) Mr. Collins: A collaboratively constructed, inspectable student model for intelligent computer assisted language learning. Instructional Science, 23: 65–87. https://doi.org/10.1007/BF00890446 CrossRefGoogle Scholar
Coleman, M. & Liau, T. L. (1975) A computer readability formula designed for machine scoring. Journal of Applied Psychology, 60(2): 283–284. https://doi.org/10.1037/h0076540 CrossRefGoogle Scholar
Collins-Thompson, K., Bennett, P. N., White, R. W., de la Chica, S. & Sontag, D. (2011) Personalizing web search results by reading level. In Berendt, B., de Vries, A., Fan, W., Macdonald, C., Ounis, I. & Ruthven, I. (eds.), CIKM ’11: Proceedings of the 20th ACM International Conference on Information and Knowledge Management. New York: Association for Computing Machinery, 403–412. https://doi.org/10.1145/2063576.2063639 Google Scholar
Collins-Thompson, K. & Callan, J. (2005) Predicting reading difficulty with statistical language models. Journal of the American Society for Information Science and Technology, 56(13): 1448–1462. https://doi.org/10.1002/asi.20243 CrossRefGoogle Scholar
Ehara, Y., Sato, I., Oiwa, H. & Nakagawa, H. (2012) Mining words in the minds of second language learners: Learner-specific word difficulty. In Kay, M. & Boitet, C. (eds.), Proceedings of the 24th International Conference on Computational Linguistics (COLING 2012). Mumbai, India, 8–15 December.Google Scholar
François, T. & Fairon, C. (2012) An “AI readability” formula for French as a foreign language. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Stroudsburg: The Association for Computational Linguistics, 466–477.Google Scholar
Haynes, M. & Carr, T. H. (1990) Writing system background and second language reading: A component skills analysis of English reading by native speaker-readers of Chinese. In Carr, T. H. & Levy, B. A. (eds.), Reading and its development: Component skills approaches. San Diego: Academic Press, 375–421.Google Scholar
Heilman, M. J., Collins-Thompson, K., Callan, J. & Eskenazi, M. (2007) Combining lexical and grammatical features to improve readability measures for first and second language texts. In Sidner, C., Schultz, T., Stone, M. & Zhai, C. (eds.), Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics: Proceedings of the main conference. Stroudsburg: The Association for Computational Linguistics, 460–467.Google Scholar
Heilman, M., Collins-Thompson, K., Callan, J. & Eskenazi, M. (2010) Personalization of reading passages improves vocabulary acquisition. International Journal of Artificial Intelligence in Education, 20(1): 73–98.Google Scholar
Hill, D. R. (2008) Graded readers in English. ELT Journal, 62(2): 184–204. https://doi.org/10.1093/elt/ccn006 CrossRefGoogle Scholar
Hokamp, C., Mihalcea, R. & Schuelke, P. (2014) Modeling language proficiency using implicit feedback. In Calzolari, N., Choukri, K., Declerck, R., Loftsson, H., Maegaard, B., Mariani, J., Moreno, A., Odijk, J. & Piperidis, S. (eds.), Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14). Reykjavik, Iceland, 26–31 May.Google Scholar
Holley, F. M. (1973) A study of vocabulary learning in context: The effect of new-word density in German reading materials. Foreign Language Annals, 6(3): 339–347. https://doi.org/10.1111/j.1944-9720.1973.tb02613.x CrossRefGoogle Scholar
Hsieh, T.-C., Wang, T.-I., Su, C.-Y. & Lee, M.-C. (2012) A fuzzy logic-based personalized learning system for supporting adaptive English learning. Educational Technology and Society, 15(1): 273–288.Google Scholar
Hsu, C.-K., Hwang, G.-J. & Chang, C.-K. (2013) A personalized recommendation-based mobile learning approach to improving the reading performance of EFL students. Computers & Education, 63: 327–336. https://doi.org/10.1016/j.compedu.2012.12.004 CrossRefGoogle Scholar
Hu, M. H.-C. & Nation, P. (2000) Unknown vocabulary density and reading comprehension. Reading in a Foreign Language, 13(1): 403–430.Google Scholar
Jin, T., Lu, X., Lin, Y. & Li, B. (2018) Chi-Editor: An online Chinese text evaluation and adaptation system. Guangzhou: LanguageData (languagedata.net/editor).Google Scholar
Kay, J. & Kummerfeld, B. (2019) From data to personal user models for life-long, life-wide learners. British Journal of Educational Technology, 50(6), 2871–2884. https://doi.org/10.1111/bjet.12878 CrossRefGoogle Scholar
Knutov, E., De Bra, P. & Pechenizkiy, M. (2009) AH 12 years later: A comprehensive survey of adaptive hypermedia methods and techniques. New Review of Hypermedia and Multimedia, 15(1): 5–38. https://doi.org/10.1080/13614560902801608 CrossRefGoogle Scholar
Krashen, S. D. (1981) The “fundamental pedagogical principle” in second language teaching. Studia Linguistica, 35(1–2): 50–70. https://doi.org/10.1111/j.1467-9582.1981.tb00701.x CrossRefGoogle Scholar
Lee, J. & Yeung, C. Y. (2018) Automatic prediction of vocabulary knowledge for learners of Chinese as a foreign language. In Proceedings of the 2018 2nd International Conference on Natural Language and Speech Processing (ICNLSP). Algiers, Algeria, 25–26 April. https://doi.org/10.1109/ICNLSP.2018.8374392 CrossRefGoogle Scholar
Lennon, C. & Burdick, H. (2014) The Lexile framework as an approach for reading measurement and success. Durham: MetaMetrics.Google Scholar
Liang, S. & Song, J. (2009) Construction of an approach for counting Chinese graded words and characters — A tool for assessing difficulty level of word in Chinese language teaching materials writing system. Modern Educational Technology, 7: 86–89.Google Scholar
Manning, C. D., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S. J. & McClosky, D. (2014) The Stanford CoreNLP natural language processing toolkit. In Bontcheva, K. & Zhu, J. (eds.), Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations. Stroudsburg: Association for Computational Linguistics, 55–60. https://doi.org/10.3115/v1/P14-5010 CrossRefGoogle Scholar
Mecartty, F. H. (2000) Lexical and grammatical knowledge in reading and listening comprehension by foreign language learners of Spanish. Applied Language Learning, 11(2): 323–348.Google Scholar
Miltsakaki, E. (2009) Matching readers’ preferences and reading skills with appropriate web texts. In Kreutel, J. (ed.), EACL ’09: Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics: Demonstrations Session. Stroudsburg: Association for Computational Linguistics, 49–52. https://doi.org/10.3115/1609049.1609062 Google Scholar
Mitrovic, A. & Martin, B. (2007) Evaluating the effect of open student models on self-assessment. International Journal of Artificial Intelligence in Education, 17(2): 121–144.Google Scholar
Oxman, S. & Wong, W. (2014) White paper: Adaptive learning systems. Downers Grove: DV X/DeVry Education Group and Integrated Education Solution.Google Scholar
Pitler, E. & Nenkova, A. (2008) Revisiting readability: A unified framework for predicting text quality. In EMNLP ’08: Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 186–195. https://doi.org/10.3115/1613715.1613742 CrossRefGoogle Scholar
Reynolds, R. (2016) Insights from Russian second language readability classification: Complexity-dependent training requirements, and feature evaluation of multiple categories. In Tetreault, J., Burstein, J., Leacock, C. & Yannakoudakis, H. (eds.), Proceedings of the 11th Workshop on Innovative Use of NLP for Building Educational Applications. Stroudsburg: Association for Computational Linguistics, 289–300. https://doi.org/10.18653/v1/W16-0534 CrossRefGoogle Scholar
Roever, C. & Pan, Y.-C. (2008) Test review: GEPT: General English Proficiency Test. Language Testing, 25(3): 403–408. https://doi.org/10.1177/0265532208090159 CrossRefGoogle Scholar
Schmitt, N., Jiang, X. & Grabe, W. (2011) The percentage of words known in a text and reading comprehension. The Modern Language Journal, 95(1): 26–43. https://doi.org/10.1111/j.1540-4781.2011.01146.x CrossRefGoogle Scholar
Shahrour, G. & Bull, S. (2008) Does “notice” prompt noticing? Raising awareness in language learning with an open learner model. In Nejdl, W., Kay, J., Pu, P. & Herder, E. (eds.), Adaptive hypermedia and adaptive web-based systems: 5th international conference, AH 2008, Hannover, Germany, July 29 – August 1, 2008: Proceedings. Berlin: Springer, 173–182. https://doi.org/10.1007/978-3-540-70987-9_20 Google Scholar
Shiotsu, T. & Weir, C. J. (2007) The relative significance of syntactic knowledge and vocabulary breadth in the prediction of reading comprehension test performance. Language Testing, 24(1): 99–128. https://doi.org/10.1177/0265532207071513 CrossRefGoogle Scholar
Sinclair, J. (1991) Corpus, concordance, collocation. Oxford: Oxford University Press.Google Scholar
Sung, Y.-T., Lin, W.-C., Dyson, S. B., Chang, K.-E. & Chen, Y.-C. (2015) Leveling L2 texts through readability: Combining multilevel linguistic features with the CEFR. The Modern Language Journal, 99(2): 371–391. https://doi.org/10.1111/modl.12213 CrossRefGoogle Scholar
Vajjala, S. & Meurers, D. (2012) On improving the accuracy of readability classification using insights from second language acquisition. In Proceedings of the 7th Workshop on Innovative Use of NLP for Building Educational Applications. Stroudsburg: Association for Computational Linguistics, 163–173.Google Scholar
van Gelderen, A., Schoonen, R., Stoel, R. D., de Glopper, K. & Hulstijn, J. (2007) Development of adolescent reading comprehension in language 1 and language 2: A longitudinal analysis of constituent components. Journal of Educational Psychology, 99(3): 477–491. https://doi.org/10.1037/0022-0663.99.3.477 CrossRefGoogle Scholar
Wu, T.-T. (2016) A learning log analysis of an English-reading e-book system combined with a guidance mechanism. Interactive Learning Environments, 24(8): 1938–1956. https://doi.org/10.1080/10494820.2015.1070272 CrossRefGoogle Scholar
Yimam, S. M., Biemann, C., Malmasi, S., Paetzold, G. H., Specia, L., Štajner, S., Tack, A. & Zampieri, M. (2018) A report on the Complex Word Identification Shared Task 2018. In Tetreault, J., Burstein, J., Kochmar, E., Leacock, C. & Yannakoudakis, H. (eds.), Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications. Stroudsburg: Association for Computational Linguistics, 66–78.CrossRefGoogle Scholar