Hostname: page-component-586b7cd67f-2plfb Total loading time: 0 Render date: 2024-11-30T20:44:04.505Z Has data issue: false hasContentIssue false
  • English
  • Français

SAMPLE SIZE PLANNING IN QUANTITATIVE L2 RESEARCH

A PRAGMATIC APPROACH

Published online by Cambridge University Press:  06 April 2020

Reza Norouzian Open the ORCID record for Reza Norouzian [Opens in a new window]
Reza Norouzian*
Affiliation:
Texas A&M University
*
*Correspondence concerning this article should be addressed to Reza Norouzian, Texas A&M University,Teaching, Learning, Culture, College Station, Texas 77845. E-mail: [email protected].
Get access Rights & Permissions [Opens in a new window]

Abstract

Researchers are traditionally advised to plan for their required sample size such that achieving a sufficient level of statistical power is ensured (Cohen, 1988). While this method helps distinguishing statistically significant effects from the nonsignificant ones, it does not help achieving the higher goal of accurately estimating the actual size of those effects in an intended study. Adopting an open-science approach, this article presents an alternative approach, accuracy in effect size estimation (AESE), to sample size planning that ensures that researchers obtain adequately narrow confidence intervals (CI) for their effect sizes of interest thereby ensuring accuracy in estimating the actual size of those effects. Specifically, I (a) compare the underpinnings of power-analytic and AESE methods, (b) provide a practical definition of narrow CIs, (c) apply the AESE method to various research studies from L2 literature, and (d) offer several flexible R programs to implement the methods discussed in this article.

Type
Research Article
Information
Studies in Second Language Acquisition , Volume 42 , Issue 4 , September 2020 , pp. 849 - 870
Check for updates
Open Practices
Open materials
Copyright
© Cambridge University Press 2020

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The experiment in this article earned an Open Materials badge and an Open Data Badge for transparent practices. The materials and data are available at https://github.com/rnorouzian/i/blob/master/i.r.

References

REFERENCES

Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford University.Google Scholar
Badjadi, N. E. I. (2016). A meta-analysis of the effects of instructional tasks on L2 pragmatics comprehension and production. In Tang, S. F. & Logonnathan, L. (Eds.), Assessment for learning within and beyond the classroom (pp. 241–268). Springer.CrossRefGoogle Scholar
Bai, B. (2018). Understanding primary school students’ use of self-regulated writing strategies through think-aloud protocols. System, 78, 15–26.CrossRefGoogle Scholar
Bandura, A. (1997). Self-efficacy: The exercise of control. W. H. FreemanGoogle Scholar
Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Houghton Mifflin.Google Scholar
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.) Erlbaum.Google Scholar
Cumming, G., & Calin-Jageman, R. (2017). Introduction to the new statistics: Estimation, open science, and beyond. Routledge.Google Scholar
DeKeyser, R. (2015). Skill acquisition theory. In VanPatten, B., & Williams, J. (Eds.), Theories in second language acquisition: An introduction (pp. 94–112). Routledge.Google Scholar
Eslami, Z. R., Mirzaei, A., & Dini, S. (2015). The role of asynchronous computer mediated communication in the instruction and development of EFL learners’ pragmatic competence. System, 48, 99–111.CrossRefGoogle Scholar
Gardner, R. C. (2000). Correlation, causation, motivation, and second language acquisition. Canadian Psychology/Psychologie Canadienne, 41, 10–23.CrossRefGoogle Scholar
Graham, S., Harris, K. R., & Stangelo, T. (2018). Self-regulation and writing. In Schunk, D. H. & Greene, J. A. (Eds.), Handbook of self-regulation of learning and performance. Routledge.Google Scholar
Green, S. B. (1991). How many subjects does it take to do a regression analysis. Multivariate Behavioral Research, 26, 499–510. https://doi.org/10.1207/s15327906mbr2603_7CrossRefGoogle Scholar
Hatch, E. M., & Lazaraton, A. (1991). The research manual: Design and statistics for applied linguistics. Newbury House Publishers.Google Scholar
Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Academic Press.Google Scholar
Jaynes, E. T. (2003). Probability theory: The logic of science. Cambridge University Press.CrossRefGoogle Scholar
Kang, E., & Han, Z. (2015). The efficacy of written corrective feedback in improving L2 written accuracy: A meta‐analysis. The Modern Language Journal, 99, 1–18.CrossRefGoogle Scholar
Kelley, K., & Rausch, J. R. (2006). Sample size planning for the standardized mean difference: Accuracy in parameter estimation via narrow confidence intervals. Psychological Methods, 11, 363–385.CrossRefGoogle ScholarPubMed
Marsden, E., Morgan‐Short, K., Thompson, S., & Abugaber, D. (2018). Replication in second language research: Narrative and systematic reviews and recommendations for the field. Language Learning, 68, 321–391.CrossRefGoogle Scholar
Marsden, E., Morgan‐Short, K., Trofimovich, P., & Ellis, N. C. (2018). Introducing registered reports at language learning: Promoting transparency, replication, and a synthetic ethic in the language sciences. Language Learning, 68, 309–320.CrossRefGoogle Scholar
Masgoret, A. M., & Gardner, R. C. (2003). Attitudes, motivation, and second language learning: A meta-analysis of studies conducted by Gardner and associates. Language Learning, 53, 123–163.CrossRefGoogle Scholar
McElreath, R. (2016). Statistical rethinking: A Bayesian course with examples in R and Stan. CRC Press.Google Scholar
McKay, S. L. (2006). Researching second language classrooms. Routledge.CrossRefGoogle Scholar
Mills, N., Pajares, F., & Herron, C. (2007). Self‐efficacy of college intermediate French students: Relation to achievement and motivation. Language Learning, 57, 417–442.CrossRefGoogle Scholar
Norouzian, R., & De Miranda, M. (2019). Data size planning for multifactor ANOVA designs via adequately narrow confidence intervals for partial eta-squared. Paper presented at the American Educational Research Association, Toronto, Canada.Google Scholar
Norouzian, R., De Miranda, M., & Plonsky, L. (2018). The Bayesian revolution in second language research: An applied approach. Language Learning, 68, 1032–1075.CrossRefGoogle Scholar
Norouzian, R., De Miranda, M., & Plonsky, L. (2019). A Bayesian approach to measuring evidence in L2 research: An empirical investigation. Modern Language Journal, 103, 248–261.CrossRefGoogle Scholar
Norouzian, R., & Eslami, Z. (2016). Critical perspectives on interlanguage pragmatic development: An agenda for research. Issues in Applied Linguistics, 20, 25–50.Google Scholar
Norouzian, R., & Farahani, A. (2012). Written error feedback from perception to practice: A feedback on feedback. Journal of Language Teaching & Research, 3, 11–22.CrossRefGoogle Scholar
Norouzian, R., & Plonsky, L. (2018a). Correlation and simple linear regression in applied linguistics. In Phakiti, A., De Costa, P., Plonsky, L., & Starfield, S. (Eds.), The Palgrave handbook of applied linguistics research methodology (pp. 395–421). Palgrave.CrossRefGoogle Scholar
Norouzian, R., & Plonsky, L. (2018b). Eta- and partial eta-squared in L2 research: A cautionary review and guide to more appropriate usage. Second Language Research, 34, 257–271.CrossRefGoogle Scholar
Norris, J. M., & Ortega, L. (2000). Effectiveness of L2 instruction: A research synthesis and quantitative meta‐analysis. Language Learning, 50, 417–528.CrossRefGoogle Scholar
Plonsky, L. (2014). Study quality in quantitative L2 research (1990–2010): A methodological synthesis and call for reform. Modern Language Journal, 98, 450–470.CrossRefGoogle Scholar
Plonsky, L., & Oswald, F. L. (2014). How big is “big”? Interpreting effect sizes in L2 research. Language Learning, 64, 878–912.CrossRefGoogle Scholar
Plonsky, L., & Ghanbar, H. (2018). Multiple regression in L2 research: A methodological synthesis and guide to interpreting R2 values. Modern Language Journal, 102, 713–731.Google Scholar
Plonsky, L., & Zhuang, J. (2019). A meta-analysis of second language pragmatics instruction. In Taguchi, N. (Ed.), Routledge handbook of SLA and pragmatics (pp. 287–307). Routledge.Google Scholar
Sánchez-Meca, J., & Marín-Martínez, F. (2008). Confidence intervals for the overall effect size in random-effects meta-analysis. Psychological Methods, 13, 31–48.CrossRefGoogle ScholarPubMed
Shintani, N., & Ellis, R. (2013). The comparative effect of direct written corrective feedback and metalinguistic explanation on learners’ explicit and implicit knowledge of the English indefinite article. Journal of Second Language Writing, 22, 286–306.CrossRefGoogle Scholar
Thompson, B. (2006). Foundations of behavioral statistics: An insight-based approach. Guilford Press.Google Scholar
Vatz, K., Tare, M., Jackson, S. R., & Doughty, C. (2013). Aptitude-treatment interaction studies in second language acquisition. In Granena, G. & Long, M. (Eds.), Sensitive periods, language aptitude, and ultimate L2 attainment (pp. 273–292). John Benjamins.CrossRefGoogle Scholar
Yang, Y.-F., & Lin, Y.-Y. (2015). Online collaborative note-taking strategies to foster EFL beginners’ literacy development. System, 52, 127–138.CrossRefGoogle Scholar