The role of language in word-problem solving: a meta-analysis
DOI:
https://doi.org/10.31949/ijeir.v1i2.2617Abstract
The relationship between language proficiency and word-problem solving has been extensively studied in the last three decades with one main finding: language proficiency is directly proportional to word-problem solving scores. Researchers have focused on language accommodations to standardized tests to level the playing field for nonnative speakers. Interestingly, several meta-analyses on language accommodation have noted that there are no significant effects on scores. At the same time, most research has reported significance and not effect sizes, which does not allow to establish comparisons between studies. Therefore, the purpose of this meta-analyses was to study the effect size of studies conducted in the US and the rest of the world to provide a new population effect size and identify possible moderator variables that have not been considered before. The main findings were that the differences in scores between native speakers and nonnative speakers (g=.360) were less pronounced than previously reported (g=.604). The participants’ age and the language of instruction and testing moderated a small percentage of the effect size, which leads to the need to include specific information about the participants to provide a clearer picture of this relationship and eventually provide language accommodations that have significant effects on scores.
Keywords:
Word-Problem Solving , Language Proficiency, Bilingualism, Word ProblemsDownloads
Download data is not yet available.
References
Abedi, J. & Lord, C. (2001). The Language Factor in Mathematics Tests. Applied Measurement in Education, 14(3), 219-234. http://dx.doi.org/10.1207/S15324818AME1403_
Abedi, J., Hofstetter, C. H., & Lord, C. (2004). Assessment accommodations for English language learners: Implications for policy-based empirical research. Review of Educational Research, 74(1), 1-28.
Acosta, B. D., Rivera, C., & Willner, L. S. (2008). Best practices in state assessment policies for accommodating English language learners: A delphi study. Arlington: Center for Equity and Excellence in Education, the George Washington University
Baker, D., Basaraba, D. & Polanco, P. (2016). Connecting the Present to the Past: Furthering the Research on Bilingual Education and Bilingualism. Review of Research in Education, 40(1), 821-883. https://doi.org/10.3102/0091732X16660691
Banks, K., Jeddeeni, A. & Walker, C. (2016). Assessing the Effect of Language Demand in Bundles of Math Word Problems. International Journal of Testing, 16(4) 269-287. https://doi.org/10.1080/15305058.2015.1113972
Barendregt, J. & Doi, S. (2016). MetaXL User Guide (Version 5.3). EpiGear International Pty Ltd.
Bautista, D., Mitchelmore, M. & Mulligan, J. (2009) Factors influencing Filipino children’s solutions to addition and subtraction word problems. Educational Psychology, 29(6), 729-745.
Bernardo, A. (2002). Language and Mathematical Problem Solving Among Bilinguals, The Journal of Psychology, 136(3), 283-297. http://dx.doi.org/10.1080/00223980209604156
Bernardo, A. (2005) Language and Modeling Word Problems in Mathematics Among Bilinguals, The Journal of Psychology: Interdisciplinary and Applied, 139(5), 413-425. http://dx.doi.org/10.3200/JRLP.139.5.413-425
Campbell, A., Adams, V., & Davis, G. (2007). Cognitive demands and second language learners: A framework for analyzing mathematics instructional contexts. Mathematical Thinking and Learning, 9(1), 3-30.
Coe, R. (2002). It’s the Effect Size, Stupid. What Effect Size Is and Why It Is Important [Paper]. The British Educational Research Association Annual Conference, Exeter, U.K. http://www.leeds.ac.uk/educol/documents/00002182.htm
Driver, M. & Powell, S. (2017). Culturally and Linguistically Responsive Schema Intervention: Improving Word Problem Solving for English Language Learners With Mathematics Difficulty. Learning Disability Quarterly, 40(1), 41-53.
Field, A. & Gillet, R. (2010). How to do a meta-analysis. British Journal of Mathematical and Statistical Psychology, 63(3), 665-694. https://doi.org/10.1348/000711010X502733
Guthormsen, A., Fisher, K., Bassok, M., Osterhout, L., DeWolf, M., & Holyoak, K. (2016). Conceptual Integration of Arithmetic Operations With Real-World Knowledge: Evidence From Event-Related Potentials. Cognitive Science, 40(3), 723-757. http://dx.doi.org/10.1111/cogs.12238
Higgins, J. P., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta‐analysis. Statistics in Medicine, 21(11), 1539-1558
Kempert, S., Saalbach, H. & Hardy, I. (2011). Cognitive Benefits and Costs of Bilingualism in Elementary School Students: The Case of Mathematical Word Problems. Journal of Educational Psychology, 103(3), 547-561.
Kieffer, M., Lesaux, N., Rivera, M. & Francis, D. (2009). Accommodations for English Language Learners Taking Large-Scale Assessments: A Meta-Analysis on Effectiveness and Validity. Review of Educational Research, 79(3), 1168-1201. https://doi.org/10.3102/0034654309332490
Kieffer, M., Rivera, M., Francis, D.J. (2012). Research-based recommendations for the use of accommodations in large-scale assessments: 2012 update. Practical Guidelines for the Education of English Language Learners. Book 4. New York: Center on Instruction.
Lager, C. (2006). Types of Mathematics-Language Reading Interactions that Unnecessarily Hinder Algebra Learning and Assessment. Reading Psychology, 27(2-3), 165-204.
Martin, B. & Fuchs, L. (2019). The Mathematical Performance of At-Risk First Graders as a Function of Limited English Proficiency Status. Learning Disability Quarterly, 42(4) 244-251. https://doi.org/10.1177/0731948719827489
Ockey, G. (2007). Investigating the Validity of Math Word Problems for English Language Learners with DIF. Language Assessment Quarterly, 4(2), 149-164
Pavón, V. & Cabezuelo, R. (2019). Analysing mathematical word problem solving with secondary education CLIL students: A pilot study. Latin American Journal Of Content & Language Integrated Learning, 12(1), 18-45. https://doi.org/10.5294/laclil.2019.12.1.2
Pennock‐Roman, M., & Rivera, C. (2011). Mean effects of test accommodations for ELLs and non‐ELLs: A meta‐analysis of experimental studies. Educational Measurement: Issues and Practice, 30(3), 10-28.
Powell, S., Berry, K. & Tran, L. (2020). Performance Differences on a Measure of Mathematics Vocabulary for English Learners and Non-English Learners with and without Mathematics Difficulty. Reading and Writing Quarterly, 36(2) 124-141. https://doi.org/10.1080/10573569.2019.1677538
Rios, J., Ihlenfeldt, S. & Chavez, C. (2020). Are Accommodations for English Learners on State Accountability Assessments Evidence‐Based? A Multistudy Systematic Review and Meta‐Analysis. Educational Measurement Issues and Practices. https://doi.org/10.1111/emip.12337
Sireci, S., Li, S., & Scarpati, S. (2003). The effect of test accommodation on test performance: A review of the literature (Research Report 495). Amherst: University of Massachusetts School of Education, Center for Educational Assessment.
StataCorp. 2013. Stata Statistical Software: Release 13. College Station, TX: StataCorp LP.
Swanson, H., Kong, J., Moran, A. & Orosco, M. (2019). Paraphrasing Interventions and Problem‐Solving Accuracy: Do Generative Procedures Help English Language Learners with Math Difficulties? Learning Disabilities Research and Practice, 34(2), 68-84. https://doi.org/10.1111/ldrp.12194
Trakulphadetkrai, N., Courtney, C., Clenton, J., Treffers-Daller, J., & Tsakalaki, A. (2020). The contribution of general language ability, reading comprehension and working memory to mathematics achievement among children with English as additional language (EAL): an exploratory study. International Journal of Bilingual Education and Bilingualism, 23(4), 473-487. https://doi.org/10.1080/13670050.2017.1373742
Abedi, J., Hofstetter, C. H., & Lord, C. (2004). Assessment accommodations for English language learners: Implications for policy-based empirical research. Review of Educational Research, 74(1), 1-28.
Acosta, B. D., Rivera, C., & Willner, L. S. (2008). Best practices in state assessment policies for accommodating English language learners: A delphi study. Arlington: Center for Equity and Excellence in Education, the George Washington University
Baker, D., Basaraba, D. & Polanco, P. (2016). Connecting the Present to the Past: Furthering the Research on Bilingual Education and Bilingualism. Review of Research in Education, 40(1), 821-883. https://doi.org/10.3102/0091732X16660691
Banks, K., Jeddeeni, A. & Walker, C. (2016). Assessing the Effect of Language Demand in Bundles of Math Word Problems. International Journal of Testing, 16(4) 269-287. https://doi.org/10.1080/15305058.2015.1113972
Barendregt, J. & Doi, S. (2016). MetaXL User Guide (Version 5.3). EpiGear International Pty Ltd.
Bautista, D., Mitchelmore, M. & Mulligan, J. (2009) Factors influencing Filipino children’s solutions to addition and subtraction word problems. Educational Psychology, 29(6), 729-745.
Bernardo, A. (2002). Language and Mathematical Problem Solving Among Bilinguals, The Journal of Psychology, 136(3), 283-297. http://dx.doi.org/10.1080/00223980209604156
Bernardo, A. (2005) Language and Modeling Word Problems in Mathematics Among Bilinguals, The Journal of Psychology: Interdisciplinary and Applied, 139(5), 413-425. http://dx.doi.org/10.3200/JRLP.139.5.413-425
Campbell, A., Adams, V., & Davis, G. (2007). Cognitive demands and second language learners: A framework for analyzing mathematics instructional contexts. Mathematical Thinking and Learning, 9(1), 3-30.
Coe, R. (2002). It’s the Effect Size, Stupid. What Effect Size Is and Why It Is Important [Paper]. The British Educational Research Association Annual Conference, Exeter, U.K. http://www.leeds.ac.uk/educol/documents/00002182.htm
Driver, M. & Powell, S. (2017). Culturally and Linguistically Responsive Schema Intervention: Improving Word Problem Solving for English Language Learners With Mathematics Difficulty. Learning Disability Quarterly, 40(1), 41-53.
Field, A. & Gillet, R. (2010). How to do a meta-analysis. British Journal of Mathematical and Statistical Psychology, 63(3), 665-694. https://doi.org/10.1348/000711010X502733
Guthormsen, A., Fisher, K., Bassok, M., Osterhout, L., DeWolf, M., & Holyoak, K. (2016). Conceptual Integration of Arithmetic Operations With Real-World Knowledge: Evidence From Event-Related Potentials. Cognitive Science, 40(3), 723-757. http://dx.doi.org/10.1111/cogs.12238
Higgins, J. P., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta‐analysis. Statistics in Medicine, 21(11), 1539-1558
Kempert, S., Saalbach, H. & Hardy, I. (2011). Cognitive Benefits and Costs of Bilingualism in Elementary School Students: The Case of Mathematical Word Problems. Journal of Educational Psychology, 103(3), 547-561.
Kieffer, M., Lesaux, N., Rivera, M. & Francis, D. (2009). Accommodations for English Language Learners Taking Large-Scale Assessments: A Meta-Analysis on Effectiveness and Validity. Review of Educational Research, 79(3), 1168-1201. https://doi.org/10.3102/0034654309332490
Kieffer, M., Rivera, M., Francis, D.J. (2012). Research-based recommendations for the use of accommodations in large-scale assessments: 2012 update. Practical Guidelines for the Education of English Language Learners. Book 4. New York: Center on Instruction.
Lager, C. (2006). Types of Mathematics-Language Reading Interactions that Unnecessarily Hinder Algebra Learning and Assessment. Reading Psychology, 27(2-3), 165-204.
Martin, B. & Fuchs, L. (2019). The Mathematical Performance of At-Risk First Graders as a Function of Limited English Proficiency Status. Learning Disability Quarterly, 42(4) 244-251. https://doi.org/10.1177/0731948719827489
Ockey, G. (2007). Investigating the Validity of Math Word Problems for English Language Learners with DIF. Language Assessment Quarterly, 4(2), 149-164
Pavón, V. & Cabezuelo, R. (2019). Analysing mathematical word problem solving with secondary education CLIL students: A pilot study. Latin American Journal Of Content & Language Integrated Learning, 12(1), 18-45. https://doi.org/10.5294/laclil.2019.12.1.2
Pennock‐Roman, M., & Rivera, C. (2011). Mean effects of test accommodations for ELLs and non‐ELLs: A meta‐analysis of experimental studies. Educational Measurement: Issues and Practice, 30(3), 10-28.
Powell, S., Berry, K. & Tran, L. (2020). Performance Differences on a Measure of Mathematics Vocabulary for English Learners and Non-English Learners with and without Mathematics Difficulty. Reading and Writing Quarterly, 36(2) 124-141. https://doi.org/10.1080/10573569.2019.1677538
Rios, J., Ihlenfeldt, S. & Chavez, C. (2020). Are Accommodations for English Learners on State Accountability Assessments Evidence‐Based? A Multistudy Systematic Review and Meta‐Analysis. Educational Measurement Issues and Practices. https://doi.org/10.1111/emip.12337
Sireci, S., Li, S., & Scarpati, S. (2003). The effect of test accommodation on test performance: A review of the literature (Research Report 495). Amherst: University of Massachusetts School of Education, Center for Educational Assessment.
StataCorp. 2013. Stata Statistical Software: Release 13. College Station, TX: StataCorp LP.
Swanson, H., Kong, J., Moran, A. & Orosco, M. (2019). Paraphrasing Interventions and Problem‐Solving Accuracy: Do Generative Procedures Help English Language Learners with Math Difficulties? Learning Disabilities Research and Practice, 34(2), 68-84. https://doi.org/10.1111/ldrp.12194
Trakulphadetkrai, N., Courtney, C., Clenton, J., Treffers-Daller, J., & Tsakalaki, A. (2020). The contribution of general language ability, reading comprehension and working memory to mathematics achievement among children with English as additional language (EAL): an exploratory study. International Journal of Bilingual Education and Bilingualism, 23(4), 473-487. https://doi.org/10.1080/13670050.2017.1373742
Published
2022-07-13
How to Cite
Flores, M., & Urrutia, M. (2022). The role of language in word-problem solving: a meta-analysis. International Journal of Educational Innovation and Research, 1(2), 191–199. https://doi.org/10.31949/ijeir.v1i2.2617
Issue
Section
Articles
License
Copyright (c) 2022 Martín Flores, Mabel Urrutia
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
