1887
  1. Home
  2. e-Journals
  3. Journal of Second Language Studies
  4. Volume 8, Issue 2
  5. Article
Volume 8, Issue 2
  • ISSN 2542-3835
  • E-ISSN: 2542-3843
USD
Buy:$35.00 + Taxes

Abstract

Abstract

Beginning in 2022, the field of applied linguistics has increasingly approached the evaluation of research quality through an ethical lens, with a particular emphasis on Questionable Research Practices (QRPs). Notably, the majority of existing investigations into QRPs have concentrated on mono-method studies, especially those employing quantitative methodologies, thereby neglecting the realm of mixed methods research (MMR). The present study seeks to illuminate the problematic areas that may contribute to QRPs within MMR studies. To this end, we analyzed 60 MMR studies published between 2011 and 2020 in leading journals within the domain of applied linguistics (AL). Our findings reveal a range of issues pertaining to MMR rhetoric and references, study purpose and design, as well as the integration of methodologies, all of which pose risks to the transparency and foundational principles of MMR. This study concludes with recommendations aimed at enhancing the quality of MMR studies.

© John Benjamins Publishing Company
Loading

Article metrics loading...

/content/journals/10.1075/jsls.00051.far
2025-09-12
2026-04-15

  • From This Site

    /content/journals/10.1075/jsls.00051.far
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

References

  1. Al-Hoorie, A. H., & Vitta, J. P.
    (2019) The seven sins of L2 research: A review of 30 journals’ statistical quality and their CiteScore, SJR, SNIP, JCR Impact Factors. Language Teaching Research, 23(6), 727–744. 10.1177/1362168818767191
     https://doi.org/10.1177/1362168818767191 [Google Scholar]
  2. Amini Farsani, M.
    (2025) Enriching quantitative phase in mixed methods research. InThe Encyclopedia of Applied Linguistics (Mixed Methods), C. A. Chapelle & M. Riazi (Eds.). 10.1002/9781405198431.wbeal20002
     https://doi.org/10.1002/9781405198431.wbeal20002 [Google Scholar]
  3. Amini Farsani, M., Jamali, H. R., Beikmohammadi, M., Daneshvar Ghorbani, B., & Soleimani, L.
    (2021) Methodological orientations, academic citations, and scientific collaboration in applied linguistics: What do research synthesis and bibliometrics indicate?System, 1001, 102547. 10.1016/j.system.2021.102547
     https://doi.org/10.1016/j.system.2021.102547 [Google Scholar]
  4. Amini Farsani, M., Babaii, E., Beikmohammadi, M., & Babaii Farsani, M.
    (2022) Mixed-methods research proficiency for applied linguists: A PLS-path modelling approach. Quality & Quantity, 56(5), 3337–3362. 10.1007/s11135‑021‑01268‑7
     https://doi.org/10.1007/s11135-021-01268-7 [Google Scholar]
  5. Alise, M., & Teddlie, C.
    (2010) A continuation of the paradigm wars? Prevalence rates of methodological approaches across the social/behavioral sciences. Journal of Mixed Methods Research, 4(2), 103–126. 10.1177/1558689809360805
     https://doi.org/10.1177/1558689809360805 [Google Scholar]
  6. Anderson, C.
    (2017) Ethics in Qualitative Language Education Research. In: Mirhosseini, S. A. (eds) Reflections on Qualitative Research in Language and Literacy Education. Educational Linguistics, vol291. Springer, Cham. 10.1007/978‑3‑319‑49140‑0_5
     https://doi.org/10.1007/978-3-319-49140-0_5 [Google Scholar]
  7. Brown, J. D.
    (2009) Open-response items in questionnaires. InHeigham, J. and Croker, R. A. (eds.), Qualitative research in applied linguistics: A practical introduction. Basingstoke, UK: Palgrave Macmillan, 200–219. 10.1057/9780230239517_10
     https://doi.org/10.1057/9780230239517_10 [Google Scholar]
  8. Chapelle, C.
    (2012) Using mixed-methods research in technology-based innovation for language learning. Paper presented at theInnovative Practices in Computer Assisted Language Learning Conference, University of Ottawa, Ontario.
     [Google Scholar]
  9. Chen, Q., & Wright, C.
    (2017) Contextualization and authenticity in TBLT: Voices from Chinese classrooms. Language Teaching Research, 21(4), 517–538. 10.1177/1362168816639985
     https://doi.org/10.1177/1362168816639985 [Google Scholar]
  10. Chong, S. W., & Plonsky, L.
    (2024) Research synthesis in language education. InInternational Encyclopedia of Language and Linguistics (3rd ed.). Elsevier. 10.1016/B978‑0‑323‑95504‑1.00010‑7
     https://doi.org/10.1016/B978-0-323-95504-1.00010-7 [Google Scholar]
  11. Creamer, E. G.
    (2018) An introduction to fully integrated mixed methods research. Sage. 10.4135/9781071802823
     https://doi.org/10.4135/9781071802823 [Google Scholar]
  12. Creswell, J. W., & Plano Clark, V. L.
    (2011) Designing and conducting mixed methods research (2nd ed.). Sage.
     [Google Scholar]
  13. (2018) Designing and conducting mixed methods research (3rd ed.). Sage.
     [Google Scholar]
  14. De Costa, P. I.
    (Ed.) (2015) Ethics in applied linguistics research: Language researcher narratives. Routledge. 10.4324/9781315816937
     https://doi.org/10.4324/9781315816937 [Google Scholar]
  15. Dolgova Jacobsen, N.
    (2018) The best of both worlds: Combining cognitive linguistics and pedagogic tasks to teach English conditionals. Applied Linguistics, 39(5), 668–693. 10.1093/applin/amw030
     https://doi.org/10.1093/applin/amw030 [Google Scholar]
  16. Dörnyei, Z.
    (2007) Research methods in applied linguistics: Quantitative, qualitative, and mixed methodologies. Oxford.
     [Google Scholar]
  17. Dufon, M.
    (1993) Ethics in TESOL research. TESOL Quarterly, 27(1), 157–160. 10.2307/3586970
     https://doi.org/10.2307/3586970 [Google Scholar]
  18. F`abregues, S., & Molina-Azorin, J. F.
    (2017) Addressing quality in mixed methods research: A review and recommendations for a future agenda. Quality and Quantity, 51(6), 2847–2863. 10.1007/s11135‑016‑0449‑4
     https://doi.org/10.1007/s11135-016-0449-4 [Google Scholar]
  19. Fàbregues, S., Molina-Azorin, J. F., & Fetters, M. D.
    (2021) Virtual special issue on “quality in mixed methods research”. Journal of Mixed Methods Research, 15(2), 146–151. 10.1177/15586898211001974
     https://doi.org/10.1177/15586898211001974 [Google Scholar]
  20. Fanelli, D.
    (2009) How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS One, 4(5), e5738. 10.1371/journal.pone.0005738
     https://doi.org/10.1371/journal.pone.0005738 [Google Scholar]
  21. Fetters, M. D., & Molina-Azorin, J. F.
    (2017) The journal of mixed methods research starts a new decade: The mixed methods research integration trilogy and its dimensions. Journal of Mixed Methods Research, 11(3), 291–307. 10.1177/1558689817714066
     https://doi.org/10.1177/1558689817714066 [Google Scholar]
  22. Fetters, M. D., Curry, L. A., & Creswell, J. W.
    (2013) Achieving integration in mixed methods designs-principles and practices. Health Services Research, 48(6), 2134–2156. 10.1111/1475‑6773.12117
     https://doi.org/10.1111/1475-6773.12117 [Google Scholar]
  23. Gass, S., Loewen, S., & Plonsky, L.
    (2021) Coming of age: The past, present, and future of quantitative SLA research. Language Teaching, 54(2), 245–258. 10.1017/S0261444819000430
     https://doi.org/10.1017/S0261444819000430 [Google Scholar]
  24. Ghanbar, H. & Rezvani, R.
    (forthcoming). Quality issues in applied Linguistics qualitative research. Journal of Modern Research in English Language Studies.
     [Google Scholar]
  25. Gonulal, T., Loewen, S., & Plonsky, L.
    (2017) The development of statistical literacy in applied linguistics graduate students. ITL-International Journal of Applied Linguistics, 168(1), 4–32. 10.1075/itl.168.1.01gon
     https://doi.org/10.1075/itl.168.1.01gon [Google Scholar]
  26. Greene, J. C., Caracelli, V. J., & Graham, W. F.
    (1989) Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255–274. 10.3102/01623737011003255
     https://doi.org/10.3102/01623737011003255 [Google Scholar]
  27. Guetterman, T. C.
    (2015) The development, design, and test of a self-assessmnet instrument of mixed methods research proficiency [Unpublished Ph.D. Dissertation]. University of Nebraska.
  28. (2017) What distinguishes a novice from an expert mixed methods researcher?Quality and Quantity, 51(1): 377–398. 10.1007/s11135‑016‑0310‑9
     https://doi.org/10.1007/s11135-016-0310-9 [Google Scholar]
  29. Guetterman, T. C., Plano Clark, V. L., & Molina-Azorin, J. F.
    (2024) Terminology and Mixed Methods Research: A Persistent Challenge. Journal of Mixed Methods Research, 18(1), 9–13. 10.1177/15586898231217855
     https://doi.org/10.1177/15586898231217855 [Google Scholar]
  30. Hall, J., & Martin, B. R.
    (2019) Towards a taxonomy of research misconduct: The case of business school research. Research Policy, 48(2), 414–427. 10.1016/j.respol.2018.03.006
     https://doi.org/10.1016/j.respol.2018.03.006 [Google Scholar]
  31. Harrison, R. L., Reilly, T. M., & Creswell, J. W.
    (2020) Methodological rigor in mixed methods: An application in management studies. Journal of Mixed Methods Research, 14(4), 473–495. 10.1177/1558689819900585
     https://doi.org/10.1177/1558689819900585 [Google Scholar]
  32. Heyvaert, M., Maes, B., & Onghena, P.
    (2013) Mixed methods research synthesis: Definition framework, and potential. Quality and Quantity, 471, 659–676. 10.1007/s11135‑011‑9538‑6
     https://doi.org/10.1007/s11135-011-9538-6 [Google Scholar]
  33. Hirose, M., & Creswell, J. W.
    (2022) Applying core quality criteria of mixed methods research to an empirical study. Journal of Mixed Methods Research, 17(1), 12–28. 10.1177/15586898221086346
     https://doi.org/10.1177/15586898221086346 [Google Scholar]
  34. Hudson, T., & Llosa, L.
    (2015) Design issues and inference in experimental L2 research. Language Learning, 65(S1), 76–96. 10.1111/lang.12113
     https://doi.org/10.1111/lang.12113 [Google Scholar]
  35. Isbell, D., Brown, D., Chan, M., Derrick, D., Ghanem, R., Gutiérrez Arvizu, M. N., Schnur, E., Zhang, M., & Plonsky, L.
    (2022) Misconduct and questionable research practices: The ethics of quantitative data handling and reporting in applied linguistics. Modern Language Journal, 106(1), 172–195. 10.1111/modl.12760
     https://doi.org/10.1111/modl.12760 [Google Scholar]
  36. Johnson, R. B.
    (2017) Dialectical pluralism: A metaparadigm whose time has come. Journal of Mixed Methods Research, 111, 156–173. 10.1177/1558689815607692
     https://doi.org/10.1177/1558689815607692 [Google Scholar]
  37. Johnson, R. B., Onwuegbuzie, A. J., & Turner, L. A.
    (2007) Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1(2), 112–133. 10.1177/1558689806298224
     https://doi.org/10.1177/1558689806298224 [Google Scholar]
  38. Kao, T. N.
    (2012) Factor analysis of English writing demotivation among central Taiwan university students (Unpublished master thesis). Taichung, Taiwan: Providence University.
     [Google Scholar]
  39. King, K. A., & Mackey, A.
    (2016) Research methodology in second language studies: Trends, concerns, and new directions. The Modern Language Journal, 1001, 209–227. 10.1111/modl.12309
     https://doi.org/10.1111/modl.12309 [Google Scholar]
  40. Kubanyiova, M.
    (2008) Rethinking research ethics in contemporary applied linguistics: The tension between macroethical and microethical perspectives in situated research. The Modern Language Journal, 92(4), 503–518. 10.1111/j.1540‑4781.2008.00784.x
     https://doi.org/10.1111/j.1540-4781.2008.00784.x [Google Scholar]
  41. Larsson, T., Plonsky, L., Sterling, S., Kytö, M., Yaw, K., & Wood, M.
    (2023) On the frequency, prevalence, and perceived severity of questionable research practices. Research Methods in Applied Linguistics, 2(3), 100064. 10.1016/j.rmal.2023.100064
     https://doi.org/10.1016/j.rmal.2023.100064 [Google Scholar]
  42. Lee, J. S., Nakamura, Y., & Sadler, R.
    (2018) Effects of videoconference-embedded classrooms (VEC) on learners’ perceptions toward English as an international language (EIL). ReCALL, 30(3), 319–336. 10.1017/S095834401700026X
     https://doi.org/10.1017/S095834401700026X [Google Scholar]
  43. Lei, L., & Liu, D.
    (2019) Research trends in applied linguistics from 2005 to 2016: A bibliometric analysis and its implications. Applied Linguistics, 401: 540–561. 10.1093/applin/amy003
     https://doi.org/10.1093/applin/amy003 [Google Scholar]
  44. Marsden, E., Mackey, A., & Plonsky, L.
    (2016) Breadth and depth: The IRIS repository. InA. Mackey & E. Marsden (Eds.), Advancing methodology and practice: The IRIS repository of instruments for research into second languages (pp.1–21). Routledge.
     [Google Scholar]
  45. Meihami, H., & Esfandiari, R.
    (Eds.) (2024) A Scientometrics Research Perspective in Applied Linguistics. Springer. 10.1007/978‑3‑031‑51726‑6
     https://doi.org/10.1007/978-3-031-51726-6 [Google Scholar]
  46. Mertens, D. M.
    (2007) Transformative paradigm: Mixed methods and social justice. Journal of Mixed Methods Research, 11, 212–225. 10.1177/1558689807302811
     https://doi.org/10.1177/1558689807302811 [Google Scholar]
  47. Moeller, A. J., & Theiler, J.
    (2014) Spoken Spanish language development at the high school level: A mixed-methods study. Foreign Language Annals, 47(2), 210–240. 10.1111/flan.12085
     https://doi.org/10.1111/flan.12085 [Google Scholar]
  48. Ortega, L.
    (2005a) Methodology, epistemology, and ethics in instructed SLA research: An introduction. Modern Language Journal, 89(3), 317–327. 10.1111/j.1540‑4781.2005.00307.x
     https://doi.org/10.1111/j.1540-4781.2005.00307.x [Google Scholar]
  49. (2005b) For what and for whom is our research? The ethical as transformative lens in instructed SLA. Modern Language Journal, 89(3), 427–443. 10.1111/j.1540‑4781.2005.00315.x
     https://doi.org/10.1111/j.1540-4781.2005.00315.x [Google Scholar]
  50. Peace, M. M.
    (2015) Other-Orientation in Nonnative Spanish and Its Effect on Direct Objects. Foreign Language Annals, 48(4), 669–687. 10.1111/flan.12166
     https://doi.org/10.1111/flan.12166 [Google Scholar]
  51. Plonsky, L.
    (2013) Study quality in SLA: An assessment of designs, analyses, and reporting practices in quantitative L2 research. Studies in Second Language Acquisition, 35(4), 655–687. 10.1017/S0272263113000399
     https://doi.org/10.1017/S0272263113000399 [Google Scholar]
  52. (2024) Study quality as an intellectual and ethical imperative: A proposed framework. Annual Review of Applied Linguistics, 1–15. 10.1017/S0267190524000059
     https://doi.org/10.1017/S0267190524000059 [Google Scholar]
  53. Plonsky, L., & Oswald, F. L.
    (2015) Meta-analyzing second language research. InL. Plonsky (Ed.), Advancing quantitative methods in second language research (pp.106–128). Routledge. 10.4324/9781315870908‑6
     https://doi.org/10.4324/9781315870908-6 [Google Scholar]
  54. Purpura, J. E., Brown, J. D., & Schoonen, R.
    (2015) Improving the validity of quantitative measures in applied linguistics research. Language Learning, 65(S1), 37–75. 10.1111/lang.12112
     https://doi.org/10.1111/lang.12112 [Google Scholar]
  55. Riazi, A. M.
    (2016) Innovative mixed-methods research: Moving beyond design technicalities to epistemological and methodological realizations. Applied Linguistics, 37(1), 33–49. 10.1093/applin/amv064
     https://doi.org/10.1093/applin/amv064 [Google Scholar]
  56. (2017) Mixed-methods research in language teaching and learning. Equinox.
     [Google Scholar]
  57. (2024) Mixed methods research (MMR): An overview. InThe Encyclopedia of Applied Linguistics (Mixed Methods), C. A. Chapelle & M. Riazi (Eds.). 10.1002/9781405198431.wbeal20001
     https://doi.org/10.1002/9781405198431.wbeal20001 [Google Scholar]
  58. Riazi, A. M., & Candlin, C. N.
    (2014) Mixed-methods research in language teaching and learning: Opportunities, issues, and challenges. Language Teaching, 47(2), 135–173. 10.1017/S0261444813000505
     https://doi.org/10.1017/S0261444813000505 [Google Scholar]
  59. Riazi, A. M., & Amini Farsani, M.
    (2024) Mixed-methods research in applied linguistics: Charting the progress through the second decade of the twenty-first century. Language Teaching, 57(2), 143–182. 10.1017/S0261444823000332
     https://doi.org/10.1017/S0261444823000332 [Google Scholar]
  60. Riazi, A. M., Farr, F., Han, Ch., & Amini Farsani, M.
    (2024) Fostering methodological awareness: The Mixed methods research affordances. Paper presented at the21st AILA World Congress, Kuala Lumpur, Malysia
     [Google Scholar]
  61. Robson, C.
    (2002) Real world research: A resource for social scientists and practitioner-researchers. John Wiley & Sons.
     [Google Scholar]
  62. Roy, D., Brine, J., & Murasawa, F.
    (2016) Usability of English note-taking applications in a foreign language learning context. Computer Assisted Language Learning, 29(1), 61–87. 10.1080/09588221.2014.889715
     https://doi.org/10.1080/09588221.2014.889715 [Google Scholar]
  63. Schoonenboom, J.
    (2022) Developing the metainference in mixed methods research through successive integration of claims. InHitchcock, J. H. and Onwuegbuzie, A. J. (eds.) The Routledge handbook for advancing integration in mixed methods research (pp.55–70). Routledge. 10.4324/9780429432828‑6
     https://doi.org/10.4324/9780429432828-6 [Google Scholar]
  64. Steneck, N. H.
    (2007) Introduction to the responsible conduct of research. Department of Health and Human Services, Office of Research Integrity. https://ori.hhs.gov/sites/default/files/rcrintro.pdf. 10.1037/e638422011‑001
     https://doi.org/10.1037/e638422011-001 [Google Scholar]
  65. Sterling, S., & Gass, S.
    (2017) Exploring the boundaries of research ethics: Perceptions of ethics and ethical behaviors in applied linguistics research. System, 701, 50–62. 10.1016/j.system.2017.08.010
     https://doi.org/10.1016/j.system.2017.08.010 [Google Scholar]
  66. Sterling, S., Winke, P., & Gass, S.
    (2016) Training in research ethics among applied linguistics and SLA researchers. InP. I. De Costa (Ed.), Ethics in applied linguistics research: Language researcher narratives (pp.15–37). Routledge.
     [Google Scholar]
  67. Tashakkori, A., & Teddlie, C.
    (1998) Mixed methodology: Combining qualitative and quantitative approaches. Sage.
     [Google Scholar]
  68. (2009) Integrating qualitative and quantitative approaches to research. InA. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp.283–318). Sage. 10.4135/9781483348858.n9
     https://doi.org/10.4135/9781483348858.n9 [Google Scholar]
  69. Teddlie, C. B., and Tashakkori, A.
    (2009) Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioural sciences. Sage.
     [Google Scholar]
  70. Thomas, M.
    (2009) Ethical issues in the study of second language acquisition: Resources for teachers. Second Language Research, 25(4), 493–511. 10.1177/0267658309349676
     https://doi.org/10.1177/0267658309349676 [Google Scholar]
  71. Van Kemenade, E., Pupius, M., & Hardjono, T. W.
    (2008) More value to defining quality. Quality in Higher Education, 14(2), 175–185. 10.1080/13538320802278461
     https://doi.org/10.1080/13538320802278461 [Google Scholar]
  72. Vitta, J. P., Hahn, A., Canning, D., Isbell, D. R., Al-Hoorie, A. H., & Nicklin, C.
    (2025) Conceptualization and frequency of sampling (mal)practices in L2 inferential quantitative research. Journal of Second Language Studies. https://www.jbe-platform.com/content/journals/10.1075/jsls.00049.vit
     [Google Scholar]
  73. Vogel, S., Herron, C., Cole, S. P., & York, H.
    (2011) Effectiveness of a guided inductive versus a deductive approach on the learning of grammar in the intermediate-level college French classroom. Foreign language annals, 44(2), 353–380. 10.1111/j.1944‑9720.2011.01133.x
     https://doi.org/10.1111/j.1944-9720.2011.01133.x [Google Scholar]
  74. Wette, R.
    (2017) Source text use by undergraduate post-novice L2 writers in disciplinary assignments: Progress and ongoing challenges. Journal of Second Language Writing, 371, 46–58. 10.1016/j.jslw.2017.05.015
     https://doi.org/10.1016/j.jslw.2017.05.015 [Google Scholar]
  75. Wu, W. C. V., Yang, J. C., Scott Chen Hsieh, J., & Yamamoto, T.
    (2020) Free from demotivation in EFL writing: The use of online flipped writing instruction. Computer Assisted Language Learning, 33(4), 353–387. 10.1080/09588221.2019.1567556
     https://doi.org/10.1080/09588221.2019.1567556 [Google Scholar]
  76. Yang, J.
    (2016) Learners’ oral corrective feedback preferences in relation to their cultural background, proficiency level and types of error. System, 611, 75–86. 10.1016/j.system.2016.08.004
     https://doi.org/10.1016/j.system.2016.08.004 [Google Scholar]
  77. Yaw, K., Plonsky, L., Larsson, T., Sterling, S., & Kyto, M.
    (2023) Timeline: Research ethics in applied linguistics. Language Teaching, 56(4), 478–494. 10.1017/S0261444823000010
     https://doi.org/10.1017/S0261444823000010 [Google Scholar]
  78. Zhang, X.
    (2020) A bibliometric analysis of second language acquisition between 1997 and 2018. Studies in Second Language Acquisition, 42(1), 199–222. 10.1017/S0272263119000573
     https://doi.org/10.1017/S0272263119000573 [Google Scholar]
  79. Zhonggen, Y., Ying, Z., Zhichun, Y., & Wentao, C.
    (2019) Student satisfaction, learning outcomes, and cognitive loads with a mobile learning platform. Computer assisted language learning, 32(4), 323–341. 10.1080/09588221.2018.1517093
     https://doi.org/10.1080/09588221.2018.1517093 [Google Scholar]
/content/journals/10.1075/jsls.00051.far
Loading
Exploring questionable research practices in applied linguistics mixed methods research studies
Journal of Second Language Studies 8, 344 (2025); https://doi.org/10.1075/jsls.00051.far
/content/journals/10.1075/jsls.00051.far
/content/journals/10.1075/jsls.00051.far
Loading

Data & Media loading...

  • Article Type: Research Article
Keyword(s): applied linguistics; mixed methods research; questionable research practices; research ethics; research quality

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error