1887
  1. Home
  2. e-Journals
  3. Journal of Second Language Studies
  4. Online First Articles
  5. Online First Article
image of The Ulysses pact
USD
Buy:$35.00 + Taxes

Abstract

Abstract

Registered reports (RRs) are gaining traction in applied linguistics as a means to enhance research transparency and credibility by disincentivizing questionable research practices that are aimed at generating statistically significant findings, and by mitigating publication bias. While the benefits of RRs are well-theorized in the literature, less is known about authors’ experiences of conducting RRs. This study explored the first-hand experiences of 12 authors of RRs in applied linguistics from an emic perspective currently underrepresented in the field. Through semi-structured interviews, we examined authors’ motivations for engaging in RRs, perceived benefits and challenges, and reflections on RRs. Our findings revealed that authors valued RRs for promoting scientific rigor and offering publication guarantee and found the process to be highly beneficial. However, tensions were also found in the process, ranging from the potentially time-consuming nature of RRs to reduced autonomy and role ambiguity experienced by some authors. Based on these insights, we offer recommendations for improving the RR process and call for greater support for all stakeholders in the process, including reviewers and editors. We also compiled a list of recommendations by our participants to aid future authors in choosing and navigating RRs.

© John Benjamins Publishing Company
Loading

Article metrics loading...

/content/journals/10.1075/jsls.00045.liu
2025-04-14
2025-04-17

  • From This Site

    /content/journals/10.1075/jsls.00045.liu
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

References

  1. Al-Hoorie, A. H., Cinaglia, C., Hiver, P., Huensch, A., Isbell, D. R., Leung, C., & Sudina, E.
    (2024) Open science: Considerations and issues for TESOL research. TESOL Quarterly. 10.1002/tesq.3304
     https://doi.org/10.1002/tesq.3304 [Google Scholar]
  2. Allen, C., & Mehler, D. M. A.
    (2019) Open science challenges, benefits and tips in early career and beyond. PLOS Biology, (), . 10.1371/journal.pbio.3000246
     https://doi.org/10.1371/journal.pbio.3000246 [Google Scholar]
  3. Benson, P.
    (2019) Narrative analysis. InA. Phakiti, P. De Costa, L. Plonsky, & S. Starfield (Eds.), The Palgrave handbook of applied linguistics research methodology (pp.–). Springer.
     [Google Scholar]
  4. Bolibaugh, C., Vanek, N., & Marsden, E.
    (2021) Towards a credibility revolution in bilingualism research: Open data and materials as stepping stones to more reproducible and replicable research. Bilingualism: Language and Cognition, (), –. 10.1017/S1366728921000535
     https://doi.org/10.1017/S1366728921000535 [Google Scholar]
  5. Chambers, C. D.
    (2013) Registered reports: A new publishing initiative at Cortex. Cortex, (), –. 10.1016/j.cortex.2012.12.016
     https://doi.org/10.1016/j.cortex.2012.12.016 [Google Scholar]
  6. Chambers, C. D., & Tzavella, L.
    (2022) The past, present and future of Registered Reports. Nature Human Behaviour, (), –. 10.1038/s41562‑021‑01193‑7
     https://doi.org/10.1038/s41562-021-01193-7 [Google Scholar]
  7. Czarniawska, B.
    (2004) Narratives in social science research. Sage. 10.4135/9781849209502
     https://doi.org/10.4135/9781849209502 [Google Scholar]
  8. DeSantis, L., & Ugarriza, D. N.
    (2000) The concept of theme as used in qualitative nursing research. Western Journal of Nursing Research, (), –. 10.1177/019394590002200308
     https://doi.org/10.1177/019394590002200308 [Google Scholar]
  9. Drax, K., Clark, R., Chambers, C. D., Munafò, M., & Thompson, J.
    (2021) A qualitative analysis of stakeholder experiences with registered reports funding partnerships. Wellcome Open Research, , . 10.12688/wellcomeopenres.17029.1
     https://doi.org/10.12688/wellcomeopenres.17029.1 [Google Scholar]
  10. Fanelli, D.
    (2009) How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS One, (), . 10.1371/journal.pone.0005738
     https://doi.org/10.1371/journal.pone.0005738 [Google Scholar]
  11. Godfroid, A., & Andringa, S.
    (2023) Uncovering Sampling Biases, Advancing Inclusivity, and Rethinking Theoretical Accounts in Second Language Acquisition: Introduction to the Special Issue SLA for All?Language Learning, lang.12620. 10.1111/lang.12620
     https://doi.org/10.1111/lang.12620 [Google Scholar]
  12. Godlee, F.
    (2002) Making reviewers visible: Openness, accountability, and credit. JAMA, (), –. 10.1001/jama.287.21.2762
     https://doi.org/10.1001/jama.287.21.2762 [Google Scholar]
  13. Hardwicke, T. E., & Ioannidis, J. P. A.
    (2018) Mapping the universe of registered reports. Nature Human Behaviour, (), –. 10.1038/s41562‑018‑0444‑y
     https://doi.org/10.1038/s41562-018-0444-y [Google Scholar]
  14. Huensch, A.
    (2024) Open science and preregistration. InL. Plonsky (Ed.), Open science in applied linguistics (pp.–). John Benjamins.
     [Google Scholar]
  15. Hummer, L., Thorn, F. S., Nosek, B. A., & Errington, T. M.
    (2017) Evaluating registered reports: A naturalistic comparative study of article impact. https://osf.io/5y8w7
     [Google Scholar]
  16. Isbell, D. R., Brown, D., Chen, M., Derrick, D. J., Ghanem, R., Arvizu, M. N. G., … Plonsky, L.
    (2022) Misconduct and questionable research practices: The ethics of quantitative data handling and reporting in applied linguistics. Modern Language Journal, (), –. 10.1111/modl.12760
     https://doi.org/10.1111/modl.12760 [Google Scholar]
  17. Kerr, N. L.
    (1998) HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, (), –. 10.1207/s15327957pspr0203_4
     https://doi.org/10.1207/s15327957pspr0203_4 [Google Scholar]
  18. Lakens, D.
    (2019) The value of preregistration for psychological science: A conceptual analysis. Japanese Psychological Review, (), –. 10.31234/osf.io/jbh4w
     https://doi.org/10.31234/osf.io/jbh4w [Google Scholar]
  19. Larsson, T., Plonsky, L., Sterling, S., Kytö, M., Yaw, K., & Wood, M.
    (2023) On the frequency, prevalence, and perceived severity of questionable research practices. Research Methods in Applied Linguistics, (), . 10.1016/j.rmal.2023.100064
     https://doi.org/10.1016/j.rmal.2023.100064 [Google Scholar]
  20. Laurinavichyute, A., Yadav, H., & Vasishth, S.
    (2022) Share the code, not just the data: A case study of the reproducibility of articles published in the Journal of Memory and Language under the open data policy. Journal of Memory and Language, , . 10.1016/j.jml.2022.104332
     https://doi.org/10.1016/j.jml.2022.104332 [Google Scholar]
  21. Liu, M., Al-Hoorie, A. H., & Hiver, P.
    (2024) Open access in language testing and assessment: The case of two flagship journals. Language Testing. 10.1177/02655322241261417
     https://doi.org/10.1177/02655322241261417 [Google Scholar]
  22. Liu, M., & Marsden, E.
    (2024) The open turn: Rethinking applied linguistics research through open scholarship. OSF. 10.31219/osf.io/9kqvf
     https://doi.org/10.31219/osf.io/9kqvf [Google Scholar]
  23. Marsden, E., Crossley, S., Ellis, N., Kormos, J., Morgan-Short, K., & Thierry, G.
    (2019) Inclusion of research materials when submitting an article to Language Learning. Language Learning, (), –. 10.1111/lang.12378
     https://doi.org/10.1111/lang.12378 [Google Scholar]
  24. Marsden, E., & Morgan-Short, K.
    (2023) (Why) are open research practices the future for the study of language learning?Language Learning, –. 10.1111/lang.12568
     https://doi.org/10.1111/lang.12568 [Google Scholar]
  25. Marsden, E., Morgan-Short, K., Trofimovich, P., & Ellis, N. C.
    (2018) Introducing registered reports at language learning: Promoting transparency, replication, and a synthetic ethic in the language sciences. Language Learning, (), –. 10.1111/lang.12284
     https://doi.org/10.1111/lang.12284 [Google Scholar]
  26. Montoya, A. K., Krenzer, W. L. D., & Fossum, J. L.
    (2021) Opening the door to registered reports: Census of journals publishing registered reports (2013–2020). Collabra: Psychology, (), . 10.1525/collabra.24404
     https://doi.org/10.1525/collabra.24404 [Google Scholar]
  27. Munafò, M. R.
    (2017) Improving the efficiency of grant and journal peer review: Registered reports funding. Nicotine & Tobacco Research, (), –. 10.1093/ntr/ntx081
     https://doi.org/10.1093/ntr/ntx081 [Google Scholar]
  28. Nosek, B. A., Spies, J. R., & Motyl, M.
    (2012) Scientific utopia: II. restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science, (), –. 10.1177/1745691612459058
     https://doi.org/10.1177/1745691612459058 [Google Scholar]
  29. Obels, P., Lakens, D., Coles, N. A., Gottfried, J., & Green, S. A.
    (2020) Analysis of open data and computational reproducibility in registered reports in psychology. Advances in Methods and Practices in Psychological Science, (), –. 10.1177/2515245920918872
     https://doi.org/10.1177/2515245920918872 [Google Scholar]
  30. Parsons, S., Azevedo, F., Elsherif, M. M., Guay, S., Shahim, O. N., Govaart, G. H., … Todorovic, A.
    (2022) A community-sourced glossary of open scholarship terms. Nature Human Behaviour, (), –. 10.1038/s41562‑021‑01269‑4
     https://doi.org/10.1038/s41562-021-01269-4 [Google Scholar]
  31. Plonsky, L.
    (2024) Open science in applied linguistics. Applied Linguistics Press.
     [Google Scholar]
  32. Plonsky, L., Brown, D., Chen, M., Ghanem, R., Arvizu, M. N. G., Isbell, D. R., & Zhang, M.
    (2024) “significance sells”: Applied linguists’ views on questionable research practices. Research Methods in Applied Linguistics, (), . 10.1016/j.rmal.2024.100099
     https://doi.org/10.1016/j.rmal.2024.100099 [Google Scholar]
  33. Saldaña, J.
    (2016) The coding manual for qualitative researchers. Sage.
     [Google Scholar]
  34. Simmons, J. P., Nelson, L. D., & Simonsohn, U.
    (2011) False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, (), –. 10.1177/0956797611417632
     https://doi.org/10.1177/0956797611417632 [Google Scholar]
  35. Steneck, N. H.
    (2006) Fostering integrity in research: Definitions, current knowledge, and future directions. Science and Engineering Ethics, , –. 10.1007/PL00022268
     https://doi.org/10.1007/PL00022268 [Google Scholar]
  36. Waltman, L., Kaltenbrunner, W., Pinfield, S., & Woods, H. B.
    (2023) How to improve scientific peer review: Four schools of thought. Learned Publishing, (), –. 10.1002/leap.1544
     https://doi.org/10.1002/leap.1544 [Google Scholar]
  37. Wood, M., Sterling, S., Larsson, T., Plonsky, L., Kytö, M., & Yaw, K.
    (2024) Researchers training researchers: Ethics training in quantitative applied linguistics. TESOL Quarterly, tesq.3323. 10.1002/tesq.3323
     https://doi.org/10.1002/tesq.3323 [Google Scholar]
/content/journals/10.1075/jsls.00045.liu
Loading
The Ulysses pact
John Benjamins ; https://doi.org/10.1075/jsls.00045.liu
/content/journals/10.1075/jsls.00045.liu
/content/journals/10.1075/jsls.00045.liu
Loading

Data & Media loading...

  • Article Type: Research Article
Keywords: peer review ; replication research ; open science ; registered reports

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error