As a co-editor-in-chief for the journal System (long name An International Journal of Educational Technology and Applied Linguistics), I was delighted to have the opportunity to speak about my reflections on peer review as part of the BERA event series Journal Peer Review: What are Journal Editors Looking for? In this blog post, I provide an overview of the talk, including some statistics, reasons for desk rejections and insights into the peer review process.

What are the editors looking for?

System is a leading journal in the field of applied linguistics. The journal has a historical focus on language education technology, but it maintains a scope covering all applied linguistics research with implications for foreign language teaching and learning. With such a broad scope, we receive a lot of papers – more than 1,100 manuscripts in 2021, and that number is already higher in 2022. System is a large journal, with five co-editors-in-chief, as well as associate, book reviews, assistant and student editors. We maintain an acceptance rate of around 17 per cent, which means we publish a lot of papers – 145 were published in 2021.

With so many submissions, we must make quick decisions about whether to progress a manuscript to the review process. We had a 63 per cent desk rejection rate in 2021, so about one out of every three submissions were considered for review. This means the first ‘peer review’ with a submission comes from the handling editor. Here I identify eight key reasons for a ‘desk rejection’:

• poor fit for aims and scope
• original contribution to knowledge is unclear
• poor rationale
• not engaged in most current discussions
• methodology (design, decisions) is unclear or lacks rigour
• arguments are not sufficiently supported by the data
• implications are not applicable to a wider audience
• inconsistencies/poorly organised.

‘We had a 63 per cent desk rejection rate in 2021, so about one out of every three submissions were considered for review.’

As a journal with a broad range, submissions to System do not always clearly address the implications for foreign language teaching and learning, and therefore fall outside the scope. Other reasons for desk rejection overlap somewhat, as an unclear contribution or reason for the study, or the absence of the most current literature, may mean the manuscript is outside the scope. Methods may be explained too briefly without sufficient, careful support from methodological literature. Data presented may not align sufficiently with the argument, leaving findings unsupported. Implications of the research do not need to be generalisable to a wide audience (as found in quantitative studies), but they need to have provided enough information that a broad international readership can make connections with the research. Finally, a poorly presented manuscript may be too much work for editors and reviewers to follow. Most desk-rejected manuscripts fall into only one of these categories, but that is enough for a rejection.

A good way to understand the aims and scope of a journal is to look at the most read and most cited papers in a particular journal, especially the most recent ones. In System, those recent papers have been focused on positive psychology in language education, particularly foreign language anxiety, enjoyment and emotion. As such, we certainly welcome submissions on these topics, but authors need to be especially clear about how their work differs from the papers already published in the journal – in short, what original contribution are they making?

What happens in the peer review process?

Approximately 35 per cent of manuscripts submitted to System progress to the double-blind review process, which means we invite scholars working in related research to review a ‘blinded’ manuscript (that is, anything to make the authors identifiable has been removed). A minimum of two reviewers will complete the review and make a recommendation to the handling editor to reject, revise or to accept. As an editor, I decide based on both independent reviewers’ recommendations along with my own impressions. Without sufficient indications that the manuscript will result in publication after revision, it will be rejected. The comments received can be very helpful for reworking the manuscript for submission to another journal, as recommended by Gao (2017).

The double-blind review process has been noted as challenging for reviewers, as there is little guidance. For a recently rejected manuscript submitted to System, there were three reviewers with different perspectives. They agreed that the study was worthwhile, but the theoretical and empirical bases were poor, which meant the findings and discussions were weak. One reviewer felt this could be addressed through revision, but the other did not. I invited a third reviewer who also raised concerns about the chances of the manuscript being published.

My strong recommendation for all those entering or engaging in journal peer review is to get experience as a reviewer whenever possible. The process of evaluating others’ work, offering constructive criticisms and ideas for revision, and communicating these with an editor as well as authors, is an invaluable professional development practice. System is published by Elsevier, which offers plenty of advice for new reviewers such as an explanation of the role of the reviewer.

This blog post relates to a series of events that the BERA ECR Network hosted in 2022. The events explored peer review in journal publishing through in-depth case studies presented by the editors of leading international journals. Further blog posts from the event series will be published on the BERA Blog this week.

References

Gao, X. (2017). Dealing with criticism when publishing qualitative research. In J. McKinley & H. Rose (Eds.). Doing research in applied linguistics (pp. 225–234). Routledge.