Peer review: Problematic or promising?

One day in April began rather badly (dogs/wet weather/mud – you know the story) and then as I started my ELRR work, it became clear it was definitely going to be memorable – for all the wrong reasons. Within an hour of sitting down at my computer I received, in quick succession

• a visceral complaint from an author about a review of their article

• a grumble from another author about the delay in accepting his paper

• separate whinges from two Associate Editors about particular reviewers (slow/too stern)

• a reviewer felt obliged to complain that an author had ignored their advice in a just-published article …

And so it went on – and on– and on …….

Yet again, the process of peer review was under scrutiny. Indeed, it seemed to me that day, that – if nothing else, eliminating peer review might stem the steady flow of whinges, moans, and grumbles about the process of blind reviewing/peer-reviewing/refereeing manuscripts, a practice common to most academic journals. Of course, the day later improved and I stopped sticking pins into plasticine authors, reviewers, and associate editors. Nevertheless, it once more raised that abiding question – what is peer review and is it really a good thing? (Bedeian 2003; Miner 2003; Kerig 2021)

What is peer review?

Peer review has a long history in the sciences, having developed as a means to enable and ensure ‘good science’ (Moxham and Fyfe 2018). Practising scholars since the 16^th century or earlier have been asked to read and critique the research of fellow scholars. More recently, such reviewing has tended to be ‘double – blind’, where the reviewers do not know who is the author and the author does not know who are the reviewers. That way, so the story goes, reviews are made objectively on the substance of the article, rather than on the stature or status of the authors. That is sometimes a flawed assumption, but it is arguable that peer review mostly works for the benefit of scholarship and the encouragement of scholars - well, so some of us say! Let me trace the process at ELRR.

Peer review and ELRR

A paper is submitted to ELRR through the comprehensive but occasionally counter-intuitive software through which authors submit their articles. The software ascribes the article a unique number and a strict timetable. The relentlessness of the early 20th century assembly line pales in comparison with modern journal software, intent as it is on surveillance and constant pressure to move an article in or out (and never between).

The Editorial Group does a first screening. The Editorial Group is another hidden powerhouse of the journal – it comprises the editors, coordinators, and editorial experts who make things happen and also fix problems, large and small. They also evaluate whether submitted papers are relevant to ELRR and are correctly set out in house style and with separate files provided for each table, figure, or picture. The selected papers are then transferred along the software path from the Editorial Group to the Editor-in-Chief (EIC), who then looks for someone in that wonderful unsung voluntary workforce, the Associate Editors (AEs). Generally, AEs are members of the journal’s Editorial Board who have expertise in the specific subfield of a paper. This is a considerable challenge in our great multidisciplinary journal where we need expertise in heterodox economics of various hues, (post-Keynesian, Sraffian, Kaleckian, Marxian), industrial relations, labour economics, industrial sociology, employment studies, financialisation of housing, global inequalities … (see this issue for the breadth of ELRR!). We need AEs with excellent quantitative expertise and AEs with thorough qualitative capabilities – and of course we need AEs with a deep understanding of ways of aligning the quantitative and qualitative research, while also imbued with the wisdom of Solomon to appreciate a good argument, sound metholodology  and a strong case (or vice versa).

Much of the article decision-making rests with AEs (sometimes called Area or Coordinating Editors rather than Associate Editor). Once they have agreed to take responsibility, the designated AE will first read the article assigned to them. (The statement ‘Read the article’ is a very few words that can be said much more quickly than acted upon. What may well be needed, is a very long, tortuous read). If the AE believes that the article has merit or potential, the AE will then begin the search for another rara avis, Scrupulous, Collegial, Scholarly Reviewers (SCSR) who have expertise in the area of the article.

At ELRR we are a double-blind peer review journal. The reviewers do not know who are the authors, and the authors do not know the reviewers. The trouble for AEs is that available SCSR are a scarce commodity for obvious reasons and discussed below. It is not unknown for AEs to approach 8–10 potential reviewers before one agrees (16 refusals is this year’s best effort – and it’s only April!). Being an AE can be frustrating and unrewarding (and probably made worse by the naggy editor in the background asking ‘how are things going?’ but that’s another story.). Eventually the AE finds two potential SCSR whom they need to convince that, for absolutely no return or personal gain, they should review the paper within a specific time frame (and probably adding the cheeky rider ‘and if your review asks for major revisions, would you agree to review the revised manuscript, too please?’).

Eventually, the process of the EIC nagging the AEs, the AEs niggling the reviewers, and the publishing software browbeating everyone, produces two reviews from two SCSR. The AE will need to interpret the reviews, perhaps highlighting some aspects and softening or emphasising criticisms. The full text of the reviews and the AE’s recommendations are then bumped along by the unabating software to the Editorial Team, especially the (EIC). The EIC (or nominee) then decides how to proceed. Usually, the EIC follows the AE recommendations pretty closely, when the EIC notifies authors of the decision about the article.

That is the simple version of the process of peer review at the ELRR but of course, there are complexities – the reviews may be conflicting either in the substance of reviewers’ concerns or their recommendations. There are four choices reviewers can recommend – Accept, Minor Revisions, Major Revisions, and Reject. But different reviewers have different ideas on what comprises minor revisions. Is a rewriting of the literature review a minor or major change? What about a significant change in methods? Or asking authors to include new, conflicting research? And reviewer demeanour varies greatly too. In some cases, reviewers are quite stern and uncompromisingly critical, while in other cases, they are collegial and kindly, even as they explicate the need for major revisions. Later, revised versions may not be up to the mark, either. In other words, the process of evaluating manuscripts requires much good judgement from reviewers, Associate Editors, and the EICs as they deal with likely complications. Indeed, it is rather surprising that we can produce 11–13 research articles/review essays in each issue, four times a year, especially given this complex – and implacable process! Similarly, it is not surprising that peer-reviewing in academic journals has often come under fire.

Problems of peer-reviewing

There is a legion of literature on how and why peer review is problematic. Scholars have asserted that peer-reviewing stifles new ideas and strengthens scholarly conservatism (Dondio et al 2019; Kaltenbrunner et al 2022). The invisible college can thrive under peer review, and in so doing, new ideas, iconoclasm, or studied dissent, can be silenced or muted. Many of our heterodox economics colleagues have dealt with such stresses in responses to their research. Their scholarship does not draw on often unquestioned neoclassical assumptions, for example, so some neoclassical scholars have deep epistemic doubts reading heterodox research. In the same way, labour relations scholars who begin with the assumption that there is inherent conflict and competition in all employment relationships, are similarly side-lined by journals which assume that good HR management or the Engine Charlie principle will resolve any conflict anywhere, including in the capitalist employment relationship. In these respects, peer review can be an effective means of suppressing dissent or the questioning of foundation assumptions of exchange and scarcity in economics, or control and surveillance in labour studies. (Kerig 2021; Krlev and Spicer 2023; Lanier 2021; Watling et al 2021). That is precisely why building interdisciplinary and multidisciplinary scholarship is important – it seeks a level, if bumpy, playing field for publishing scholarship.

Secondly, as the tortured description of the peer review process at ELRR above reveals, peer review is not only extraordinarily time-consuming, but it also depends on the e xploitation and selfless goodwill of many good scholars. Virtually no tenure or academic promotion ever rested on the number of articles reviewed or the quality of peer review. In this respect, the ELRR was an early adopter of Publons, a form of software that enables scholars to track and highlight their work as peer-reviewers (Wilkinson and Down 2018). We always ask reviewers if they wish their reviews to be rated and recorded. However, anecdotal evidence suggests that promotion committees in most universities do not yet accord peer-reviewing, whether verified or not, much weight, so in many respects, peer review is the ultimate in academic altruism. Our reviewers and Associate Editors give their scholarly expertise and their precious time to support the discipline, the subfield, and the genre, in which they research and teach. A publication can bring its own rewards, but peer-reviewing may be seen to bring nothing formal or gainworthy – at least at first glance. There is also a certain irony that a journal such as ELRR, committed to dissent, heterodoxy, and social justice, depends so heavily on peer review!

Moreover, barring a few interesting articles on how to be a good peer-reviewer, there are no agreed standards for peer review – precisely because academic tradition has it that scholars will just know what is good and what is not good in the journal articles. And perhaps, decades or eons ago, that was the case but that was before the commodification of scholarship, the rise of neoliberal education, and the corporatisation of publishing, before academic journals were ranked and ordered according to a myriad of quantitative and subjective factors, and their impact factors measured in a multitude of ways. Today, just knowing what is good in a discipline, subfield or genre will to some extent depend on scholarship but it also depends on many may other factors, such as metrics or funding. The pressure on ‘publish or perish’ is only slightly less than the pressure to gain prestigious research grants – and often these cannot be achieved without evidence of publications. (McKeown 2022) So the peer review circle rolls around and around.

So what is good about peer review?

So why do we do it? Why do AEs try 5, 10, 15 times to find good reviewers? Why do reviewers agree to read rigorously, wholly new material with no provenance, and then provide constructive and helpful comments? Why do authors submit to journals with high ideals? There are quick answers to this. Peer review legitimises scholarship, while at the same time building scholarly understanding and knowledge for reviewers as well as authors. (Dewidar et al 2022; Kerig 2021) We can assume that reviewers and editors have integrity, and are mostly altruistic. Moreover, despite the spectre of ‘Reviewer Two’ (Krlev and Spicer 2023; Watling et al 2021), these reviewers and editors also gain from brief forays into national and international scholarly communities. Indeed, this is a prime method of disseminating knowledge, surely a deeply important aspect of all scholarship. We are not only researchers, but we must also be communicators of our research. Knowledge should be shared, scrutinised, discussed, and shared again, not kept tightly held by a (self-chosen) few (Lim 2021). So, it is our role to ensure good research is not only disseminated but also subjected to thoughtful and collegial scrutiny. In the time-consuming, initial reading of an article by AEs or reviewers, we are extending our appreciation of international scholarship in a global world – we are participating in a (sometimes muffled) dialogue, sharing our ideas and learning from others. That is the strength of peer review.

In short then, peer review enables collaboration and international dissemination of ideas, and in so doing offers a potentially powerful antidote to imperatives of competitiveness and academic silos. We need to keep working to make it better, and perhaps less time-consuming, but that does not mean we should change our ways (Dewidar 2022; Lim 2021). We get better stronger scholarship, the dissemination of ideas, and effective collegiality – that is the promise of peer review.

Concluding messages

My message to authors – Take heed of reviewers. Please believe that they are (almost always) scholars with integrity who have given up 5 or 6 hours of their scarce time to read your manuscript critically, to draft comments cogently, and then to send those comments and recommendations to editors. Of course, your reviewers are also human – they may make mistakes or misinterpret what you have said. Address their comments – don’t reject them out of hand. Finally, do please be patient with the reviewers – they have research and teaching and marking and administration demands and families, to say nothing of families and football – and despite the best efforts of Publons, they will probably stand to gain nothing tangible from their reviewing. They are doing their best for you. And, also, thank you dear authors for submitting your best work to our journal. We need you too!

My message to AEs Yours is a focussed and sometimes near thankless job of using your expertise to decide which articles should continue towards publication and of finding reviewers with appropriate expertise who will read and respond collegially to a manuscript. Please keep being fabulous Associate Editors, but please, if at all possible, please try to check the progress of articles every two or three days. Please try not to leave it for a few weeks before you find excellent expert reviewers. But also – thank you for being such marvellous and essential links in the journal’s chain.

My message to reviewers (and a gentle hint to those who are sometimes too busy to review?) Thank you for your reviews. Thank you for helpful, rigorous, encouraging comments and for clear understanding of what authors are saying – or are trying to say. Thank you for those 4-5-6 hours spent on reading, evaluating, mulling over, each manuscript. Thank you for keeping scholarly journals alive and well. Please keep agreeing to review – we all need you!

Yes, peer review is time-consuming, annoying, demanding – but it is also building links and helping research and communicating new ideas and knowledge – and that, after all, is what we are about, isn’t it?

And now to this issue 34-2

The first four articles of this issue are further testament to the huge respect accorded to great Cambridge-Australian economist, G C Harcourt. Prof Peter Kriesler, one of Australia’s most erudite heterodox economists, will introduce those first four articles.

The latter part of 34-2 (June 2023) is rather a world tour. The first article is from John Burgess and Sagi Mathew. It is a model of clarity for article titles. This may seem quite trivial, but it has become very obvious in recent years that tricksy or insinuating article titles are, ceteris paribus, less likely to be cited than those that are unambiguous. So, strengthening the chance of citation for the authors and the journal – Burgess and Mathew’s title is a winner. ‘Two Decades of Industrial Disputation at an Indian Auto Plant: Lean Production versus Local Cultural Values’ offers a thorough, thoughtful, and indeed model, case study highlighting the ways in which expatriate managers’ lack of respect for local customs and workers’ grievances, can have costly and lengthy negative impacts. Moving the focus from India to Turkey, Mercan and Barlin’s article explores the impact of the sudden and massive influx of refugees from Syria into Turkey over two time periods. Their analysis finds this major shock had some surprisingly positive effects on labour market efficiency. The next article is from a scholar in England, dealing with a global issue. In an era when financialisation is dominant, Dunn explores what is needed for financial literacy. He argues the need to go beyond simplistic assumptions, instead giving weight to structural inequalities, highlighting many barriers to consumers being able to gain real financial literacy. From the Iberian Peninsula, Xabier Arrizabalo Montoro, Mario del Rosal Crespo, and Francisco Javier Murillo Arroyo offer an incisive, critical analysis of growing precarious employment forms in Spain, such as bogus internships. The research offers a gloomy conclusion for continuing poor wages and conditions in Spain’s youth labour market. Perhaps the most novel article in this issue (in a highly competitive field) is Fragmented Control of Platform Game Work in China by Mengyang Zhao. The author skilfully applies labour process theory to occupations not even envisaged a decade ago – the video game testers, trainers, and boosters on gaming platforms in China. Zhao demonstrates cogently the fragmented but effective surveillance and control mechanisms used, and the forms of resistance adopted by these precarious workers in the gamer industry.

Our Contested Terrains article for this issue is likely to provoke comment and discussion, perhaps an important objective for Professor Piotr Żuk, who explores the current war in Ukraine and its impact on labour and social justice. We always ask our reviewers of Contested Terrains articles to allow for the immediacy or controversy around issues under investigation. Articles should always be scholarly, but some allowance for polemics or passion is also important. This is very evident in Żuk’s insightful article.

The last offerings in this issue begin with three highly articulate and insightful obituaries. Rowan Cahill discusses the complexities of the wonderful John Michael O’Brien (21 December 1945–10 January 2023). This is followed by two complementary and admiring discussions of the greatly respected economist Luigi Ludovico Pasinetti (12 September 1930–31 January 2023) by Kumaraswamy Velupillai and Joseph Halevi, respectively. All three obituaries are readable discussions of two great people which also point once more, to the centrality of context for the individual. The final article is an extensive and evocative review essay by Braham Dabscheck, analysing the mammoth Joseph Fishkin and William E Forbath, The Anti-Oligarchy Constitution: Reconstructing the Economic Foundations of American Democracy.

So, as has become the norm for the ELRR, this issue once again has authors from across the world and covers economic, social, and labour issues from almost every continent. All items were peer-reviewed – and it behoves me once more, to offer my sincerest appreciation and admiration to the authors, peer reviewers, and Associate Editors for this issue. To our heterodox and interdisciplinary communities, I thank you and salute you!