Enhancing visibility of the gender dimension in research: Organising a Summer School in peer reviewing

A pilot half day workshop on peer review processes took place on the 29th of May, 2018 at the University of Warwick to gauge interest about this topic and get feedback about the content and structure of the subsequent summer school. PhD candidates and early career researchers expressed great interest and provided useful feedback for our summer school which informed the final programme. In this half day workshop, 2 academics (1 Editorial Board member and 1 member of a Research Council Peer Review College) and 1 Senior Editor (from Warwick’s academic journal) provided a brief insights into their experiences of the peer review process. In the second half of the workshops, participants peer reviewed conference abstracts and engaged into a discussion about how to provide constructive feedback.

Based on this experience and feedback, a summer school took place from September 10th to 14th, 2018, at the University of Warwick. The first two days (10th, 11th) were targeted at early career researchers from natural sciences and engineering and the 4th and 5th day (13th, 14th) were targeted at those from social sciences and arts and humanities. Each session was limited to 35 participants in order to enable active participation of all attendees. The morning sessions included talks from academics and representatives of publishing houses and scientific journals focused on the peer review process, from submission of a paper/funding proposal to the outcome of the peer review while the afternoon sessions were practical, hands-on workshops on how to conduct peer review, led by experienced editors and peer reviewers (see programme). There were also 2 dedicated sessions on considerations of the peer review process in relation to ethics, plagiarism, conflicts of interest and most importantly biases in the peer review process and considerations of sex and gender in research and publishing. The 3rd day (12th) was comprised of a writing bootcamp and a CV workshop where both natural and social scientists had access to a quiet room with desks, lunch, refreshments and experienced facilitators to help them review papers, proposals, and CVs. Participants were invited to ask questions and raise any concerns and were given time for networking.

After the summer school, the material used presented by speakers was uploaded to the summer school’s website and participants were informed via email.

We chose to have a mix of internal (from the University of Warwick) and external speakers. Due to the rich academic expertise of Warwick staff, we contacted were able to include academics with long experience in editorial boards and evaluation of funding bids (from various different disciplines departments including Chemistry, Life Sciences, Sociology, Economics). We also identified 9 external speakers who held senior positions in academic publishing (Elsevier, Nature; Springer Nature) and/or had extensive experience and recognition in peer review process, but also had been involved in initiatives of peer review which touched upon such issues as: biases, sex and gender or diversity (speakers from the European Association of Science Editors (EASE) and gender studies’ academics).

See overview of the programme: https://warwick.ac.uk/fac/soc/pais/research/projects/plotina/summerschool/programme/

Communication

We circulated the invitation for this action through various newsletters (university-wide), including the Learning and Development Centre (LDC) and the Doctoral Training Centres at the University of Warwick. We contacted departmental administrators across various faculties and departments to promote the event to early career researchers. We also disseminated the call for participation to other universities within reasonable travelling distances, including the University of Birmingham, the University of Oxford, the University of Southampton, and Aston University, though we stopped advertising when the event was full.

1. Participants may not think the discussion about biases in knowledge production and sex/gender considerations is relevant to them or this topic.
2. Getting highly esteemed speakers to contribute to this programme (they have very busy schedules).
3. People signed up to attend and then did not bother to come.

1. Be clear about the benefits of attending such an event and ensure that you get and respond to feedback from participants beforehand and during the programme.
2. Start planning well-ahead of the event and motivate potential speakers by the benefit to early career researchers.
3. Charging a small registration fee can encourage commitment and/or discourage casual sign-ups

Think about capacity and how many participants you can accommodate. While the talks can be open to a larger number of participants, when it comes to practical workshops, it is more difficult to run them with more than 35 participants (considering that they often do group work, 7 groups of 5 people or 5 groups of 7 people can work well). We had an overwhelming response in registration (90 tried to register, we allowed for 70 and we had 20 people in the waiting list). However, there was a big dropout rate so we recommend allowing for 1015% over-subscribing to mitigate this. For this reason, you might also want to consider adding the following statement in the registration form for running such a free event ‘By registering you are holding a place at this event. You need to notify the organisers as early as possible if you cannot attend so your place can be offered to another participant. Failure to do so might lead to a charge of x amount’. A small registration fee (e.g. to cover food costs) might be the best way to gain commitment.

Due to time constraints and hectic schedules, events like this could be run in 1.5 days tailored either to specific disciplines or broader disciplinary areas. Participants appreciated disciplinary expertise (at least broad disciplinary expertise) and would have valued a more ‘compact’ schedule to get the most of it. However, the half day pilot programme was a bit too compressed and participants did not get to try it out.

Develop the content of the programme and identify speakers 4-5 months before the day of the event.

Check whether you can capture (audio/video) the talks and get consent (from speakers and participants) to develop a resource that could be accessible to the wider community.

Budget for: food/coffee, travel/accommodation expenses for speakers, printing and folders, administrative assistance.

The summer school was well received. We were overwhelmed with interest for the event (70 individuals registered) and we had to put participants on a waiting list (+20). The delegates who filled out the evaluation form (29 filled in the form) gave positive feedback. Among the topics they found particularly interesting and beneficial were the following: talks and discussion on how to conduct peer reviews, discussion about gender imbalance in the research context, and the practical sessions. Participants also mentioned that they benefited from the speakers’ variety of disciplines and backgrounds and commented positively on how well prepared the speakers were. 

Participants highlighted that the event was very comprehensive and it covered everything about peer review. Some of them would have liked to participate in more practical sessions and hands on exercises e.g. review longer articles or real articles submitted to the Warwick journal.

An evaluation form was designed on the summer school’s website (an evaluation form template is available and it can be used to develop an online form print and disseminate it at the end of the event). Participants were informed verbally about it at the end of the summer school and via email. They also received a reminder email a week later. The form was simple and brief including open ended questions to encourage participants to fill the form and expand on their feedback. The form was comprised of:

• Which topics or aspects of the summer school did you find most interesting or useful?
• What else do you feel should have been included? 
• Do you have any further comments/ suggestions?

It was challenging to run the practical workshops with high numbers of participants. If run again we would split the participants in groups for the practical workshops.

The summer school was offered for free and there was a big dropout rate so we recommend allowing for 10-15% over-subscribing to mitigate this and charging a small registration fee.