There are limited opportunities for early career researchers to understand the peer review process and sometimes getting insight, advice and/or training on how to conduct and respond to peer reviewing depends on informal networks and willing supervisors/line managers that not all individuals will have access to.
Enhancing visibility of the gender dimension in research: Organising a Summer School in peer reviewing
The objective of this summer school was to provide academic-led information about the peer review process. The speakers included academic members of editorial boards, editors, publishing directors, and senior career consultants. The summer school was addressed to PhD students and postdoctoral researchers.
Things you need to do for this event:
Before the event
- Decide and develop the content of the summer school, structuring the programme, drawing the budget
- Identify and invite speakers 4−5 months before the date of the event to ensure that they are available and they can commit to contributing to the programme
- Liaise with speakers regarding accommodation and travel, provide expense claim forms and be explicit about what costs will be covered
- Arrange the logistics of the event (food/drink − providing some catering makes participants feel valued and more involved); liaise with catering services
- Develop material (name badges, signs for event, folders with info on the programme and speakers)
- Develop relevant webpages to advertise the aims of the event, speakers, programme and launch a registration page
- Promote and disseminate the event internally (within the institution) − see section Communication − if that is the target audience. Alternatively, address your external audience. Develop a poster and send emails to target groups, as well as advertise using any training, personal development websites etc. which early career researchers use to find available training. Express succinctly an outline of the main objectives, the proposed structure and the target audience for the summer school.
A couple of days before/on the day of the event
- Send reminder email to speakers with the final programme, venue information and sending presentations/material in advance
- Send reminder email to participants about the final programme, venue information and registration
- Develop material (name badges, signs for event, folders with info on the programme and speakers for registration)
- Get material from speakers and print them
- Ensure technical equipment and venue appropriateness in advance (go before the event to check that computer/projector/microphone are available and working).
- It is important to have 1−2 individuals available on the day(s) of registration to support speakers and participants.
After the event
- Follow up with participants after the event; send them an evaluation form to provide feedback (this may be better done at the conclusion of the workshop) and also provide information about the availability of the slides and additional resources provided by the speakers and the organiser
- Follow up with speakers to ensure that they give consent for their slides and other material to be shared electronically (they might want to amend them slightly)
- Write a thank you note to all speakers for all their help. For external speakers, remind them to send all their receipts for costs claimed.
- Collect all costs and receipts to finalise the budget.
Forty seven PhD students and postdoctoral researchers participated in the summer school. Speakers from a diverse array of backgrounds shared their experience and provided valuable insights into the peer review process, including sex/gender issues. Participants had the opportunity to enhance their knowledge on the peer review process, attending talks on the peer review process and funding bodies, how to respond to peer reviews, and how to write peer reviews. Participants also participated in a writing boot camp and attended a CV workshop and a peer review training course (delivered by Dr Julia Vilstrup Mouatt, head of Publons’ peer review training course, the Publons Academy). An evaluation form was delivered to participants.
Researchers and research: gender equality and sex and gender perspective
Type of action
University of Warwick
Higher education institution
Action level of implementation
Researchers and professors
A pilot half day workshop on peer review processes took place on the 29th of May, 2018 at the University of Warwick to gauge interest about this topic and get feedback about the content and structure of the subsequent summer school. PhD candidates and early career researchers expressed great interest and provided useful feedback for our summer school which informed the final programme. In this half day workshop, 2 academics (1 Editorial Board member and 1 member of a Research Council Peer Review College) and 1 Senior Editor (from Warwick’s academic journal) provided a brief insights into their experiences of the peer review process. In the second half of the workshops, participants peer reviewed conference abstracts and engaged into a discussion about how to provide constructive feedback.
Based on this experience and feedback, a summer school took place from September 10th to 14th, 2018, at the University of Warwick. The first two days (10th, 11th) were targeted at early career researchers from natural sciences and engineering and the 4th and 5th day (13th, 14th) were targeted at those from social sciences and arts and humanities. Each session was limited to 35 participants in order to enable active participation of all attendees. The morning sessions included talks from academics and representatives of publishing houses and scientific journals focused on the peer review process, from submission of a paper/funding proposal to the outcome of the peer review while the afternoon sessions were practical, hands-on workshops on how to conduct peer review, led by experienced editors and peer reviewers (see programme). There were also 2 dedicated sessions on considerations of the peer review process in relation to ethics, plagiarism, conflicts of interest and most importantly biases in the peer review process and considerations of sex and gender in research and publishing. The 3rd day (12th) was comprised of a writing bootcamp and a CV workshop where both natural and social scientists had access to a quiet room with desks, lunch, refreshments and experienced facilitators to help them review papers, proposals, and CVs. Participants were invited to ask questions and raise any concerns and were given time for networking.
After the summer school, the material used presented by speakers was uploaded to the summer school’s website and participants were informed via email.
We chose to have a mix of internal (from the University of Warwick) and external speakers. Due to the rich academic expertise of Warwick staff, we contacted were able to include academics with long experience in editorial boards and evaluation of funding bids (from various different disciplines departments including Chemistry, Life Sciences, Sociology, Economics). We also identified 9 external speakers who held senior positions in academic publishing (Elsevier, Nature; Springer Nature) and/or had extensive experience and recognition in peer review process, but also had been involved in initiatives of peer review which touched upon such issues as: biases, sex and gender or diversity (speakers from the European Association of Science Editors (EASE) and gender studies’ academics).
See overview of the programme: https://warwick.ac.uk/fac/soc/pais/research/projects/plotina/summerschool/programme/
We circulated the invitation for this action through various newsletters (university-wide), including the Learning and Development Centre (LDC) and the Doctoral Training Centres at the University of Warwick. We contacted departmental administrators across various faculties and departments to promote the event to early career researchers. We also disseminated the call for participation to other universities within reasonable travelling distances, including the University of Birmingham, the University of Oxford, the University of Southampton, and Aston University, though we stopped advertising when the event was full.
- Participants may not think the discussion about biases in knowledge production and sex/gender considerations is relevant to them or this topic.
- Getting highly esteemed speakers to contribute to this programme (they have very busy schedules).
- People signed up to attend and then did not bother to come.
- Be clear about the benefits of attending such an event and ensure that you get and respond to feedback from participants beforehand and during the programme.
- Start planning well-ahead of the event and motivate potential speakers by the benefit to early career researchers.
- Charging a small registration fee can encourage commitment and/or discourage casual sign-ups
Tips/strategies – Lessons learnt
Think about capacity and how many participants you can accommodate. While the talks can be open to a larger number of participants, when it comes to practical workshops, it is more difficult to run them with more than 35 participants (considering that they often do group work, 7 groups of 5 people or 5 groups of 7 people can work well). We had an overwhelming response in registration (90 tried to register, we allowed for 70 and we had 20 people in the waiting list). However, there was a big dropout rate so we recommend allowing for 1015% over-subscribing to mitigate this. For this reason, you might also want to consider adding the following statement in the registration form for running such a free event ‘By registering you are holding a place at this event. You need to notify the organisers as early as possible if you cannot attend so your place can be offered to another participant. Failure to do so might lead to a charge of x amount’. A small registration fee (e.g. to cover food costs) might be the best way to gain commitment.
Due to time constraints and hectic schedules, events like this could be run in 1.5 days tailored either to specific disciplines or broader disciplinary areas. Participants appreciated disciplinary expertise (at least broad disciplinary expertise) and would have valued a more ‘compact’ schedule to get the most of it. However, the half day pilot programme was a bit too compressed and participants did not get to try it out.
Develop the content of the programme and identify speakers 4-5 months before the day of the event.
Check whether you can capture (audio/video) the talks and get consent (from speakers and participants) to develop a resource that could be accessible to the wider community.
Budget for: food/coffee, travel/accommodation expenses for speakers, printing and folders, administrative assistance.
More detailed Outcomes/Impact
The summer school was well received. We were overwhelmed with interest for the event (70 individuals registered) and we had to put participants on a waiting list (+20). The delegates who filled out the evaluation form (29 filled in the form) gave positive feedback. Among the topics they found particularly interesting and beneficial were the following: talks and discussion on how to conduct peer reviews, discussion about gender imbalance in the research context, and the practical sessions. Participants also mentioned that they benefited from the speakers’ variety of disciplines and backgrounds and commented positively on how well prepared the speakers were.
Participants highlighted that the event was very comprehensive and it covered everything about peer review. Some of them would have liked to participate in more practical sessions and hands on exercises e.g. review longer articles or real articles submitted to the Warwick journal.
An evaluation form was designed on the summer school’s website (an evaluation form template is available and it can be used to develop an online form print and disseminate it at the end of the event). Participants were informed verbally about it at the end of the summer school and via email. They also received a reminder email a week later. The form was simple and brief including open ended questions to encourage participants to fill the form and expand on their feedback. The form was comprised of:
- Which topics or aspects of the summer school did you find most interesting or useful?
- What else do you feel should have been included?
- Do you have any further comments/ suggestions?
Reflection, what we would do differently
It was challenging to run the practical workshops with high numbers of participants. If run again we would split the participants in groups for the practical workshops.
The summer school was offered for free and there was a big dropout rate so we recommend allowing for 10-15% over-subscribing to mitigate this and charging a small registration fee.