.jpg)
.jpg)
This document provides detailed guidelines to Area Chairs (AC) for MICCAI 2020. Please read the instructions carefully if this is your first time as AC for MICCAI, and just as carefully if you have been on the program committee before, as there are some changes in the review process compared to previous MICCAIs. Specifically, we adopted the AC model, in which each AC has specific expertise and more responsibilities for the papers assigned to him/her. Being an AC is a demanding task that involves managing the review process for a set of papers, selecting papers for orals and awards, and interacting with the Organizing Committee (OC) and other ACs.
Having successfully moved to a double-blinded peer-review process at MICCAI 2017, there will be no face-to-face Program Committee meeting for MICCAI 2020. Instead there will be a teleconference before the review process begins on March 12th to explain the process and to answer any questions related to the review process, and a second teleconference near the end of the process, where the MICCAI 2020 OC will communicate the overall results and statistics of the review process and any arising issues to the ACs, while at the same time collecting feedback from the ACs.
The role of AC therefore continues to entail a significant commitment.
Important Dates for Area Chairs is available from this page.
ACs are expected to play an active role in ensuring high quality, timely, fair and constructive reviews. ACs also help the OC select the best papers to be presented at MICCAI 2020. The ACs are involved in the following steps of the review process
Authors will submit their papers using Microsoft's Conference Managing Toolkit (CMT). The MICCAI Conference review process will be double-blind, i.e. the ACs and reviewers won't know the authors, and the authors won't know the reviewers or ACs. Paper assignment to ACs will be completed automatically using CMT and the Toronto Paper Matching System (TPMS), which has been successfully used for past MICCAIs. The system is complex, considering paper matching scores, subject areas, conflicts and global load balancing - but remember, it is automatic and not necessarily optimal for each individual AC. If you feel a paper has been incorrectly assigned to you, please let program-chairs@miccai2020.org know; we will re-assign the paper to another AC and you will get a replacement paper that has been manually selected. We anticipate that such reassignments will be rare.
Note that MICCAI 2020 will be running double-blind - neither reviewers nor ACs will see the author list, nor will the authors see the reviewers or ACs. Reviewers will not see the ACs names. The CMT system will provide a list of reviewers for each paper to the AC. This ordered list is generated based on subject areas (relevance) and TPMS. Based on their expertise and judgment, the AC will select at least 9 and a maximum of 15 reviewers; this is needed to ensure that the bidding and matching system works smoothly. You can filter the list by subject areas and sort by relevance according to keyword matching [score 0-1.0, 1.0=best match] or TPMS rank [1.512,1=best match] to help you choose reviewers. Select your reviewers and save for each paper.
While possible, it is difficult to add a reviewer who is currently not in the reviewer database because we cannot automatically identify conflicts and maintain load balancing through CMT and TPMS. If you wish to add a reviewer not in the database, you must (1) ensure that the person is willing to review, and then (2) contact submission platform manager Kitty Wong (submission_support@miccai2020.org ) and we will include the reviewer manually within CMT for that paper during Phase 2. Given the overhead involved in this process, it is expected that it will be a relatively infrequent exception.
Once all ACs have selected the initial set of reviewers, a reviewer paper bidding process will start, after which the CMT system will re-optimize matching of reviewers to papers based on a large weighting placed on the AC's suggested ranked list of reviewers and the reviewers ranking of paper preferences. This final optimization will consider the TPMS score, relevance, and conflict domains from CMT while performing load balancing across all papers, reviewers and ACs. The highest scoring three reviewers on the revised list will be sent the paper for review automatically.
Each AC will be responsible for shepherding the review process for a maximum of 25 papers. The main duty of the AC duties during this phase is to monitor the quality of the reviews. If reviews are too brief, lack quality, or are inappropriate in some way then it is the ACs responsibility to contact the reviewer and ask them for improvements. Should the AC be unsatisfied with the quality of a review, and fail to get further feedback from the reviewer, then the AC will be able to ask (program-chairs@miccai2020.org) for additional reviewer(s)' input on the paper, beyond the original three reviewers. Any reviewer who does not provide a quality review will be identified in the reviewer database.
The submission platform manager will be responsible for reminding reviewers about deadlines and for monitoring whether reviewers have logged into the system. If 3 reviews have not been provided by the required date then the OCs will work in collaboration with the ACs to secure additional reviews.
After April 22nd, ACs will need to provide meta- reviews which summarize the key strengths and weaknesses of the paper identified by reviewers. For papers that will be sent for rebuttal, you should identify any questions/points that need to be specifically addressed. Please remind authors that the purpose of the rebuttal is to provide clarification or to point out misunderstandings, it is not to promise additional experiments!
ACs should also select appropriate keywords and categories for each paper; this information will be used to help in planning poster and oral sessions.
ACs will then score the paper as an early accept/ rebuttal / early reject. The OC will send out guidance on the proportion of papers to place in each category once the total number of papers submitted is known. We will also provide some aggregate information on the distribution of reviewer scores as each AC may not have a representative sample. Although we do not want to increase the number of rebuttal papers too much, ACs are able to exercise discretion when making their decision. If they feel that the reviewers have scored a paper too highly it can be moved from provisional accept to rebuttal; similarly, if ACs feel the paper has received inappropriately low scores then it can be moved from reject to rebuttal. If ACs feel that it is necessary to do this, they should provide supporting information in their meta-review.
During this period, the OCs may contact the ACs for suggestions of suitable reviewers for papers that are missing 3 reviews. The assignment of additional reviewers will be made by the submission platform manager as it will be necessary to check for conflicts first.
The rebuttal process provides a way for authors to correct possible misinterpretations in the reviewers' findings and inform the AC's recommendation for scoring. Once the reviews of the papers have been sent to the authors, authors have one week to submit their rebuttal. Important: Only the ACs will read the rebuttal.
During this period the OC will assign an additional 2 ACs to each paper sent for rebuttal.
Rebuttals and all reviews will be visible to the primary AC and 2 secondary ACs. The primary AC will review their meta review in light of the rebuttal and the 2 new ACs will also provide a meta review if they wish to provide additional feedback to authors. All 3 ACs will then make a binary accept/reject decision and this will be used by the OC to make the final decisions. The OC will provide information about the proportion of papers to be accepted or rejected but it is anticipated that there will be some variation between individual ACs.
We invite and strongly encourage all ACs to participate a teleconference, tentatively scheduled for June 16th, where the MICCAI 2020 OC will communicate the overall results and statistics of the review process and present any arising issues to the ACs. In these teleconferences, ACs also will be asked to provide feedback on the overall review process and formulate recommendations on any need for adjustments or improvements. Specific details about the program and responsibilities will be announced prior these teleconferences.
This paragraph should remind all ACs of best practices[1] of being an AC.
[1]Acknowledgements: These guidelines were inspired by CVPR 2014 guidelines: http://www.pamitc.org/cvpr14/ac_guidelines.php