Skip to yearly menu bar Skip to main content


AC & SAC Guidelines

Frequently asked questions

Frequently asked questions can be found here.

Contacting the program chairs

If you encounter a situation that you are unable to resolve on your own, please contact the program chairs. Please refrain from writing to the program chairs at their own email addresses.

Introduction

Thank you for agreeing to serve for NeurIPS 2019! The community needs outstanding people like yourself to make NeurIPS a success, and we will work hard to make your duties as easy as possible. This page provides an overview of SAC and AC responsibilities and key dates.

In addition to the guidelines below, please familiarize yourself with the reviewer responsibilities and guidelines.  You will be interacting significantly with reviewers, and the quality and timeliness of their feedback is critical to your role, so please make sure you understand what is expected of them and how this has been communicated.

Key dates

  • SACs, ACs & reviewers enter domain conflicts, subject areas, TPMS information, etc.

    • April 1 -- April 15

  • SACs bid on ACs

    • 1 week: Mon April 15 -- Sun April 22

  • PCs assign ACs to SACs

    • 1 week: Mon April 23 -- Sun April 30

  • Abstract Submission deadline

    • Thu May 16 (4pm ET; 8pm UTC)

  • Paper Submission deadline

    • Thu May 23 (4pm ET; 8pm UTC)

  • ACs enter bids on submissions

    • 1 week: Fri May 17--Tues May 28

  • ACs/SACs enter individual conflicts

    • 1 week: Fri May 24--Tues May 28

  • PCs clean submissions, test for duplicates, run TPMS, assign ACs to papers

    • 1 week: Fri May 24--Thu May 30

  • Reviewers bid on submissions
    ACs bid on reviewers for submissions
    SACs micro-adjust AC assignments if needed (e.g. conflicts, bad assignment)

    • 1 week: Fri May 31--Thu Jun 6

  • PCs assign submissions to reviewers

    • 3 days: Fri Jun 7--Mon Jun 10

  • SACs & ACs micro-adjust reviewer assignments

    • 1 week: Tues Jun 11--Mon Jun 17

  • Reviewers write reviews

    • 4 weeks: Tues Jun 18--Mon Jul 15

  • SACs & ACs chase late reviews, add reviewers

    • 9 days: Thu Jul 16--Wed Jul 24

  • Authors respond to reviews

    • 1 week: Thu Jul 25--Wed Jul 31

  • ACs & reviewers discuss reviews & responses

    • 2 weeks: Thu Aug 1--Wed Aug 14

  • ACs make initial decisions with SACs (ACs write meta-review, SACs available to focus on borderline cases)

    • 1 week: Thu Aug 8--Wed Aug 14

  • ACs write metareviews (SACs focus on borderline cases)

    • 1 week: Thu Aug 15--Wed Aug 21

  • PCs & SACs finalize decisions (ACs & reviewers involved as necessary)

    • 1 week: Thu Aug 22--Wed Aug 28

  • Notification date

    • Wed Sep 4

General information

  • Please respect deadlines and respond to emails as promptly as possible!

  • It is crucial that we are able to reach you in a timely manner. We will send most emails from CMT (i.e., email@msr-cmt.org). Such emails are sometimes accidentally marked as spam. Please check your spam folder regularly and if you find such an email in there, please whitelist the CMT email address so that you will receive future emails from CMT.

  • If you have changed or plan to change your email address, please update CMT accordingly. We have no way of knowing whether an email sent to you from CMT has bounced, so it is crucial that you make sure that CMT has the correct email address for you at all times. You should also make sure that your domain conflicts in CMT are up to date; these are important for preventing conflicts during the review process.

  • The NeurIPS definitions of conflicts of interest (and instructions for entering them) have changed a little from last year, so please make sure you read this year’s definitions.

  • NeurIPS uses the Toronto Paper Matching System (TPMS) to assign submissions to ACs and reviewers. Please log into TPMS here and make sure that your profile is up to date.

  • All participants must agree to abide by the NeurIPS code of conduct.

Responsibilities

  • Senior Area Chair. With the growth in submissions, NeurIPS recently incorporated the role of senior area chair (SAC). SACs work alongside the area chairs (ACs) and program chairs (PCs). Each SAC oversees the work of a small number of ACs (around 7), making sure that the reviewing process goes smoothly. SACs serve as the first port of call for ACs if they need assistance or guidance. The reviewing process is double blind at the level of ACs (i.e., ACs cannot see author identities), but not at the level of SACs and program chairs; SACs are therefore responsible for identifying conflicts of interest (and other unusual activity, such as suspicious bidding patterns) and re-assigning submissions to ACs or reviewers accordingly. SACs are also responsible for helping ACs chase late reviewers, calibrating decisions across ACs, and discussing borderline papers. During the final decision-making phase, SACs will discuss all proposed decisions with the PCs. There is no physical SAC/AC meeting; most discussions with reviewers, ACs, and PCs will take place via CMT or email, with some video conferences toward the end of the reviewing process. Reviewer identities are visible to other reviewers, ACs and SACs. After decisions have been made, reviews and meta-reviews will be made public (but reviewer and SAC/AC identities will remain anonymous).

  • Area Chair. Each area chair (AC) oversees around 20 submissions, making sure that the reviewing process goes smoothly. ACs are the principal contact for reviewers during the whole reviewing process. ACs are responsible for helping the program chairs (PCs) recruit reviewers, recommending reviewers for submissions, chasing late reviewers, facilitating discussions among reviewers, writing meta-reviews, evaluating the quality of reviews, and helping make decisions. The reviewing process is double blind at the level of ACs; each AC will work with a senior area chair (SAC), who is responsible for identifying conflicts of interest (and other unusual activity, such as suspicious bidding patterns) and re-assigning submissions to ACs or reviewers accordingly. SACs serve as the first port of call for ACs if they need assistance or guidance throughout the reviewing process. SACs also calibrate decisions across ACs. There are no physical meetings; most discussions with reviewers, SACs, and PCs will take place via CMT or email, with some video conferences toward the end of the reviewing process. Reviewer identities are visible to other reviewers, ACs and SACs. AC identities are also visible to reviewers. After decisions have been made, reviews and meta-reviews will be made public (but reviewer and SAC/AC identities will remain anonymous).

  • Reviewer. Each reviewer will be assigned around 4--6 submissions to review. Reviewers are responsible for reviewing submissions, reading author responses, discussing submissions and author responses with other reviewers and area chairs (ACs), and helping make decisions. The reviewing process is double blind at the level of reviewers. There are no physical meetings; discussions with other reviewers and ACs will take place via CMT or email. After decisions have been made, reviews and meta-reviews will be made public (but reviewer and AC identities will remain anonymous). This year, as an incentive, ACs will be asked to evaluate the quality of each review using three scores: “exceeded expectations”, “met expectations,’ and “failed to meet expectations.” The 400 or so highest-scoring reviewers will be awarded free NeurIPS registrations. The next 2000 or so highest-scoring reviewers will have registrations reserved for them (for a limited time frame). The lowest-scoring reviewers may not be invited to review for future conferences.

SAC best practices

  • It is okay to be unavailable for part of the review process (e.g., on vacation for a few days), but if you will be unavailable for more than that -- especially during important windows (e.g., decision-making) -- you must let the program chairs know ASAP.

  • With great power comes great responsibility! Take your job seriously and be fair.

  • If you notice a conflict of interest with a submission that is assigned to one of your ACs, please contact the program chairs immediately. (Note that our definitions have changed a little from last year, so please carefully read this year’s definitions here.)

  • DO NOT talk to other SACs about submissions assigned to your ACs without prior approval from the program chairs; other SACs may have conflicts with these submissions.

  • DO NOT talk to other SACs or ACs about your own submissions (i.e., submissions you are an author on) or submissions with which you have a conflict of interest.

  • Be professional and listen to the reviewers and ACs, but do not give in to undue influence.

  • If an AC wants to make a decision that is not clearly supported by the reviews, please check that they justify their decision appropriately, including, but not limited to, reading the submission in depth and writing a detailed meta-review that explains their decision.

  • Help calibrate decisions by working closely with your ACs. It is your responsibility to figure out how best to work with your ACs during this process (e.g., over email, phone, video conferences, etc). Pay particularly close attention to borderline papers.

AC best practices

  • It is okay to be unavailable for part of the review process (e.g., on vacation for a few days), but if you will be unavailable for more than that -- especially during important windows (e.g., discussion, decision-making) -- you must let your SAC know ASAP.

  • With great power comes great responsibility! Take your job seriously and be fair.

  • If you notice a conflict of interest with a submission that is assigned to you, please contact your SAC immediately so that the paper can be reassigned. (Note that our definitions have changed a little from last year, so please carefully read this year’s definitions here.)

  • Please make sure that every submission in your batch is matched with suitable reviewers whose opinion you can trust on this submission.  It really pays off to invest some time before the reviewing process starts to ensure that your batch has expert reviewers.

  • If two submissions are detected to be sufficiently close by our tools, reviewers will receive both and you will be informed; please ensure that the arguments to discard or accept each submission are thoroughly discussed.

  • DO NOT talk to other ACs about submissions that are assigned to you without prior approval from your SAC; other ACs may have conflicts with these submissions. In general, your primary point of contact for any discussions should be your SAC. Your SAC does not have any conflicts with any of the submissions that are assigned to you.

  • DO NOT talk to other SACs or ACs about your own submissions (i.e., submissions you are an author on) or submissions with which you have a conflict of interest.

  • Be professional and listen to the reviewers, but do not give in to undue influence.

  • Make sure your reviewers read and (if appropriate) respond to all author responses.

  • After the author response phase, you must initiate a discussion via CMT for each submission and make sure the reviewers engage in the discussion phase. Monitor and moderate the discussion to ensure that it is respectful of everyone’s opinion.

  • Read all reviews carefully. After reading a review, please evaluate its quality by indicating (on CMT) whether it “exceeded expectations,” “met expectations,” or “failed to meet expectations.” You should use the information in the reviewer instructions (see the section “Review Content” below), as well as how helpful the review and subsequent discussion by the reviewer were in making your decision about the submission. The 400 or so highest-scoring reviewers will be awarded free NeurIPS registrations. The next 2000 or so highest-scoring reviewers will have registrations reserved for them (for a limited time frame). The lowest-scoring reviewers may not be invited to review for future conferences.

  • Your meta-review should explain your decision to the authors. Your comments should augment the reviews, and explain how the reviews, author response, and discussion were used to arrive at your decision. Dismissing or ignoring a review is not acceptable unless you have a good reason for doing so. If you want to make a decision that is not clearly supported by the reviews, perhaps because the reviewers did not come to a consensus, please justify your decision appropriately, including, but not limited to, reading the submission in depth and writing a detailed meta-review that explains your decision.