Skip to yearly menu bar Skip to main content


Area Chair (AC) Guidelines

Thank you for agreeing to serve on the program committee of NeurIPS 2023! As an area chair (AC), your job is to ensure that all the submissions you are assigned have high quality reviews and good discussions. You should become familiar with the technical contents of all your submissions and are responsible for making the initial acceptance decisions, with guidance from your senior area chair (SAC).

If you encounter a situation that you are unable to resolve on your own:

  • If it is an issue involving OpenReview, please contact info@openreview.net.
  • If it is a general question, please contact the program chairs at neurips2023pcs@gmail.com.
  • If you have specific questions related to the handling of a particular paper, please leave a comment on the OpenReview forum for that paper with the readers set to SACs and PCs.

Frequently Asked Questions

You can find the AC FAQ at the bottom of this page.

Important Dates

Dates may be subject to change.

  • Reviewers bid on papers: Sat, May 20 – Fri, May 26
  • Check paper assignments: Wed, June 14
  • First round reviewing: Thu, June 8 – Thu, July 6 
  • Invite emergency reviewers Fri, July 7 – Mon, July 10
  • Second round reviewing: Wed, July 12 - Wed, July 26
  • Reviews released to authors: Wed, Aug 2
  • Author Rebuttal: Wed, Aug 2 - Wed, Aug 9
  • Reviewer-Author Discussions: Thu, Aug 10 – Mon, Aug 21
  • Reviewer-AC Discussions: Thu, Aug 10  - Fri, Aug 25 
  • Metareviews due:  Thu, Aug 31
  • SAC-PC Discussions: Fri, Sep 8 – Thu, Sep 14
  • Author Notifications: Fri, Sept 22

Main Tasks

  1. Preparation:
    • Please ensure that your preferred email address is accurate in your OpenReview profile. We will send most emails from OpenReview (noreply@openreview.net). Such emails are sometimes accidentally marked as spam (and sometimes as updates in Gmail). Please whitelist the OpenReview email address so that you will receive future emails from OpenReview.
    • Please log into OpenReview and make sure that your profile is up to date. Import your recent papers using Profile->Edit Profile->Add DBLP papers to profile.
    • Read and agree to abide by the NeurIPS code of conduct.
    • Read what constitutes conflict of interest for NeurIPS.
    • In addition to the guidelines below, please familiarize yourself with the reviewer responsibilities and guidelines. You will be interacting significantly with reviewers, so please make sure you understand what is expected of them. You may also refer to the SAC guidelines.
  2. ✅ Bid on papers: Sat, May 20 – Fri, May 26
    • Both ACs and reviewers will bid on papers. We will then assign papers to ACs using a combination of bids and a matching algorithm based on OpenReview profiles. 
    • While it is possible that this may result in less than optimal assignments, we rely on your experience and insight in handling papers from a variety of topics rather than on your specific topic expertise. Please bear in mind that we have a large number of submissions, so your flexibility is greatly appreciated. 
  3. ✅ Adjust reviewer assignments: Wed, June 1 – Tue, June 7
    • Make sure that every submission in your batch is matched with 4 suitable reviewers whom you can trust on this submission, ideally with a diverse set of opinions.
    • In case of major mis-assignments, please edit the reviewers keeping the total to 4. You can continue inviting reviewers as needed after the assignment is released to reviewers. Note that because the review process is already underway, reviewers will need to agree to review each additional paper before they can be assigned.  You can follow these instructions for sending invitations to review a specific paper.
    • It really pays off to invest some time before the reviewing process starts to ensure that your batch has expert reviewers!
  4. ✅ Invite emergency reviewers: Fri, July 7 – Mon, July 10
    • Reviews are due July 6. You might have to send multiple reminder emails, but please be courteous. If a reviewer is unable to deliver a review, please find a replacement (emergency) reviewer by Mon, July 10.
  5. ✅ Review assessment and emergency reviewing period: Mon, July 10 – Mon, July 24
    • Read all reviews carefully and fill out a “review check” form in OpenReview to flag whether the paper needs more reviews (borderline, poor quality, or missing reviews), or if reviewer collusion is suspected (e.g., high scores do not match the quality of the paper.)
    • If a review is substandard, you should ask the reviewer to improve it. Please remember to be polite and provide concrete guidance.
    • You may also need to get an additional review if the current reviews are particularly unhelpful, either in the existing pool or by inviting external reviewers.
    • Make sure that any questionable papers are flagged for ethics review. These papers will be assigned ethics reviewers, who will effectively join the paper's assigned program committee. Their reviews will be released to the authors, who will have an opportunity to respond to these reviews.
  6. ✅ Discuss with reviewers and authors: Aug 10 – Aug 25 
    • The discussion period is separated into several phases. Please lead the discussions and encourage the reviewers to engage actively from the first phase. You can leave messages on OpenReview with the appropriate visibility settings (authors, reviewers, senior area chair, or program chairs).
    • Author-Reviewer Discussions (Aug 10 – Aug 21): During this first phase, reviewers are asked to read all other reviews and the author responses and engage in an open exchange with the authors. Please ping the reviewers, as this is a short one-week period.
    • Make sure your reviewers read and respond to all author responses. Even though the discussion period is two weeks, only this first half is with the authors, so it is important that any additional questions to the authors following the author rebuttal are asked during the first week.
    • Reviewer-AC Discussions (Aug 16 – Aug 25): During this second phase, reviewers discuss among themselves with ACs' moderation. Monitor and moderate the discussion to ensure that it is respectful of everyone’s opinion. Please steer the discussions to the important disagreements among the reviewers and try to establish common understanding of the main contributions and limitations based on all reviews, author rebuttal, and author-reviewer discussions. 
  7. ✅ Discuss with SACs and write the meta reviews: Thu, Aug 24 – Thu, Aug 31
    • This is considered the last phase of the discussions, and you will be writing the meta reviews based on your reading of the paper, the reviews, and the two previous phases of discussions. You can elicit further comments and clarifications from the reviewers. Please reach out to your SAC if you need help. Please help us by being on time with your meta reviews, as we have only a couple of weeks before the notifications are sent to the authors on Sep 22. 
    • Write a meta-review that explains your recommendation (accept, reject) to the authors. Your comments should augment the reviews, and explain how the reviews, the author response, and the discussions were used to arrive at your recommendation. Do not dismiss or ignore a review unless you have a good reason for doing so. If the reviewers cannot come to a consensus, you should read the paper carefully and write a detailed meta-review. You are expected to discuss such difficult cases with your SAC.
  8. ✅ Make accept/reject recommendations with SACs: Fri, Sep 1 – Thu, Sep 7
    • SACs will read your meta reviews. Discuss your accept/reject recommendations with your SAC, especially the borderline cases as well as ones where you are making a recommendation that goes against the reviewers. The SAC will help you calibrate across other ACs.
  9. This year we have an optional rating of the reviews. This will happen after the notifications, and we will contact you separately to opt-in. Please participate in this extra task to help improve the NeurIPS peer reviewing process.
  10. Author Notification: Friday, September 22.

Best Practices

  • Please respect deadlines and respond to emails as promptly as possible.
  • It is okay to be unavailable for part of the review process (e.g., on vacation for a few days), but if you will be unavailable for more than that—especially during important windows (e.g., discussion, decision-making)—you must let your SAC know as soon as you can.
  • If you notice a conflict of interest with a submission that is assigned to you, please contact your SAC immediately so that the paper can be reassigned.
  • Be professional and listen to the reviewers, but do not give in to undue influence. We expect you to be familiar with all the papers that are assigned to you and to be able to argue about their technical content and contributions. Your responsibility is to make good decisions, not just facilitate reviewer discussions.
  • Be kind. It is important to acknowledge that personal situations may lead to late or unfinished work among reviewers. In the event that a reviewer is unable to complete their work on time, we encourage you to be considerate of the personal circumstances; you might have to pick up the slack in some cases. In all communications, exhibit empathy and understanding.
  • DO NOT talk to other ACs about submissions that are assigned to you; other ACs may have conflicts with these submissions. If you feel that it’s important to discuss one submission in the context of another, please email program-chairs@neurips.cc. In general, your primary point of contact for any discussions should be your SAC. Your SAC does not have any conflicts with any of the submissions that are assigned to you.
  • DO NOT talk to other SACs or ACs about your own submissions (i.e., submissions you are an author on) or submissions with which you have a conflict of interest.
  • If you notice unethical or suspect behavior involving either authors or reviewers, please notify your SAC and program chairs.

Writing the meta-review

These guidelines are due to Chris Williams and John Lafferty and lightly adapted.

  • Don't focus too much on the scores. Instead, look carefully at the comments. Judge the quality of the review rather than taking note of the reviewer's confidence score; the latter may be more a measure of personality.
  • Indicate that you have read the author response, even if you just say "the rebuttal did not overcome the reviewer's objections."
  • If you use information that is not in the reviews (e.g., from corresponding with one of the reviewers after the rebuttal period), tell the authors (a) that you have done so and (b) what that information is.
  • If you find yourself wanting to overrule a unanimous opinion of the referees, the standards for your summary should be at the level of a full review. In these cases, it would probably be best to solicit an auxiliary review.
  • Please attempt to take a decisive stand on borderline papers. Other than papers where there is a genuine disagreement, much of our work will involve borderline papers where no one confidently expresses excitement, nor are any major problems identified. These are the tough decisions where we need your judgment!
  • Try to counter biases you perceive in the reviews. Unfashionable subjects should be treated fairly but often aren't, to the advantage of the papers on more mainstream approaches. To help the NeurIPS community move faster out of local minima, it is important to encourage risk and recognize that new approaches can't initially yield state-of-the-art competitive results. Nor are they always sold according to the recipes we are used to.

Confidentiality

You must keep everything relating to the review process confidential. Do not use ideas, code and results from submissions in your own work until they become publicly available (e.g., via a technical report or a published paper for ideas/results, via open source for code, etc.). Do not talk about or distribute submissions (whether it is the code, or the ideas and results described in them) to anyone without prior approval from the program chairs. Code submitted for reviewing cannot be distributed.

Frequently Asked Questions

Q: How do I contact the Senior Area Chair (SAC) and/or the Program Chairs?

A:  You can communicate with your SAC via OpenReview by writing a comment with visibility limited to your SAC on the discussion page of the paper in question.  The comment will be visible to the SAC and Program Chairs by default, but it will not trigger a notification. For more general questions to PCs, please email the list.

 

Q: Is it ok if a paper has the appendix as part of the main PDF?

A: No, the appendix should be part of the supplemental material. 

 

Q: What if the paper does not have the paper checklist?

A: This year the checklist is submitted via the OpenReview submission form, so it is not part of the paper PDF.

 

Q: What is the page limit?

A: The main part of the paper should be 9 pages long. After that, only the references are allowed. If that is not the case, please mark it as a format violation.

 

Q: What else counts as format violations?

A: Anything that breaks anonymity, changing the format, such as margins or vertical spaces. If in doubt, please mark it as a violation, and we will take a look.