Skip to yearly menu bar Skip to main content


NeurIPS 2021 Guidance for Workshop Proposals

(Document updated: )

With the rapid growth and interest in NeurIPS and its associated workshops, the competition for workshops has intensified. In order to attempt to mitigate confusion and anxiety regarding what is expected, the workshop chairs have agreed on the following guidance for proposals to hold a NeurIPS workshop in 2021. Organizers of workshop proposals should take care to respect every piece of guidance provided here and provide explicit answers to the questions implied throughout and explicitly address the selection criteria listed below.

Timeline (Dates Updated June 25)

  1. Workshop Application Open: May 28, 2021

  2. Workshop Application Deadline:  Jun 22, 2021, AoE

  3. Workshop Acceptance Notification:  Jul 21, 2021, AoE

  4. Suggested Submission Date for Workshop Contributions: Sep 17, 2021, AoE

  5. Mandatory Accept/Reject Notification Date:  Oct 21, 2021, AoE

  6. Mandatory SlidesLive upload for speaker videos:  Nov 01, 2021 AOE

Note that the final submission date for workshop contributions is suggested. There is a trade-off between how much time workshops give authors to submit versus reviewers to review in the period between Jul 21, 2021 and Oct 21, 2021.

Most workshops will be one-day events. Workshops may request to be two-day events, however, only a few two-day workshops will be accepted, and other two-day applications may be asked to cut down their programs to one day.

NeurIPS 2021 workshops will primarily be virtual events. A small subset of workshops may be selected for hybrid or in-person events -- all of whom must indicate this preference in the proposal and include a plan for hosting and logistics of the hybrid or in-person events. Note that the in-person plan will be reviewed separately and does not affect the proposal acceptance.

The global author notification deadline is Oct 21, 2021, and the new dates reflect the expected requirements of the virtual conference. Workshops that do not meet this deadline will have their speaker tickets withheld.

Workshop Goals

  1. Workshops provide an informal, cutting edge venue for discussion of work in progress and future directions. Good workshops have helped to crystallize common problems, explicitly contrast competing frameworks, and clarify essential questions for a subfield or application area.

  2. Workshops are a structured means of bringing together people with common interests to form communities. Good workshops will bring together speakers from outside and inside the NeurIPS community, to incorporate different perspectives and build community.

 

Selection Criteria

  1. The degree to which the proposal is focused on an important and topical problem, and the degree to which it is expected that the community will find the workshop interesting, exciting, and valuable.

  2. Diversity and inclusion, in all forms. (See expectations below.)

  3. The degree to which the proposed program offers an opportunity for discussion.

  4. Quality of proposed invited speakers. Workshop organizers are encouraged to confirm tentative interest from proposed invited speakers and mention this in their proposal.

  5. Organizational experience and ability of the team.

  6. Other dimensions in the expectations below not explicitly listed in these criteria.

  7. Points of difference. What makes this workshop enticingly different from the hundreds of NeurIPS workshops held previously?

  8. Details of logistics for holding the workshop online - audience engagement, etc

  9. *Only* if the workshop organizers are interested in having an in-person meeting or a hybrid of in-person and online: propose what aspects would be held in person, the location(s), discuss feasibility from the public health perspective, and logistics (including what support you’d need from the NeurIPS foundation). The NeurIPS foundation and Workshop Chairs may support some of these initiatives if it is safe to do so. At most one workshop per day of those requesting to be in person or hybrid will be allowed such format. By default otherwise, all (other) workshops will be held virtually.

Workshop proposal format

No more than two pages of proposal, no more than two pages of organizer information, (optional) discussion about how the workshop could be held in hybrid format, and unlimited references

The two pages (or fewer) for the main proposal must include:

  • A title and a brief description of the workshop topic and content.

  • A list of invited speakers, if applicable, with an indication of which ones have already agreed and which are indicative.

  • An account of the efforts made to ensure diversity of the organizers and speakers (WiML, Black in AI, and LXAI directories, among others, may be a useful resource). Also an account of any efforts to include diverse participants (e.g., via mentoring, subsidies, or the wording and topics in the CFP).

  • An estimate of the number of attendees.

  • A description of special requirements and technical needs.

  • [Optional] If the workshop is requesting a two-day event, a discussion of why a two-day event is preferred.

  • If the workshop has been held before, a note specifying how many submissions the workshop received, how many papers were accepted (extended abstract/long format), and how many attendees the workshop attracted.

  • A very brief advertisement or tagline for the workshop, up to 140 characters, that highlights any key information you wish prospective attendees to know, and which would be suitable to be put onto a web-based survey (see below).

  • A URL for the workshop website.

The two pages (or fewer) for information about organizers must include:

  • The names, affiliations, and email addresses of the organizers, with one-paragraph statements of their research interests, areas of expertise, and experience in organizing workshops and related events.

  • A list of Programme Committee members, with an indication of which members have already agreed. Organizers should do their best to estimate the number of submissions (especially for recurring workshops) in order to (a) ensure a sufficient number of reviewers so that each paper receives 3 reviews, and (b) anticipate that no one is committed to reviewing more than 3 papers. This practice is likely to ensure on-time and more thorough and thoughtful reviews.

Hybrid Plan (Optional; one-page max): Workshop organizers who are interested in having an in-person meeting or some hybrid of in-person and online format may propose what aspects would be held in person, the location(s), discuss feasibility from the public health perspective, and logistics (including what support you’d need from the NeurIPS foundation). The NeurIPS foundation and Workshop Chairs may support some of these initiatives if it is safe to do so. By default otherwise, all workshops will be held virtually.

 

Assessment Process and Criteria

The workshop chairs will appoint a number of reviewers who will provide written assessments of the proposals against the criteria listed above. Their reports will be considered by the workshop chairs who will jointly decide upon the selected workshops (subject to the notes on COIs listed below). The final decisions will be made by the workshop chairs via consensus and judgment; we will not simply add up scores assigned to all the criteria.

Hard Constraints/Workshop Requirements

Global Notification Deadline Prior to Oct 21, 2021: By submitting a workshop proposal, workshop organizers commit to notifying those who submit contributions (including talks and posters) to the workshop of their acceptance status before Oct 21, 2021. A timeline should be included in the proposal that will allow for this. This deadline of Oct 21, 2021 will be published on the NeurIPS main web page and cannot be extended under any circumstances.

Managing Chair and Reviewer Conflicts of Interest

  1. Workshop chairs cannot be organizers or give invited talks at any workshop but can submit papers and give contributed talks.
  2. Workshop reviewers cannot review any proposal on which they are listed as an organizer or invited speaker, or which they have a conflict of interest with as defined by the NeurIPS guidelines, and may not accept invitations to speak at any workshop they have reviewed after the workshop is accepted.
  3. Workshop chairs and reviewers cannot review or shape acceptance decisions about workshops with organizers from within their organization. (For large corporations, this means anyone in the corporation worldwide).

Managing Organizer Conflicts of Interest

  1. Workshop organizers cannot give talks at the workshops they organize. They can give a brief introduction to the workshop and/or act as a panel moderator.

  2. Workshop organizers should state in their proposal how they will manage conflicts of interest in assessing submitted contributions. At a minimum, an organizer should not be involved in the assessment of a submission from someone within the same organization.

Other Guidance and Expectations for Workshop Proposals

  1. We encourage, and expect, diversity in the organizing team and speakers. This includes the diversity of viewpoints and thinking regarding the topics discussed at the workshop, gender, race, geography, affiliations, seniority, etc. If a workshop is part of a series, the organizer list should include people who have not organized in the past. Organizers should articulate how they have addressed diversity in their proposal in each of these senses.

  2. Since the goal of the workshop is to generate discussion, sufficient time and structure need to be included in the program for this. Proposals should explicitly articulate how they will encourage broad discussion.

  3. Workshop proposals should list explicitly what the problems are they would like to see solved or at least advances made, as part of their workshop. They should explain why these are important problems and how the holding of their proposed workshop will contribute to their solution.

  4. Workshops are not a venue for work that has been previously published in other conferences on machine learning or related fields. Work that is presented at the main NeurIPS conference should not appear in a workshop, including as part of an invited talk. Organizers should make this clear in their calls and explain in their proposal how they will discourage the presentation of already finalized machine learning work.

  5. We encourage workshop submissions of varying lengths and scopes. Organizers should state whether their workshops are meant to be large-attendance talk format or small group presentations. Organizers should articulate what they hope to achieve from the format proposal beyond the talks listed.

  6. Workshops should allow for the choice of attendance based on content. Good workshops will put talk titles up publicly prior to site publication and note the archival status of their submissions. Organizers should articulate how they will do this.

  7. Organizing a workshop is a complex task, and proposals should outline the organizational experience and skills of the proposed organizers (as a team). We encourage junior researchers to be involved in workshop organization but prefer some collective experience in organizing a complex event.

Frequently Asked Questions From Past Workshops

  1. Workshop Series
    We neither encourage nor discourage workshops on topics that have appeared before. Membership of an existing sequence of workshops is irrelevant in the assessment of a workshop proposal (it neither helps nor hinders). Workshop proposals will be evaluated solely on their merits for this year’s conference.

  2. Overlapping Proposals
    We will not forcibly merge proposals. If multiple strong proposals are submitted on similar topics, we will choose a single proposal to accept. We will then reach out to the organizers of the rejected proposals to ask whether they would like us to share their proposals with the organizers of the accepted workshop. The organizers of the accepted workshop may then optionally initiate a merge.

Common Pitfalls From Past Workshops (see 2019 workshop summary for more discussion)

  1. Leaning too heavily on past success
    Proposals for workshops that are part of a series sometimes leaned too heavily on the declared popularity of previous workshops. In some cases, this led to proposals that were less creative and innovative than what we had hoped to see.
  2. Unconfirmed or irrelevant speakers
    The vast majority of proposals included lists of confirmed invited speakers. This made it hard to champion any workshop that didn’t have at least a few speakers confirmed, especially when many unconfirmed big-name speakers were listed (it’s unlikely all would say yes), or when the diversity statement centered on the assumed presence of unconfirmed speakers. There were also several proposals featuring long lists of “celebrity” speakers without clear relevance to the topic of the workshop.
  3. Insufficient time for discussion
    Too many invited speakers—some proposals listed a dozen or more—does not make for a great audience experience, and a workshop with nothing but long-form talks is unlikely to lead to new breakthroughs. We encourage organizers to allocate a larger amount of time to contributed content and open discussion.
  4. Going too big
    We saw only a few proposals that we felt were too narrow, but many we found too broad. There seems to be a tendency to overreach for the sake of going big, while we’d prefer to see more focused workshops.
  5. Too many organizers
    Several proposals had remarkably large organizing committees. It’s not clear why more than five or six organizers would be necessary for a workshop, and it raises concerns about name dropping or organizers added just for an appearance of diversity.
  6. Diversity lip service
    While we were pleased overall by the effort that organizers put into diversity, a lack of diversity in a proposal could be fatal. We were particularly wary of proposals that claimed to be big on diversity while having a full lineup of North American white male speakers or a list of organizers who all recently graduated from the same institution.
 

2021 Workshop Chairs

Anna Goldenberg, University of Toronto & SickKids Research Institute
Ndapa Nakashole, University of California - San Diego
Sanmi Koyejo, University of Illinois at Urbana-Champaign & Google Research
Tristan Naumann, Microsoft Research

neurips2021workshopchairs@googlegroups.com
Technical or website questions about workshops go to lee@salk.edu.