Academic Integrity in Research Paper Submissions and Reviews

September 19, 2024

 

This policy is an expansion of existing policies regarding the Code of Conduct and the Code of Ethics in regard to specific matters related to academic integrity throughout the submission and review process of research paper submissions.

The integrity of research paper submissions and the peer review process is vital to NeurIPS and our scientific community. NeurIPS will not tolerate evaluations that are judged based on favoritism, personal loyalties, mutual gain, or any ground other than scientific and academic merit. NeurIPS proposes to allow data sharing across specified research field related “Sister Organizations” (defined below) as one way to identify those who engage in collusion and other research misconduct and give every researcher in our field an equal opportunity to have their ideas heard.

I. Definitions

“Submission(s)” is defined as any paper, manuscript, or other research, including underlying research data, that is submitted to NeurIPS or any Sister Organization for the purposes of publication or presentation at the NeurIPS Conference, or a conference of any Sister Organization.

“Submission and Review Information” is defined as:

1. The title and content of any Submission;
2. The name, employer(s), and educational background of the author(s) of any Submission;
3. The name, employer(s) and educational background of anyone who agrees to review a Submission (i.e., a “Reviewer”);
4. The ratings and comments provided by any Reviewer regarding a Submission; and
5. Statistics regarding the review process, including but not limited to: the number of submissions a particular Reviewer has reviewed, the length of those reviews, the dates those reviews occurred, whether any of those reviews were late and, if so, how long they were late.

"Submitter(s)” is defined as any listed author on a Submission. In addition, if NeurIPS believes that an individual contributed to a Submission but was not listed as an author to avoid detection under this policy, or otherwise engage in scientific fraud or academic dishonesty, NeurIPS reserves sole discretion to treat that un-listed individual as a “Submitter” under this policy.

“Sister Organization(s)” is defined as other related academic organizations that support or engage in research, publication or scientific conferences in our field including, but not limited to: the International Conference on Learning Representations (ICLR), and the International Conference on Machine Learning (ICML). “Sister Organization” may also include any organization that serves as an intermediary between submitters and reviewers, including websites, clearinghouses, and other platforms that accept, store, and transmit submissions, and provide a method to provide comments, scores, and other feedback regarding submissions to submitters, NeurIPS and other Sister Organizations. This includes, but is not limited to: https://OpenReview.net. NeurIPS maintains sole discretion, at any time, to add or remove entities from this definition.

“Misrepresentation” is defined as a false statement of fact or truth in any Submission or any rating or comment provided by a Reviewer.

“Falsification” is defined as intentional misrepresentation of research results, supporting data and materials, or noted references.

“Coercion” is defined as any attempt to intimidate, harm, demean, use force against, or use one’s position or authority to persuade or take advantage of someone for the betterment of themselves.

“Collusion” is defined as any agreement or understanding between two or more Submitter(s), Reviewer(s) or other person(s) to give Submission(s) artificially high or low ratings, or otherwise artificially improve or damage a Submission’s chance of acceptance or publication by NeurIPS or any Sister Organization out of self-interest, personal loyalties, or mutual gain, not the merits of the Submission.

“Share” or “sharing” means to transmit information to a secure server, that cannot be accessed by un-authorized third-parties, in such a way that the authorized party can see, read, and/or apply algorithms to the information, but cannot save or transmit the information outside of the secure server.

“Present” or “presenting” means to convey information, in-person or electronically, in such a way that the audience can see, read, and/or hear the information, but cannot save or record the information. This includes, but is not limited to, an in-person meeting where participants are barred from recording the discussion or taking photos, and virtual meetings or presentations where recording, downloading, and screenshotting similarly have been disabled.

II. Limited Sharing and Presenting of Information to Ensure Academic Integrity

Any Submitter to NeurIPS, or Reviewer, acknowledges and agrees to the following provisions, and that they are intended to ensure the highest ethical standards while furthering the field of Artificial Intelligence and Machine Learning:

• NeurIPS will maintain or designate a secure server, that cannot be accessed by un-authorized third-parties, for the purposes of analyzing, presenting, and sharing Submission and Review Information and thereby verifying that all Submissions are judged solely on their merit. NeurIPS maintains sole discretion to grant, modify or revoke access to its Submission and Review Information on that secure server, including with its own assigned investigator(s) or the Ethics & Grievance Committee. NeurIPS will not grant a Sister Organization (including, but not limited to, that organization’s investigator(s) or Ethics & Grievance Committee) access to the secure server, or otherwise share Submission and Review Information with the Sister Organization, unless that Sister Organization agrees that it will not attempt to make any copies of NeurIPS’ Submission and Review Information.
• In turn, Sister Organizations may provide NeurIPS (including, but not limited to, NeurIPS’ Program Chairs, their assigned investigator(s), or the Ethics & Grievance Committee) access to that Sister’s Organization Submission and Review Information on the secure server, or otherwise present that information to NeurIPS, so NeurIPS may analyze Submission and Review Information and verify that all Submissions are judged solely on their merit.
• If the initial investigation by the Program Chairs or their designated Investigator(s) indicates that collusion or other academic dishonesty may have taken place, and it therefore needs to coordinate internally with NeurIPS or with a Sister Organization to complete the investigation, determine potential sanctions, or impose sanctions, NeurIPS may also present Submission and Review Information to NeurIPS personnel beyond the Program Chairs or their designated investigator(s). Similarly, in this context, NeurIPS may also present Submission and Review Information to personnel of a Sister Organization beyond that organization’s investigator(s) or Ethics Committee, as described above.
• Providing Sister Organization(s) access to Submission and Review Information, or otherwise sharing or presenting Submission and Review Information with a Sister Organization, may occur at any time, including after a paper has been published or presented at the NeurIPS conference, or a conference by a Sister Organization.
• Because this policy is rooted in long-standing concepts of academic integrity and fairness, NeurIPS also reserves sole discretion to analyze, share and present Submission and Review Information that was created and/or collected before the announcement of this policy, and also may share or present that Submission and Review Information with other Sister Organizations, for the purposes of maintaining academic integrity and fairness. Any Submitter to NeurIPS, or any Reviewer, further agrees to a limited waiver of any privacy interest they may have in their name, current employer, academic background, or any other personal identifying information they have provided to NeurIPS, including but not limited to the Submission and Review Information. All Submitters and Reviewers further agree to grant NeurIPS a limited, royalty-free license to hare and resent their Submission and all Submission and Review Information, so that NeurIPS may work with authorized Sister Organizations to ensure academic integrity.

III. Investigation and Sanctions

Sanctions against a Submitter or Reviewer who has violated this policy or any other portion of the Code of Conduct will depend on the nature and severity of the offense(s) and after a review of all relevant information. Because violations of this policy or the Code of Conduct affect NeurIPS and all of its community members, a review may be conducted without the consent or participation of the reporting or accusing party.

Investigations will be conducted by the Program Chairs, or by an individual or sub-committee designated by the Program Chairs. That designee will then report their findings back directly to the Program Chairs. The Program Chairs will determine whether this policy has been violated, and it will provide recommended sanctions (examples provided below), to the NeurIPS Board of Directors. This recommendation may include presenting Submission and Review Information, as discussed above. The Board of Directors may consult with outside counsel or any other person or entity that it deems necessary to assist it during this process, including sharing or presenting Submission and Review Information. The Board of Directors, by majority vote, will determine whether to accept the recommended sanction, or instead impose a different sanction which could be more or less severe than the sanction recommended by the Program Chairs. As further described below, a Submitter or Reviewer will have an opportunity to appeal prior to the imposition of any sanction.

At all times the privacy of the parties involved in any investigation are held to be confidential and are assumed innocent until proof through investigation reveals otherwise. The Code of Conduct Procedures also provides additional information regarding how investigations will be conducted.

Sanctions may include, but are not restricted to, any combination of the following:

• Formal or informal warning or counseling to the offender.
• Expulsion from a particular conference with no refund.
• Suspension from participation in future conferences, NeurIPS Foundation events, making Submissions, or serving as a Reviewer (for a set number of conferences, permanently, or indefinitely until NeurIPS determines additional review is needed).
• Reporting the incident to the offender’s employer, educational institution, funding agencies or any other entity affiliated with the offender.
• Reporting the incident to local authorities or law enforcement.
• Requirement of conciliatory efforts that may include an apology, informal mediation, or other steps intended to facilitate restoration of relationships.
• Other action(s) as deemed appropriate.

Any sanction may be voluntary or mandatory; for a specified or unspecified period of time; subject to periodic review and/or achievement of specific action items; or imposed permanently. If a sanction is to be imposed by the Board of Directors, an appeals process will be made available through the Ethics & Grievance Committee. Requests for appeal should be emailed to neurIPShotline@gmail.com. NeurIPS will then provide directions for submitting a written appeal, and additional information regarding the appeals process.

This policy expands and becomes a subset of the Code of Conduct and the Code of Ethics of the NeurIPS Foundation and will remain in effect indefinitely unless explicitly rescinded or amended, in writing, by the NeurIPS Board of Directors, in its sole discretion. NeurIPS expects all Submitters, Reviewers, and other conference participants to work proactively to promote academic integrity within all NeurIPS activities, to prevent violations of the Code of Conduct and Code of Ethics, as well as to work collaboratively to resolve violations if they occur.