Workshop: Navigating the Broader Impacts of AI Research
Carolyn Ashurst, Rosie Campbell, Deborah Raji, Solon Barocas, Stuart Russell
2020-12-12T05:30:00-08:00 - 2020-12-12T15:00:00-08:00
more: https://nbiair.com/
Abstract: Following growing concerns with both harmful research impact and research conduct in computer science, including concerns with research published at NeurIPS, this year’s conference introduced two new mechanisms for ethical oversight: a requirement that authors include a “broader impact statement” in their paper submissions and additional evaluation criteria asking paper reviewers to identify any potential ethical issues with the submissions.
These efforts reflect a recognition that existing research norms have failed to address the impacts of AI research, and take place against the backdrop of a larger reckoning with the role of AI in perpetuating injustice. The changes have been met with both praise and criticism some within and outside the community see them as a crucial first step towards integrating ethical reflection and review into the research process, fostering necessary changes to protect populations at risk of harm. Others worry that AI researchers are not well placed to recognize and reason about the potential impacts of their work, as effective ethical deliberation may require different expertise and the involvement of other stakeholders.
This debate reveals that even as the AI research community is beginning to grapple with the legitimacy of certain research questions and critically reflect on its research practices, there remains many open questions about how to ensure effective ethical oversight. This workshop therefore aims to examine how concerns with harmful impacts should affect the way the research community develops its research agendas, conducts its research, evaluates its research contributions, and handles the publication and dissemination of its findings. This event complements other NeurIPS workshops this year devoted to normative issues in AI and builds on others from years past, but adopts a distinct focus on the ethics of research practice and the ethical obligations of researchers.
These efforts reflect a recognition that existing research norms have failed to address the impacts of AI research, and take place against the backdrop of a larger reckoning with the role of AI in perpetuating injustice. The changes have been met with both praise and criticism some within and outside the community see them as a crucial first step towards integrating ethical reflection and review into the research process, fostering necessary changes to protect populations at risk of harm. Others worry that AI researchers are not well placed to recognize and reason about the potential impacts of their work, as effective ethical deliberation may require different expertise and the involvement of other stakeholders.
This debate reveals that even as the AI research community is beginning to grapple with the legitimacy of certain research questions and critically reflect on its research practices, there remains many open questions about how to ensure effective ethical oversight. This workshop therefore aims to examine how concerns with harmful impacts should affect the way the research community develops its research agendas, conducts its research, evaluates its research contributions, and handles the publication and dissemination of its findings. This event complements other NeurIPS workshops this year devoted to normative issues in AI and builds on others from years past, but adopts a distinct focus on the ethics of research practice and the ethical obligations of researchers.
Video
Chat
Chat is not available.
Schedule
2020-12-12T05:30:00-08:00 - 2020-12-12T05:45:00-08:00
Welcome
2020-12-12T05:45:00-08:00 - 2020-12-12T06:15:00-08:00
Morning keynote
Hanna Wallach, Rosie Campbell
2020-12-12T06:15:00-08:00 - 2020-12-12T07:15:00-08:00
Ethical oversight in the peer review process
Sarah Brown, Heather Douglas, Iason Gabriel, Brent Hecht, Rosie Campbell
Discussion on potential interventions such as 'broader impact statements' and there effectiveness
2020-12-12T07:15:00-08:00 - 2020-12-12T07:30:00-08:00
Morning break
2020-12-12T07:30:00-08:00 - 2020-12-12T08:30:00-08:00
Harms from AI research
Anna Lauren Hoffmann, Nyalleng Moorosi, Vinay Prabhu, Deborah Raji, Jacob Metcalf
Case studies and mitigation approaches
2020-12-12T08:30:00-08:00 - 2020-12-12T09:30:00-08:00
Predictive Policing: Should researchers engage?
Logan Koepke, CATHERINE ONEIL, Tawana Petty, Cynthia Rudin, Deborah Raji
2020-12-12T09:30:00-08:00 - 2020-12-12T10:30:00-08:00
Lunch and watch lightning talks (in parallel) from workshop submissions
In advance of the next session (which is discussions with paper authors), please take some time to watch the videos of the submitted papers over lunch.
2020-12-12T10:30:00-08:00 - 2020-12-12T11:30:00-08:00
Discussions with authors of submitted papers
Please join our Gather Town to meet the paper authors! - Walk up to the title of the paper you're interested in, then press 'x' to view the paper - There is a lounge area to the North of the paper discussion room if you would like to have more informal conversations
2020-12-12T11:30:00-08:00 - 2020-12-12T12:30:00-08:00
Responsible publication: NLP case study
Miles Brundage, Bryan McCann, Colin Raffel, Natalie Schulter, Zeerak Waseem, Rosie Campbell
2020-12-12T12:30:00-08:00 - 2020-12-12T12:45:00-08:00
Afternoon break
2020-12-12T12:45:00-08:00 - 2020-12-12T13:45:00-08:00
Strategies for anticipating and mitigating risks
Ashley Casovan, Timnit Gebru, Shakir Mohamed, Jess Whittlestone, Solon Barocas
2020-12-12T13:45:00-08:00 - 2020-12-12T14:45:00-08:00
The roles of different parts of the research ecosystem in navigating broader impacts
Josh Greenberg, Liesbeth Venema, Ben Zevenbergen, Lilly Irani, Solon Barocas
2020-12-12T14:45:00-08:00 - 2020-12-12T15:00:00-08:00
Closing remarks
AI in the “Real World”: Examining the Impact of AI Deployment in Low-Resource Contexts
Chinasa T. Okolo
Auditing Government AI: Assessing ethical vulnerability of machine learning
Alayna A Kennedy
Non-Portability of Algorithmic Fairness in India
Nithya Sambasivan, Erin Arnesen, Ben Hutchinson, Vinod Prabhakaran
Ethical Testing in the Real World: Recommendations for Physical Testing of Adversarial Machine Learning Attacks
Ram Shankar Siva Kumar, Maggie Delano, Kendra Albert, Afsaneh Rigot, Jonathon Penney
An Open Review of OpenReview: A Critical Analysis of the Machine Learning Conference Review Process
David Tran, Alex Valtchanov, Keshav R Ganapathy, Raymond Feng, Eric Slud, Micah Goldblum, Tom Goldstein
Training Ethically Responsible AI Researchers: a Case Study
Hang Yuan, Claudia Vanea, Federica Lucivero, Nina Hallowell
Anticipatory Ethics and the Role of Uncertainty
Priyanka Nanayakkara, Nicholas Diakopoulos, Jessica Hullman
The Managerial Effects of Algorithmic Fairness Activism
Bo Cowgill, Fabrizio Dell'Acqua, Sandra Matz
Like a Researcher Stating Broader Impact For the Very First Time
Grace Abuhamad, Claudel Rheault
Biased Programmers? Or Biased Data? A Field Experiment in Operationalizing AI Ethics
Bo Cowgill, Fabrizio Dell'Acqua, Augustin Chaintreau, Nakul Verma, Samuel Deng, Daniel Hsu
An Ethical Highlighter for People-Centric Dataset Creation
Margot Hanley, Apoorv Khandelwal, Hadar Averbuch-Elor, Noah Snavely, Helen Nissenbaum
Overcoming Failures of Imagination in AI Infused System Development and Deployment
Margarita Boyarskaya, Alexandra Olteanu, Kate Crawford