Workshop: Algorithmic Fairness through the Lens of Causality and Interpretability

Awa Dieng, Jessica Schrouff, Matt J Kusner, Golnoosh Farnadi, Fernando Diaz

2020-12-12T01:00:00-08:00 - 2020-12-12T12:10:00-08:00
Abstract: Black-box machine learning models have gained widespread deployment in decision-making settings across many parts of society, from sentencing decisions to medical diagnostics to loan lending. However, many models were found to be biased against certain demographic groups. Initial work on Algorithmic fairness focused on formalizing statistical measures of fairness, that could be used to train new classifiers. While these models were an important first step towards addressing fairness concerns, there were immediate challenges with them. Causality has recently emerged as a powerful tool to address these shortcomings. Causality can be seen as a model-first approach: starting with the language of structural causal models or potential outcomes, the idea is to frame, then solve questions of algorithmic fairness in this language. Such causal definitions of fairness can have far-reaching impact, especially in high risk domains. Interpretability on the other hand can be viewed as a user-first approach: can the ways in which algorithms work be made more transparent, making it easier for them to align with our societal values on fairness? In this way, Interpretability can sometimes be more actionable than Causality work.

Given these initial successes, this workshop aims to more deeply investigate how open questions in algorithmic fairness can be addressed with Causality and Interpretability. Questions such as: What improvements can causal definitions provide compared to existing statistical definitions of fairness? How can causally grounded methods help develop more robust fairness algorithms in practice? What tools for interpretability are useful for detecting bias and building fair systems? What are good formalizations of interpretability when addressing fairness questions?

Website: www.afciworkshop.org

Video

Chat

Chat is not available.

Schedule

2020-12-12T01:00:00-08:00 - 2020-12-12T01:20:00-08:00
AFCI2020: Opening remarks
Awa Dieng
Gathertown: https://gather.town/app/SaW73Rp8mVWAKH8L/AFCI%20Workshop%202020
2020-12-12T01:20:00-08:00 - 2020-12-12T01:47:00-08:00
Tutorial: A brief tutorial on causality and fair decision making
Moritz Hardt
Moritz Hardt
2020-12-12T01:47:00-08:00 - 2020-12-12T01:55:00-08:00
Tutorial: Questions
Questions for the speaker and introduction to the next talk.
2020-12-12T01:55:00-08:00 - 2020-12-12T02:25:00-08:00
Invited Talk: On Prediction, Action and Interference
Ricardo Silva
Ultimately, we want the world to be less unfair by changing it. Just making fair passive predictions is not enough, so our decisions will eventually have an effect on how a societal system works. We will discuss ways of modelling hypothetical interventions so that particular measures of counterfactual fairness are respected: that is, how are sensitivity attributes interacting with our actions to cause an unfair distribution outcomes, and that being the case how do we mitigate such uneven impacts within the space of feasible actions? To make matters even harder, interference is likely: what happens to one individual may affect another. We will discuss how to express assumptions about and consequences of such causative factors for fair policy making, accepting that this is a daunting task but that we owe the public an explanation of our reasoning. Joint work with Matt Kusner, Chris Russell and Joshua Loftus
2020-12-12T02:25:00-08:00 - 2020-12-12T02:30:00-08:00
Questions: Invited talk, R. Silva
2020-12-12T02:30:00-08:00 - 2020-12-12T02:38:00-08:00
Short break -- Join us on Gathertown
2020-12-12T02:38:00-08:00 - 2020-12-12T02:40:00-08:00
Introduction to contributed talks
2020-12-12T02:40:00-08:00 - 2020-12-12T02:50:00-08:00
Contributed Talk 1: The Importance of Modeling Data Missingness in Algorithmic Fairness
Naman Goel, Amit Deshpande
2020-12-12T02:50:00-08:00 - 2020-12-12T03:00:00-08:00
Contributed Talk 2: Foundations for Languages for Interpretability and Bias Detection
Bernardo Subercaseaux, Jorge Pérez, Pablo Barceló
2020-12-12T03:00:00-08:00 - 2020-12-12T03:10:00-08:00
Contributed Talk 3: Fairness in Risk Assessment: Post-Processing to Achieve Counterfactual Equalized Odds
Alan Mishler, Edward Kennedy, Alexandra Chouldechova
2020-12-12T03:10:00-08:00 - 2020-12-12T03:13:00-08:00
Introduction to invited talk by Hoda Heidari
2020-12-12T03:13:00-08:00 - 2020-12-12T03:38:00-08:00
Invited Talk: Fairness through the lens of equality of opportunity and its connection to causality
Hoda Heidari
I begin by presenting a mapping between existing mathematical notions of fairness and economic models of Equality of opportunity (EOP)—an extensively studied ideal of fairness in political philosophy. Through our conceptual mapping, many existing definitions of fairness, such as predictive value parity and equality of odds, can be interpreted as special cases of EOP. In this respect, the EOP interpretation serves as a unifying framework for understanding the normative assumptions underlying existing notions of fairness. I will conclude by discussing a causal interpretation of EOP-based notions of fairness and some thoughts on defining counterfactual notions of fairness.
2020-12-12T03:38:00-08:00 - 2020-12-12T03:45:00-08:00
Questions: Invited talk by H. Heidari + Intro to Breakout Sessions 1
Add your questions for the speaker in Rocket.chat and the moderator will pass them to the speaker. You can also join the Zoom call to ask your question live.
2020-12-12T03:45:00-08:00 - 2020-12-12T03:55:00-08:00
Short break -- Join us on Gathertown
2020-12-12T03:55:00-08:00 - 2020-12-12T04:55:00-08:00
Virtual Breakout Session 1
2020-12-12T04:55:00-08:00 - 2020-12-12T05:00:00-08:00
Introduction to Poster session
2020-12-12T05:00:00-08:00 - 2020-12-12T06:00:00-08:00
Poster session 1 -- Join us on Gathertown
2020-12-12T06:00:00-08:00 - 2020-12-12T07:55:00-08:00
Long break -- Join us on Gathertown
2020-12-12T07:55:00-08:00 - 2020-12-12T08:00:00-08:00
Introduction to invited talk by Jon Kleinberg
2020-12-12T08:00:00-08:00 - 2020-12-12T08:30:00-08:00
Invited Talk: The Roles of Simplicity and Interpretability in Fairness Guarantees
Jon Kleinberg
We explore two arguments at the interface between the interpretability of algorithms and their fairness properties. We first discuss how a well-regulated algorithm for screening decisions, because it makes notions like feature sets and objective functions explicit, can be audited for evidence of discrimination in ways that would be essentially impossible for human decision-making. We then consider connections to a related fundamental point -- that as we simplify algorithms, reducing the range of features available to them, there is a precise sense in which we can find ourselves sacrificing accuracy and equity simultaneously. This talk will be based on joint work with Jens Ludwig, Sendhil Mullainathan, Manish Raghavan, and Cass Sunstein.
2020-12-12T08:32:00-08:00 - 2020-12-12T08:40:00-08:00
Questions: Invited talk, J. Kleinberg
Ask your questions to the speaker in Rocket Chat and the moderator will pass them along. To ask questions live, please join the zoom call linked at the top of the page.
2020-12-12T08:40:00-08:00 - 2020-12-12T08:50:00-08:00
Contributed talks 4: Counterfactual Learning of Fair Interventions
Kristina Lerman
2020-12-12T08:50:00-08:00 - 2020-12-12T09:00:00-08:00
Contributed talks 5: Fairness and Robustness in Invariant Learning: A Case Study in Toxicity Classification
Elliot Creager, David Madras, Richard Zemel
2020-12-12T09:00:00-08:00 - 2020-12-12T09:10:00-08:00
Contributed talks 6: Group Fairness by Probabilistic Modeling with Latent Fair Decisions
YooJung Choi, Guy Van den Broeck
2020-12-12T09:10:00-08:00 - 2020-12-12T09:15:00-08:00
Short Break -- Join us on Gathertown
2020-12-12T09:15:00-08:00 - 2020-12-12T09:18:00-08:00
Introduction to invited talk by Lily Hu
2020-12-12T09:18:00-08:00 - 2020-12-12T09:40:00-08:00
Invited Talk: Does Causal Thinking About Discrimination Assume a Can Opener?
Lily Hu
Increasingly, proof of whether systems, algorithmic and not, are racially discriminatory typically takes the form of statistical evidence supposedly showing race to have causally influenced some outcome. In this talk, I will discuss the relationship between quantitative social scientific methods on causal effects of race and our normative thinking about racial discrimination. I argue that all causal inference methodologies that look to quantify causal effects of race embed what amount to substantive views about what race as a social category is and how race produces effects in the world. Though debates among causal inference methodologists are often framed as concerning which practices make for good statistical hygiene, I suggest that quantitative methods are much more straightforwardly normative than most scholars, social scientists and philosophers alike, have previously appreciated. Thinking causally about race is, I want to suggest, at least just as hard as the substantive discrimination question. For answering the question about race and causation in the social world, requires answers to substantive normative questions about race, racial discrimination, and racial injustice more broadly. And so thinking about how race acts causally is not easier or even a helpful reduction for answering the moral and political question. If we’ve “solved” the causal problem, we’ve “solved” the substantive normative questions about race, racial discrimination, and racial injustice more broadly. It reminds one of the following joke: A physicist, a chemist, and an economist who are stranded on a desert island with no implements and a can of food. The physicist and the chemist each devise an ingenious mechanism for getting the can open. The economist says, "Assume we have a can opener!" My argument is that tackling the racial discrimination problem by assuming we can draw a diagram of how race acts causally in the world is a bit like that: it is to assume we have what it is that we precisely need; it is to assume a can opener!
2020-12-12T09:40:00-08:00 - 2020-12-12T09:55:00-08:00
Questions: Invited talk by L. Hu + Intro to Breakout Sessions 2
Ask your questions to the speaker in Rocket Chat and the moderator will pass them along. To ask questions live, please join the zoom call linked at the top of the page.
2020-12-12T09:55:00-08:00 - 2020-12-12T10:55:00-08:00
Virtual Breakout Session 2
2020-12-12T10:55:00-08:00 - 2020-12-12T11:55:00-08:00
Poster session 2 -- Join us on Gathertown
2020-12-12T11:55:00-08:00 - 2020-12-12T12:10:00-08:00
AFCI2020: Closing remarks
Jessica Schrouff