Skip to yearly menu bar Skip to main content


Invited Talk, Q&A
in
Workshop: Algorithmic Fairness through the lens of Metrics and Evaluation

Invited Talk: Evaluating the Ethical Competence of LLMs

Seth Lazar

[ ]
Sat 14 Dec 11:25 a.m. PST — noon PST

Abstract:

Existing approaches to evaluating LLM ethical competence place too much emphasis on the verdicts—of permissibility and impermissibility—that they render. But ethical competence doesn’t consist in one’s judgments conforming to those of a cohort of crowdworkers. It consists in being able to identify morally relevant features, prioritise among them, associate them with reasons and weave them into a justified conclusion. We identify the limitations of existing evals for ethical competence, provide an account of moral reasoning that can ground better alternatives, and discuss the practical—and philosophical—implications if LLMs ultimately do prove to be adept moral reasoners.

Chat is not available.