Competition
The Concordia Contest: Advancing the Cooperative Intelligence of Language Agents
Chandler Smith · Rakshit Trivedi · Jesse Clifton · Lewis Hammond · Sasha Vezhnevets · Marwa Abdulhai · Dylan Hadfield-Menell · Natasha Jaques · Joel Leibo · Tim Baarslag · Minsuk Chang · José Hernández-Orallo
West Meeting Room 215, 216
Building on the success of the Melting Pot contest at NeurIPS 2023, which challenged participants to develop multi-agent reinforcement learning agents capable of cooperation in groups, we are excited to propose a new contest centered on cooperation between language model (LM) agents in intricate, text-mediated environments. Our goal is to advance research on the cooperative intelligence of such LM agents. Of particular interest are the agents capable of using natural language to effectively cooperate with each other in complex environments, even in the face of challenges such as competing interests, differing values, and potential miscommunication. To this end, we will leverage the recently released Concordia framework, an open-source library for defining open-ended environments where LM agents like those of Park et al. (2023) can interact with one another by generating free-form natural text describing what they intend to do or say. Concordia provides a suite of mixed-motive social dilemma scenarios where cooperation is valuable but hard to achieve. The proposed contest will challenge the participants to develop LM agents that exhibit cooperative intelligence in a variety of Concordia scenarios designed to assess multiple distinct skills of cooperation, including promise-keeping, negotiation, reciprocity, reputation, partner choice, compromise, and sanctioning. Participants will be scored based on the ability of their trained agents in executing skillful cooperation, particularly in the presence of new co-players in unforeseen (held-out) scenarios. Given the rapid development of LMs and the anticipated increase in the use of personalised LM agents, we contend that their propensity and ability to cooperate well with a diverse array of other actors (human or machine) will soon be of critical importance.
Live content is unavailable. Log in and register to view live content
Schedule
Sun 9:00 a.m. - 9:10 a.m.
|
Welcome
(
Introduction
)
>
|
Chandler Smith 🔗 |
Sun 9:10 a.m. - 9:30 a.m.
|
Measuring Cooperative Intelligence: How & Why
(
Invited Talk
)
>
|
Lewis Hammond 🔗 |
Sun 9:30 a.m. - 10:00 a.m.
|
Introducing The Concordia Contest: Advancing the Cooperative Intelligence of Language Agents
(
Invited Talk
)
>
|
Sasha Vezhnevets 🔗 |
Sun 10:00 a.m. - 10:45 a.m.
|
Competition Overview and Invited Team Talks
(
Invited Talk
)
>
|
Chandler Smith 🔗 |
Sun 10:45 a.m. - 11:00 a.m.
|
Winner Announcement
(
Invited Talk
)
>
|
Chandler Smith 🔗 |
Sun 11:10 a.m. - 12:00 p.m.
|
Panel Discussion with Diyi Yang, Fei Fang, José Hernández-Orallo and Joel Leibo
(
Invited Talk
)
>
|
Chandler Smith · Fei Fang · Diyi Yang · José Hernández-Orallo 🔗 |