Poster
in
Workshop: Algorithmic Fairness through the lens of Metrics and Evaluation
Understanding The Effect Of Temperature On Alignment With Human Opinions
Maja Pavlovic · Massimo Poesio
Keywords: [ NLP ] [ Evaluation Metrics and Techniques ]
Sat 14 Dec 9 a.m. PST — 5:30 p.m. PST
With the increasing capabilities of LLMs, recent studies focus on understanding whose opinions are represented by them and how to effectively extract aligned opinion distributions.We conduct an empirical analysis of three straightforward methods for obtaining distributions and evaluate the results across a variety of metrics. Our findings suggest that simple parameter adjustments can return better aligned outputs in subjective tasks. distributional alignment. Assuming models reflect human opinions may be limiting, highlighting the need for further research on how human subjectivity affects model uncertainty and further distributional alignment evaluation methods.