Poster
in
Workshop: Evaluating Evaluations: Examining Best Practices for Measuring Broader Impacts of Generative AI
LLMs and Personalities: Inconsistencies Across Scales
Tommaso Tosato · David Lemay · Mahmood Hegazy · Irina Rish · Guillaume Dumas
Keywords: [ Personality Assessment ] [ Scaling ] [ Persona Prompting ] [ LLMs ] [ Trait Consistency ]
We investigated the application of human psychometric assessments to LLMs by administering the Big Five Inventory (BFI) to models of different sizes. Our results reveal large variability due to question order shuffling, challenging the notion of a stable ``personality.'' Notably, models prompted to take up the assistant persona led to more predictable scaling, with larger models exhibiting more socially desirable and less variable traits. In contrast, non-conventional personas displayed unpredictable behaviors.