Do LLMs Possess a Personality? Making the MBTI Test an Amazing Evaluation for Large Language Models

The field of large language models (LLMs) has made significant progress, and their knowledge storage capacity is approaching that of human beings. Furthermore, advanced techniques, such as prompt learning and reinforcement learning, are being employed to address ethical concerns and hallucination problems associated with LLMs, bringing them closer to aligning with human values. This situation naturally raises the question of whether LLMs with human-like abilities possess a human-like personality? In this paper, we aim to investigate the feasibility of using the Myers- Briggs Type Indicator (MBTI), a widespread human personality assessment tool, as an evaluation metric for LLMs. Specifically, extensive experiments will be conducted to explore: 1) the personality types of different LLMs, 2) the possibility of changing the personality types by prompt engineering, and 3) How does the training dataset affect the model’s personality. Although the MBTI is not a rigorous assessment, it can still reflect the similarity between LLMs and human personality. In practice, the MBTI has the potential to serve as a rough indicator. Our codes are available at here1.

Introduction. With the advent of the epoch-making product, ChatGPT2, numerous larger language models (LLMs) and Chatbots have emerged (Zhao et al., 2023). Thanks to this, users can ask questions in the form of a natural sentence, and then LLMs utilize their knowledge to provide detailed answers effortlessly. Furthermore, an increasing body of literature suggests (Rao et al., 2023) that LLMs possess self-improvement and reasoning capabilities that are reminiscent of human cognition, leading to the possibility that LLMs may possess virtual personalities and psychological traits. Given these developments, it naturally raises the question of whether LLMs with human-like abilities possess a human-like personality? In fact, pioneers have borrowed some human personality assessments (e.g., MBTI) to evaluate the personality of LLMs (e.g., GPT3) (Li et al., 2023b). Among them, the MBTI test (i.e., Myers-Briggs Type Indicator), one of the most widespread human personality assessment tools, will be borrowed to help us explore the personality of LLMs.

Discussion / Conclusion. In this work, we investigate the question: Do LLMs with human-like abilities exhibit humanlike personalities? To address this question, we comprehensively examine the MBTI as a preliminary assessment tool for LLMs from various perspectives. After extensive experiments, our observations lead to several key conclusions: 1) LLMs exhibit diverse personalities; 2) LLMs that lack sufficient instruction tuning are resistant to the change of MBTI types but can be influenced by explicit and implicit prompts after tuning; 3) The type of training corpus can impact the MBTI type; 4) While MBTI is not a rigorous assessment, it can serve as a rough indicator. In the future, we aim to expand our research by integrating additional pre-training datasets. In this regard, we are particularly intrigued by tasks that enhance commonsense comprehension and reasoning abilities, such as math dataset. In regards to personality indicators, there is potential for future research on AGI to utilize a broader range of personality tests for LLMs.