Self-reflecting Large Language Models: A Hegelian Dialectical Approach

Investigating NLP through a philosophical lens has recently caught researcher’s eyes as it connects computational methods with classical schools of philosophy. This paper introduces a philosophical approach inspired by the Hegelian Dialectic for LLMs’ self-reflection, utilizing a self-dialectical approach to emulate internal critiques and then synthesize new ideas by resolving the contradicting points. Moreover, this paper investigates the effect of LLMs’ temperature for generation by establishing a dynamic annealing approach, which promotes the creativity in the early stages and gradually refines it by focusing on the nuances, as well as a fixed temperature strategy for generation. Our proposed approach is examined to determine its ability to generate novel ideas from an initial proposition. Additionally, a Multi Agent Majority Voting (MAMV) strategy is leveraged to assess the validity and novelty of the generated ideas, which proves beneficial in the absence of domain experts. Our experiments show promise in generating new ideas and provide a stepping stone for future research.

Introduction. The evolution of generative AI and foundational models, particularly the revolution in Natural Language Processing (NLP) driven by the advent of Large Language Models (LLMs), has unlocked new opportunities and made significant strides toward achieving human-level reasoning, innovations and scientific discoveries (Zhang et al., 2024c; Wu et al., 2023; Zhang et al., 2024b; Smith & Doe, 2023). Nowadays, users of LLMs frequently employ strategies like In-Context Learning (ICL), one of the simplest and most efficient paradigms in Natural Language Understanding (NLU). This method involves guiding a pre-trained model using instructions or demonstrations (e.g., providing examples to tackle a new task without additional training or fine-tuning), thereby harnessing the model’s inherent capabilities like zero-shot and few-shot reasoning (Dong et al., 2024). Despite the numerous successes and advantages of LLM reasoning, ensuring factual accuracy during reasoning remains a significant challenge (Abdali et al., 2024b;c).

Discussion / Conclusion. This paper presents a novel technique for LLMs’ selfreflection, by framing it as a self-dialectical approach using the principles of Hegel’s dialectic. our approach employs an iterative process in which an initial idea (thesis) is evaluated through generated critiques (antithesis), and subsequently refined into a new idea (synthesis) which incorporates the best elements of both. Additionally, we explore the effect of LLMs temperature on the novelty of generated ideas by establishing two configurations: 1) a constant temperature and, 2) a dynamic annealing temperature settings. Our goal is to generate new ideas that satisfy two metrics: validity which measures whether the synthesis process is followed correctly and novelty, which assesses if the synthesis adds new information and is more novel than the thesis. In order to do so, we employ MAMV a multi-agent framework i.e., to collaborate and vote for these metrics. Our experiments demonstrate that responses varied even with the same hyper-parameters and prompts, resulting in different numbers of steps and novelty scores.