Agent-Centric Projection of Prompting Techniques and Implications for Synthetic Training Data for Large Language Models

Abstract: Recent advances in prompting techniques and multi-agent systems for Large Language Models (LLMs) have produced increasingly complex approaches. However, we lack a framework for characterizing and comparing prompting techniques or understanding their relationship to multi-agent LLM systems. This position paper introduces and explains the concepts of linear contexts (a single, continuous sequence of interactions) and non-linear contexts (branching or multi-path) in LLM systems. These concepts enable the development of an agent-centric projection of prompting techniques, a framework that can reveal deep connections between prompting strategies and multi-agent systems. We propose three conjectures based on this framework: (1) results from non-linear prompting techniques can predict outcomes in equivalent multi-agent systems, (2) multi-agent system architectures can be replicated through single-LLM prompting techniques that simulate equivalent interaction patterns, and (3) these equivalences suggest novel approaches for generating synthetic training data.

Introduction. Large Language Models (LLMs) are a recent development in Generative Artificial Intelligence that can mimic human-like behavior (Park et al., 2023), especially in conversations (Cai et al., 2023). LLMs have also shown a kind of general intelligence (Radford et al., 2019; Yogatama et al., 2019). Central to harnessing the capabilities of LLMs is the concept of prompting, a strategy that significantly influences task performance by instructing LLMs in specific ways (Chen et al., 2023). HYPOTHESIS AND GOALS: In this position paper, we hypothesize that viewing prompting techniques through a proposed agent-centric lens can help uncover structural equivalences between single-LLM prompting and multi-agent approaches. Our goal is to (1) introduce a unified framework for comparing these techniques, (2) develop and examine conjectures about their relationship, and (3) outline how this perspective can inform the generation of synthetic training data. Consider a simple math problem. When we directly prompt an LLM, “What is 13 × 27?”, we might receive a single numeric answer.

Discussion / Conclusion. • AGENT-CENTRIC PROJECTION OF PROMPTING TECHNIQUES: We demonstrate approaches that allow prompting techniques with non-linear context to be understood as multi-agent systems and vice versa. This projection provides a framework for analyzing, comparing, and improving both prompting techniques and multi-agent system architectures. • SYNTHETIC TRAINING DATA GENERATION: Our core arguments suggest two novel approaches for generating high-quality synthetic training data for LLMs: (1) converting successful non-linear prompting traces into linear training data and (2) augmenting real-world task traces with synthetic agent collaboration artifacts. These approaches could provide structured, high-quality data specifically suited for training LLMs in multi-agent and complex reasoning tasks. • REAL-WORLD APPLICATIONS AND ETHICAL CONSIDERATIONS: As these systems become more capable, their deployment in real-world scenarios becomes more feasible. With this comes the need for rigorous ethical considerations, especially concerning autonomy, decision making, and human-AI interaction.