Conceptual Design Generation Using Large Language Models

ABSTRACT Concept generation is a creative step in the conceptual design phase, where designers often turn to brainstorming, mindmapping, or crowdsourcing design ideas to complement their own knowledge of the domain. Recent advances in natural language processing (NLP) and machine learning (ML) have led to the rise of Large Language Models (LLMs) capable of generating seemingly creative outputs from textual prompts. The success of these models has led to their integration and application across a variety of domains, including art, entertainment, and other creative work. In this paper, we leverage LLMs to generate solutions for a set of 12 design problems and compare them to a baseline of crowdsourced solutions. We evaluate the differences between generated and crowdsourced design solutions through multiple perspectives, including human expert evaluations and computational metrics. Expert evaluations indicate that the LLM-generated solutions have higher average feasibility and usefulness while the crowdsourced solutions have more novelty. We experiment with prompt engineering and find that leveraging few-shot learning can lead to the generation of solutions that are more similar to the crowdsourced solutions.

Introduction. Research in engineering design points to the benefits of generating a large and diverse set of initial concepts during the early stage of design [1,2,3]. To support these efforts, research has begun to investigate the use of both human-powered (e.g., crowdsourcing) and computational methods that can quickly generate supplemental design concepts to support human design teams. For example, repositories of existing design knowledge have been leveraged to generate and retrieve variants of conceptual designs [4]. Other search systems involve using semantic networks to retrieve potential design solutions via the form of knowledge graphs that connects words and phrases of natural language with corresponding edges [5, 6, 7]. Also, design solutions can be crowdsourced from online crowdworkers as potential sources of inspiration for designers [8,9]. In other domains, recent advances in ML have led to the development of large generative models that are capable of outputting creative results [10, 11].

Discussion / Conclusion. Our long-term motivation is to support designers during the conceptual design stage by using pre-trained LLMs to generate creative design solutions more quickly and cheaply than current methods, such as crowdsourcing. This paper assesses whether design solutions generated by LLMs are fundamentally different from crowdsourced design solutions and uses expert and computational evaluations to evaluate the differences. Results reveal that, with the use of few-shot learning, LLMs are capable of generating design solutions that are similar to crowdsourced solutions, but these modifications lead to a decrease in the diversity of solutions that LLMs are capable of generating. Expert evaluations reveal that LLMs generate solutions that are more feasible and useful than crowdsourced solutions but less novel. This paper provides a foundation for future work to explore the use of LLMs in providing conceptual design ideas to designers.