Do generative UI tools actually implement their stated design rationales?

The paper names "Design Theater" as "plausible and confident design rationales that have little relationship to the actual implementation" in generative UI tools. To measure it, the authors built a benchmark of 24 UI-generation tasks spanning structural, styling, and functional requirements, and evaluated 120 interfaces produced by five tools — ChatGPT, Claude, Firebase Studio, Vercel v0, and Bolt — under each tool's default configuration. Across the sample, "roughly 25% of user-facing design rationales are not implemented in the generated interface," and "the implementation failure increases to 34% for functional requirements." A second metric, Principle Adherence Score, found tools recognized only about half the UX principles implicitly embedded in prompts (mean 0.54), and "four of five tools implementing 6% or fewer functional principles" such as visibility of system status, user control and freedom, and error prevention and recovery.

The paper defines Design Theater "by its effect on the reader": a rationale written in the language of professional design reads as evidence of deliberate, principle-driven decision-making "whether or not those commitments are realized in the artifact," and this "persuasive reasoning invites less scrutiny of the generated artifact, allowing overreliance to emerge before verification occurs." The authors argue the most consequential gaps are also the least visible — a missing color choice or layout inconsistency shows up in the rendered screen, but "missing state management, inaccessible interactions, weak error recovery, or absent user control may not be obvious from surface-level inspection" — which is precisely the category where four of five tools scored worst.

This extends Do generated analysis UIs really work better than chat? onto a different axis: that note's trade-off is between clarity and authoring friction once a generative UI exists, while this paper's gap is between what a tool says it built and what it actually built, independent of whether the user likes using it. It also complicates Do generated interfaces outperform text-based chat for most tasks? — a stated preference for generated interfaces over chat says nothing about whether the generated interface implements what its own narration claims. And it shares a mechanism with Where do vibe coding students actually spend their debugging time?: both describe non-experts evaluating AI output at the surface (the rendered prototype, the plausible rationale) rather than the underlying implementation, because that is the layer they have the expertise to inspect.

The study is, in the authors' own words, "artifact-centered": it establishes that rationale-to-implementation mismatch exists in these five tools on these 24 tasks, but it does not measure whether designers, product managers, or novice creators actually notice the gap, or how noticing it would change trust, review, or deployment decisions. The finding also can't be generalized beyond the five tools and task set tested. What it does support, at the strength the evidence allows, is that a generated interface's own narrated rationale should not be treated as a substitute for checking the interface itself, especially for interaction-design requirements that don't show up on screen.