1 Introduction↩︎

Performance dashboards are visualization systems designed for organizations such as enterprises and government agencies to measure and monitor their operational performance effectively [1]–[3]. Often deployed in command centers and public spaces, performance dashboards have become increasingly critical in facilitating data communication and supporting informed decision-making across diverse domains [4], [5]. Unlike dashboards designed to support exploratory analysis by data professionals, a performance dashboard is a full-fledged business information system built upon business intelligence and data integration infrastructures, which provides timely and high-level visibility into key performance indicators [2]. An exemplary case of a performance dashboard is “Alibaba Double Eleven Product Sales Monitoring Dashboard” (Fig. 1). The dashboard was designed for real-time sales tracking during the annual shopping extravaganza, enabling businesses to monitor key metrics like total sales, regional distribution, and top products through an intuitive interface. Alibaba’s success in the digital economy is made visible through the dashboard. More than a promotional campaign, Double Eleven is setting benchmarks for online retail and reshaping contemporary shopping culture. The examples demonstrate the transformative role of performance dashboards in addressing both commercial and societal needs.

However, based on our in-depth discussions with professional designers, performance dashboard design is a complex task, requiring the careful manipulation and coordination of various elements, including data visualizations, embellishments, layout structures, and animations [5]–[7]. Consequently, the design process often demands substantial expertise, including visual encoding, graphical design, and aesthetic considerations. Additionally, in real-world industrial settings, designers are often required to rapidly produce design prototypes during early-stage collaboration and negotiation with clients. It helps save costs before formal implementation and addresses potential data availability and security limitations. As a result, designers frequently engage in prototype design without access to the actual datasets, relying instead on abstract requirements or high-level data descriptions. Although existing works have explored interactive authoring and recommendation tools for multi-view dashboards [8], [9], these methods are mostly not tailored to performance dashboard prototype generation. MultiVision [10] and DashBot [11] are data-centric, leveraging data characteristics and machine learning techniques to recommend charts tailored to user needs. They have primarily focused on data mining and analysis, with relatively little attention to visual design, such as layout composition. Rather than supporting analytical exploration, performance dashboards serve primarily as communication platforms for operational performance. Meanwhile, design tools like Tableau [12] and Flourish [13] are not tailored for performance dashboards and require substantial manual effort to create them. In industrial settings, clients often expect an efficient creation process and deployment-ready dashboard prototypes. While design ideation systems such as LADV [14] have enabled convenient chart generation directly from visual inputs like hand-drawn sketches, there remains limited exploration into authoring tools that support dashboard prototype construction purely through natural language instructions.

Motivated by the limitations, we aim to develop an authoring tool to generate performance dashboard prototypes. There are two major challenges to overcome. First, dashboards are applied across various scenarios and exhibit substantial internal diversity. For instance, dashboards designed for data analysis differ significantly from those intended for external presentation. Despite current researches characterize design goals [6], data presentations [7] and layout arrangement [7], [15] of general dashboard designs. There is no systematic design guideline tailored explicitly for performance dashboards that focus on data external presentation. The second challenge is to generate prototypes from vague, high-level text without specific data. In real-world industrial scenarios, designers often provide abstract descriptions such as “Global e-commerce sales dashboard showing country-level sales, daily sales by category, and quarterly net profit.” without a dataset, making it difficult to determine appropriate visualization types and layout structures. Accurately translating the unstructured text into compliant prototypes requires dedicated contextual inference and intelligent decision-making abilities, which impose high demands on our algorithm.

To address the challenges, we propose \({DashChat}\), an interactive system that leverages large language models (LLMs) to generate performance dashboard design prototypes from natural language automatically. For the first challenge, we collected 114 high-quality performance dashboards from various domains and summarized a design space that characterizes their view types, view layouts, analysis tasks, and embellishments. The analysis results are then injected into LLMs as a reference during generation. Second, we built a multi-agent pipeline powered by LLMs to generate effective and aesthetic prototypes. We leverage LLMs’ powerful language comprehension to develop intelligent agents with decision-making and diverse tool-use functionalities. Additionally, due to the high computational cost of LLMs, our pipeline employs a parallel framework to enhance efficiency. Based on the pipeline, we developed an interface consisting of the Chat Panel and History Panel, which supports users in authoring performance dashboard prototypes through conversations with agents and reviewing previous specifications. It also assists users in clarifying their prototype requirements by offering multiple potential options, while supporting an iterative trial-and-error process informed by user feedback. In conclusion, the key contributions of our work include:

• A performance dashboard design space identified through an empirical analysis of 114 high-quality performance dashboards, which establishes a structured knowledge base to steer prototype generation.

• A multi-agent automatic dashboard prototype generation pipeline for realizing an interactive system DashChat, that enables designers to construct effective and aesthetic performance dashboard prototypes through conversational interaction. It demonstrates the potential of LLMs to support real-world design workflows by translating natural language input into visual encodings, reducing the entry barrier for non-technical stakeholders.

• A quantitative study showing the effectiveness of our multi-agent generation pipeline and a user study to verify the usefulness and efficiency of our system.

2 Related Work↩︎

We review prior research on dashboard design patterns and guidelines, automated dashboard authoring tools, and LLM-powered agents and visualization design ideation.

2.1 Dashboard Design Patterns and Guidelines↩︎

The dashboard is a visual display consolidated within a single screen, designed to present the most critical information for supporting one or more data-driven insights [16], [17]. Fewet al. [18] argued that it can be applied both for monitoring in static forms and for analytical tasks when dynamic and interactive. Researchers have agreed on some design guidelines: (i) Avoiding visual clutters [18], [19]; (ii) Selecting indicators with care and avoiding too much data [20], [21]; (iii) Aligning with user workflows [22]; (iv) Providing consistency, interaction affordances and manage complexity [6]; (v) Organizing charts symmetrically and according to time [23]; (vi) Balancing the tradeoff between detail density in a single view and the need for data access and exploration through charts [7]. Researchers also proposed common design patterns as useful speculative tools in the design process. Sarikayaet al. [6] raised some common dashboard designs but mainly concluded features from multiple views of infographics and other storytelling genres. Bachet al. [7] proposed a widely applicable dashboard pattern based on a contemporary corpus, dividing it into content and composition. The content pattern includes data, meta information, and visual representations, while the composition pattern covers layout, screenspace, structure, interaction, and color. Similarly, Chenet al. [15] studied real-world dashboard designs and proposed two fundamental measures, composition that quantifies view types and configuration which characterizes view layouts. Meanwhile, many regarded dashboard design as a layout design problem and proposed rule-based layout approaches based on design principles [6], [24]–[26]. On the contrary, others explored exemplar-based approaches based on design instances. O’Donovanet al. [25], Zhenget al. [27], and Al-maneeaet al. [28] optimized the layout arrangement of multiple-view visualization in infographics, magazines, visual analytics systems, respectively.

This work focuses on performance dashboards, which are primarily used within industrial settings to present operational performance and hold a prominent position in the business intelligence market. Despite their importance, research specifically targeting performance dashboards is limited. To fill the gap, we introduce a performance dashboard design space that captures their common design patterns.

2.2 Dashboard Generation and LLM-Powered Agents↩︎

To lower the barrier to authoring dashboards, an increasing amount of work has focused on automating this process. The application of rule-based methods integrates visualization knowledge into the design of metrics and the consequent chart recommendations [29]–[32]. Researchers have explored machine learning-based methods for dashboard generation. Bidirectional long short-term memory models were applied in MultiVision [10] for multiple-view visualization generation by combining single charts with high rank scores. DashBot [11] employed reinforcement learning methods integrating domain knowledge to dashboard generations. Additionally, natural language processing models can be applied to data feature extraction or recommendation next-step queries for dashboard exploration [33], [34]. Data Illustrator [35] proposed a vector drawing method that employs lazy data binding to visualization generation. LADV [14] enables the identification and generation of dashboard templates from users’ hand sketches, thus facilitating efficient ideation and authoring. We continue to explore LLMs for the automation of dashboard authoring, focusing specifically on design prototyping and investigating dashboard generation via natural language input.

The powerful generative capabilities of LLMs have been successfully applied as intelligent question-answering systems in many domains, such as finance [36], health [37], biology [38], and law [39]. LLM-powered agents [40]–[42] have demonstrated significant human-like decision-making capabilities and supported interaction with humans using natural language [43]–[45]. Recently, researchers in the visualization community have also begun to integrate LLM-powered agents into the visualization creation process, leveraging their innovative and multimodal generative capacity [46], [47]. Liet al. [48] evaluated the performance of LLMs in data visualization generation with Vega-Lite. Prompt4Vis [49] utilized an in-context learning method to push the boundaries of LLMs for text-to-vis tasks. Compared to traditional machine learning or deep learning methods, LLMs’ capability to generate coherent and contextual content [50] can facilitate reasoning in generative visualization, improving the interpretability of the design process. LLM4Vis [51] can recommend visualizations with human-like explanations. ChartGPT [52] developed a step-by-step reasoning workflow to generate visualizations that answer users’ data analytical problems. Furthermore, LLM-powered agents also enable efficient and creative stylization by supporting text-to-style adaptation, automated aesthetic refinement, and personalized visual transformation. LIDA [53] utilized LLMs and IGMs to generate stylized and data-faithful infographics from datasets. Xiaoet al. [54] and Wuet al. [55] embedded semantic context into charts based on generative AI. Following this technical trend, we present a novel exploration within the text-to-vis domain and leverage the strong generation capabilities of LLMs to support an end-to-end dashboard generation workflow. We introduce a multi-agent pipeline that interprets natural language input and automatically completes chart type, layout, and styling, enabling efficient and accurate generation without manual authoring.

2.3 Human-AI Collaboration for Design↩︎

Human-AI collaboration for design has become a significant research frontier in HCI [56], ranging from AI-as-tool to genuinely co-creative, mixed-initiative systems [57]. AI can play a significant role in supporting designers throughout the design process by identifying contextual needs, externalizing tacit ideas, and exploring a broad design space through automatic variation [58]. FashionQ [59] harnesses the capabilities of AI to enhance creativity and support decision-making processes in professional fashion design. ColorCook [60] assists designers interactively in creating expressive and effective dashboard color schemes aligned with data semantics. Vinci [61] takes a user-provided product image and taglines as input to automatically select suitable design elements and layout templates, producing an aesthetically coherent poster. Designers engage not only as recipients of AI-generated outputs, but also as co-creators, who interact with AI systems to iteratively shape and refine design outcomes, rather than passively consuming results [62], [63]. AIdeation [64] proposed a human-centered AI system that enables concept artists to efficiently explore creative directions through a flexible and iterative workflow. Keyframer [65] introduces a natural language interface for motion design, aiming to make the animation process more intuitive and accessible. By enabling a dynamic feedback loop between user input and system response, it empowers animators to explore, iterate, and refine motion ideas with greater creative autonomy. Uusitaloet al. [66] indicated that industrial designers varied in both design tenure and prior exposure to GAI technologies, allowing for a nuanced understanding of how such tools are being integrated into real-world workflows.

The emergence of LLMs has further advanced human-AI collaboration in design. The powerful linguistic reasoning capabilities enable designers to articulate nuanced design intents through natural language, significantly lowering the interaction barrier in design workflows [64], [67], [68]. Chiouet al. [69] indicated that integrating human and AI contributions within the design process facilitates new forms of self-expression and enriched modes of communication. By combining image exemplars with textual cues, IntentTuner [70] integrates user intentions into the fine-tuning workflow of text-to-image generative models, supporting intent-driven data augmentation. While recent advances in LLMs have demonstrated strong potential in supporting creative tasks and accelerating design workflows, few tools have directly addressed the real-world demands of performance dashboard prototype design, often under tight timelines and evolving client requirements. To bridge this gap, we develop an interactive system, DashChat, that enables users to author performance dashboard prototypes through conversational interaction and facilitates an iterative, feedback-driven trial-and-error design process.

3 Formative Study and Design Requirements↩︎

In this section, we present a formative study that underpins our research. This process revealed a set of critical challenges in current practice. We derive six design requirements (R1-R6) directly from these findings, which serve as the foundational principles for our system’s design.

3.1 Methodology↩︎

Our study centered on semi-structured interviews with 10 industry professionals to investigate the performance dashboard development lifecycle.

3.1.1 Participants↩︎

We recruited 10 participants (6 males and 4 females; ages 29 to 41) through social media: 3 visualization researchers (VR1-VR3), 2 business analysts (BA1-BA2), 3 data analysts (DA1-DA3), and 2 visualization engineers (VE1-VE2). All participants had 5 to 10 years (mean = 7.7, SD = 1.64) of professional experience developing performance dashboards for enterprise and government clients.

3.1.2 Procedure↩︎

We conducted semi-structured interviews with each participant, lasting 1 to 1.5 hours. The interviews focused on workflow, prototyping practices, tool-related pain points, and communication challenges, which followed a standardized protocol focusing on: (1) How stakeholders coordinate during the preparation phase of dashboard development. (2) Current practices for rapid prototyping without access to real datasets. (3) Pain points in existing tools when handling iterative design changes. (4) How domain-specific requirements influence design decisions. (5) Communication challenges between designers and engineers during data integration. After the interviews, the insights were cross-validated with our firsthand experience collaborating with the DataV [71] team and by analyzing documentation from 3 of their recent projects.

3.2 Findings↩︎

Our analysis of the interview data revealed a significant gap between idealized academic pipelines and the realities of performance practice. We synthesized the findings into three core challenges that form the primary motivation.

3.2.1 Workflow in Industrial Practice↩︎

We first summarize this typical workflow. The process often begins with a pre-data phase, where business analysts and designers craft initial prototypes from vague client requirements, primarily for negotiation and securing contracts. This is followed by a highly iterative design phase, where these prototypes undergo numerous revisions based on stakeholder feedback, often still in the absence of real data. Finally, the workflow enters a data integration phase, where engineers connect the finalized designs to actual production data and orchestrate the interactive links among them.

3.2.2 F1: The “Pre-Data” Prototyping Dilemma↩︎

A dominant theme was the critical need to create prototypes before any real data is available, a phase driven by client negotiation.

• Client-Driven Need: BA1 highlighted the commercial pressure: “Clients expect to see multiple design options before signing contracts, but we do not have access to their actual data yet.”

• Text-to-Visual Gap: This phase begins with abstract, textual requirements. BA3 noted the inefficiency of current tools: “We need to create convincing demos based only on textual requirements. However, existing tools cannot translate this into visual components efficiently.”

3.2.3 F2: The Burden of Iterative and Non-Linear Design↩︎

Participants consistently described the design process as chaotic and non-linear, characterized by constant iteration and unforeseen delays.

• Frequent Requirement Changes: DA2 shared a common experience: “After initial prototyping with simulated data, clients frequently change requirements. Last month, we had to redesign 40% of a manufacturing dashboard after three rounds of feedback.”

• Data Delays and Versioning Chaos: VE1 emphasized that even after a project starts, “data delivery delays are common.” This, combined with frequent changes, leads to version control issues. “Keeping track of different iterations is chaotic,” recalled VE2.

3.2.4 F3: The Complexity of Domain-Specific Design↩︎

A one-size-fits-all approach to design is insufficient. Prototypes must be convincing and adhere to specific domain conventions.

• Aesthetic and Layout Rules: Participants noted that different domains have distinct visual languages. For instance, DA1 mentioned that government dashboards require “dark blue palettes with clear typography,” while VR2 stated that manufacturing dashboards need “precise scale matching with live sensor data.”

• Need for Plausible Data Simulation: To make prototypes believable, the simulated data must be meaningful. DA1 explained, “When creating prototypes, we often have to guess data distributions. We might simulate 10 different traffic flow patterns for a smart city dashboard just to show possible visualizations.”

3.3 Design Requirements↩︎

Grounded in the core findings identified in our formative study (F1-F3) and literature [7], [14], [15], [48], [52], [55], we synthesized our findings into six foundational design requirements (R1-R6):

1. Efficient prototyping with textual input. To directly address the “pre-data” dilemma and the text-to-visual gap (F1), the system must offer a direct text-to-prototype methodology. This enables users to translate textual ideas into tangible formats without preparing data, thus streamlining the labor-intensive initial ideation phase.

2. Visual mapping with simulated domain-relevant data. To create convincing mock data in the absence of real data (F1) and to meet the need for plausible and domain-specific visualizations (F3), the system must populate charts with meaningful simulated data. The data should reflect the essentials of the user’s input and the relevant domain context, making the prototype a more faithful representation of the final product.

3. Strategic layout planning. In line with the need for strategic designs (F3), the system should thoughtfully arrange components based on their interrelationships and established layout patterns to ensure visual clarity and effective data communication.

4. Aesthetic and stylistic designs. To adhere to the strict domain-specific conventions (F3), the system must intelligently apply aesthetic elements. This includes selecting appropriate color patterns and incorporating common embellishments to ensure the prototype is not only functional but also stylistically and aesthetically coherent.

5. Optimization of LLMs generation performance. The inherent complexity of fulfilling the above requirements (R1-R4) demands a robust technical foundation. Therefore, the system must employ strategies to optimize the performance of LLMs, supplying it with the necessary domain knowledge to ensure both efficient and high-quality generation.

6. Augmenting interface interactivity. To combat the burdensome iterative workflow and versioning chaos (F2), the interface must support fluid and continuous refinement. This involves enabling multi-turn conversational input for complex changes while also offering direct manipulation controls for quick, targeted adjustments, thereby enhancing user efficiency and control.

4 Survey on Performance Dashboard Design↩︎

To discover the common design practices in practical performance dashboards, we conduct a formative study based on a collection of examples from various real industrial scenarios.

4.1 Methodology↩︎

Although numerous dashboard sources are available online, we focus on the specific presentation of industry dashboards. Dashboard examples are selected from industry-based templates constructed by DataV [71] and other commercial dashboard authoring tools. \(146\) performance dashboards created by professional designers, combining aesthetics and effectiveness, are first collected. Then, those focusing solely on display rather than data presentation, and those with only a single view are excluded. We finalize a dataset with \(114\) samples, covering a wide variety of domains, shown in Fig.  (e.g., retail (11.4%; 13), health (10.5%; 12), police (7.9%; 9)).

As our summary of common design practices inform LLM-powered agents in the system pipeline (R5), it complements existing dashboard design patterns rather than providing a complete taxonomy [7], [15], [72]. Bachet al. [7] argued that dashboard content components follow common structures, categorized into page layout, screenspace, content structure, interactions, and color scheme. We conduct discussions with the team members from DataV [71] and learn that performance dashboard design patterns aim to maximize screen use for visuals while minimizing audience interpretation bias. Meanwhile, most clients attempted to apply a dashboard for data insight presentations. Thus, we only consider the display relations of views, excluding their interaction relations.

A qualitative analysis of the dataset is conducted for extracting views and dashboard designs from the aspects including View Type, View Layout, Analysis Task, and Embellishment. These can be separated into dashboard and single-view designs. Inspired by Chenet al. [15], we first examine the relationships between all the views in dashboards, including their types and the position distribution of these views, namely the view layout. Then, we delved into every element in each view to determine the relationship between the analysis task of the view and the typical display form. Additionally, we investigate the application of embellishments in single views.

4.2 Dashboard Design↩︎

We explore the structure and layout of performance dashboards, drawing on the concepts of View Type and View Layout from Chen et al. [15]. These attributes define the chart types and spatial organization of views, indicating the content relationships and layout hierarchies within dashboards.

4.2.1 View Type↩︎

View Type refers to the visual mapping method used to transform data into a visual form. We base our selection of supported chart types on the comprehensive taxonomy proposed by Chenet al. [15], which identifies 14 common types for multi-view visualizations. To focus our generative model on the most relevant and prevalent chart types for performance dashboards, we first conducted a preliminary analysis of a corpus of existing dashboard examples. Our survey revealed that certain types, such as Panel (often used for interactive controls) and Tree (for hierarchical data), appear with extremely low frequency in typical dashboard contexts (see Fig. 3 (a) for a visual summary of this distribution). This empirical finding, combined with the limited utility of interactive controls like Panel in our static prototype generation context, led us to exclude both from our scope. Additionally, while SciVis (Scientific Visualization) is present in some specialized dashboards, its highly domain-specific and custom nature makes it unsuitable for generalized automated generation at this stage. Therefore, while our analysis acknowledges the full spectrum of chart types, our prototype generation process concentrates on the remaining 11 types that are most pertinent to performance dashboards, such as bar charts, line charts, and maps.

4.2.2 View Layout↩︎

View Layout is defined by how views are positioned and grouped within a dashboard (Fig.3 (b)). Building on the concept of bounding box”, which is the position and size of a view in display space, we formalize layouts based on bounding box arrangements [15]. Following Chen et al.’s hierarchical approach, we adopt a two-level hierarchy where level-2 nodes often group views of the same type, traditionally termed as “small multiples” by Tufte [73]. However, we hold an alternative opinion that the views of the same visual type can present a specific analysis presentation for a particular analytical perspective that cannot be simply concluded as the level-2 nodes in the dashboard structure. Meanwhile, there also exist multiple neighboring visual types for conveying the same data insights. Therefore, we extend the definition of ” small multiple” that it qualifies as long as it conveys a unified data insight, even if the charts differ in type. This broader interpretation enhances visual communication for specific analytical tasks, as detailed in 4.3.1.

To model dashboard layouts, we retain the hierarchical structure but refine the computation of layout relationships. A node representing multiple views can split into child nodes when the splitting direction changes; otherwise, sibling nodes represent individual views. Our analysis reveals that 80% layout trees derived from samples exhibit a two-level hierarchy (Fig. 3 (b)), reflecting a preference for concise designs that maximize readability. Based on this insight, we incorporate four primary layout patterns into our generation process.

4.3 Single View Design↩︎

We disaggregate each view of the dashboard into individuals to investigate the features of the single-view display. We extract \(674\) individual views from \(114\) dashboard examples. Since our goal is to investigate the views which contain multiple visual elements, we extend the definition of small multiples that are considered as a single view which is made up of multiple visual elements that correspond to the same data insights or analysis tasks. Therefore, \(470\) views are removed as they only contain single visualizations in views, which do not fulfill the definition of small multiples. Therefore, we hold \(204\) “small multiples” as the sample data.

4.3.1 Analysis Task↩︎

We believe that individual views with multiple visualizations simultaneously serve specific analysis tasks (Fig.4 (a)) while communicating unified data insights. We categorize these views based on the analysis task criteria from DataShot [29]. However, survey results from DataShot suggest no significant correlation between visualization types and low-level analysis tasks [74], as designers often innovate in chart selection. Instead, plotting data characteristics are more closely tied to visualization types, so we exclude visualization types from the consideration. Based on the survey findings, we summarize four primary categories of analysis tasks:

Highlight emphasizes specific values, extremes, proportions, or outliers within a dataset under defined criteria [29]. For instance, displaying “the number of flights departing on a given day” in aviation.

Decomposition can be conceived as the tasks that combine two or more data attribute sets, trying to explore the relationship among them, also defining as correlate. From the examples, this mostly appears when one of the sets is a subset of the other. For instance, this can be applied when “the different profit shares of a given company based on displaying the profitability of all companies” in the field of finance.

Comparison emphasize differences between multiple data attributes sharing similar features or significance, or differences across target objects for the same attribute. For instance, this occurs when analyzing “the number of daytime and nighttime vehicle trips” in transportation.

Overview provides a comprehensive view of datasets covering multiple attributes or complete plotted data, prioritizing completeness and accuracy. Contrary to Highlight task, this method places greater emphasis on data completeness and accuracy. For example, presenting “prizes won by sports teams and their distributions” in sports.

4.3.2 Embellishment↩︎

During the disaggregation process, we identified a significant number of embellishments such as borders, dividers, and icons in the dashboard examples, totaling \(1028\) instances. These elements play a key role in enhancing visual association and improving topic comprehension [75]. Although embellishments are not formally defined in data visualization, they are generally regarded as non-essential graphical components that do not represent data directly but serve aesthetic or metaphorical purposes [76]. To analyze their usage, we categorized embellishments in \(503\) single view examples based on their types. Due to their similar functional roles, borders and dividers were grouped as Borders, while icons were analyzed separately. The dimension includes two primary elements: Borders and Icons.

Borders provide recognizable margins to include all visual elements in views. Apart from aesthetic additions, the elements also indicate the relevance of the charts included and visually guide the audience to see the entire view as a whole.

Icons focus on providing contextual information of the plotting data that help communicate abstract information into concrete mainly for topic comprehension enhancement.

5 System Pipeline↩︎

The DashChat system is designed as a multi-agent pipeline that translates conversational user requests into performance dashboard prototypes (see Fig. 5). The pipeline generates an initial prototype as a reference for iterations and a visual medium for idea exchange, rather than a final product. Drawing insights from the survey in 4, we summarize key features of the performance dashboard derived from real industrial scenarios. The pipeline is implemented with Node.js, backed by Qwen2.5-Max [77]. After piloting several commercial and open-source LLMs, we chose it for its strong instruction-following capability, competitive latency, and favorable unit cost. We present the prompt templates and some technique details in the supplementary materials.

5.1 User Input and Task Creation↩︎

This module forms the foundation of our pipeline, translating inputs into actionable tasks for the LLM-powered agent to generate prototypes (R1). It processes both natural language and interactive selections to formalize user intent.

The flexibility of natural language often results in ambiguous, context-dependent commands (e.g.,“change that chart”), turning intent parsing into a high-risk inference problem. To address this, we introduce Iteration-Oriented Context-Aware Intent Extraction (Fig. 5 (A)), which reframes parsing as a constrained and verifiable generation task. Our approach utilizes a structured prompt that elicits the Chain-of-Thought (CoT) strategy [78]. Our prompt integrates conversational history, a JSON-style domain-specific languages (DSL) (see supplementary materials) of the current dashboard state and the target visualization elements (e.g., element ids), and several few-shot examples. These examples demonstrate not only the target JSON format but also the desired step-by-step reasoning process [79]. It ensures a reliable and predictable foundation for the entire workflow. The module also supports Selection Input (Fig. 5 (B)). It allows users to select predefined templates or widgets, bypassing language ambiguity for rapid and error-free prototype generation.

5.2 Task Planning and Knowledge Integration↩︎

The Task Planning Module organizes and prioritizes tasks generated by the Task Creation Module, ensuring logical and efficient execution. This module addresses two critical difficulties that not only ensure a logical execution order for complex requests, but also infuse the generation process with specialized domain knowledge. We leverage prior experience in dashboard construction to manage task dependencies, isolate independent tasks, and prioritize interdependent ones. Additionally, it integrates Retrieval-Augmented Generation (RAG) [80] by retrieving contextual knowledge from our knowledge base to support task execution. We utilize a text embedding model to embed the dataset mentioned in 4, as the knowledge documents, into vectors, enabling efficient retrieval by the LLM agent.

Task Dependency Analysis and Sequencing: A single user request often decomposes into multiple sub-tasks with inherent dependencies (e.g., a chart must be created before its style can be modified). Executing these tasks in an arbitrary order can lead to logical errors and failed generations. To resolve this, the module first performs a task dependency analysis, modeling the relationships between tasks as a Directed Acyclic Graph (DAG) that defines a fixed execution order (Fig. 6). Actions corresponding to each task, defined as operations necessary for task completion, are added to an action group in sequence, ready for execution. This guarantees a robust and logical workflow, preventing cascading failures and ensuring that each step builds upon a valid prior state. Subsequently, the Task Queue Construction process (Fig. 5 (C)) traverses this DAG to generate a linear, ordered execution queue. This ensures that all prerequisite tasks are completed before their dependent tasks begin, transforming an unordered task set into a structured plan and guaranteeing a robust workflow that prevents cascading failures.

Knowledge Integration (Fig. 5 (D)): To compensate for the lack of the specialized knowledge of data visualization best practices from generic LLMs, we employ Retrieval-Augmented Generation (RAG) strategy (Fig. 5 (D)). We constructed a curated knowledge base from the dataset in 4, containing structured design examples in JSON format. Each example details a description, recommended charts and their analysis task, and style choice. Based on the current task, the agent formulates a semantic query and performs a vector search to retrieve the most relevant ones. The top three retrieved examples are then prepended to the LLM context. This approach grounds the creativity of LLMs in a corpus of proven best practices, ensuring the resulting dashboard prototypes are not only plausible but also professionally sound and aligned with industry standards.

5.3 Task Implementation↩︎

The module is a critical component of our pipeline, executing the ordered workflow generated by the Task Planning Module, ensuring efficient task completion. It processes independent tasks in parallel while respecting dependencies and invoking dependent tasks asynchronously (Fig. 5 (E)). To support this, the module integrates specialized sub-agents for prototype generation: Composition, Assembly and Stylization. The tools use dedicated memory to store and pass intermediate results to dependent tasks, ensuring seamless workflow integration.

5.3.1 Composition↩︎

The Composition Agent is responsible for the generation of individual data visualization components such as charts and KPIs, fulfilling the requirement (R2). Its primary function is to transform a single high-level analytical goal from the task queue into one or more meaningful and structured chart representations based on our DSL.

Data Simulation and Visual Mapping: Generating plausible data is critical for a convincing prototype. Our process mitigates hallucination by separating schema definition from data generation:

• Schema Definition via LLM: Based on the display task, the LLM is prompted to define a data schema in a structured JSON format. For instance, there is a task as “show product category distribution”. This schema specifies column names (e.g., product name, sales volume), data types (e.g., String, Integer), and statistical properties (e.g., a value range for sales volume, a list of possible product name categories). To guide the LLM, the prompt includes few-shot examples and leverages insights from datasets like NvBench [81] to inform realistic data distributions.

• Deterministic Data Generation: With the schema defined, a separate, deterministic script generates the data points according to the specified constraints. This two-step process prevents the LLM from directly fabricating numerical values, significantly reducing the risk of nonsensical or random data. The LLM’s role is confined to creative schema design, while a predictable process handles the generation. Finally, the tool maps the generated data fields to visual encodings (e.g., product name to the x-axis, sales volume to the y-axis).

The output of the Composition Agent is therefore a collection of structured DSL snippets, each representing a fully specified chart ready for the subsequent Assembly stage.

5.3.2 Assembly↩︎

The agent organizes visual elements systematically to ensure clarity, readability, and alignment with user requirements (R3). It integrates Layout Tree Generation and Arrangement and Small Multiple Generation, guided by prior research and empirical data (4).

Layout Tree Generation: The tool constructs a hierarchical layout tree to define spatial relationships between views. Inspired by composition pattern analysis [15], the layout tree is modeled as a multilevel hierarchy. The structure is as follows: \[Layout: = \{\underbrace{view_{1}, [view_{2}, ...,view_{i}], view_{n} }_{\leq 4}\}\] where the two-tuple \(view_{i}:= (type_{i}, [width_{i}, height_{i}])\), symbolizes the main chart type applied in the \(view_{i}\) and the relative width and height of the view in the display screen. Particular, the results of the survey (Fig. 3 (b)) indicate that the layout tree gained from the samples is a 2-level hierarchy where the level-1 node quantity mainly ranges from 2 to 4, implying that the layout designs for industry dashboards are as concise as possible to ensure readability. Therefore, when calculating the layout tree for views, we only consider a tree with a 2-level hierarchy and at most 4 level-1 nodes.

When it comes to considering the pairwise relationships between two view types, we believe that the total number of views in a dashboard influences the information amount generated when the views are selected or laid out. A dashboard with a greater number of views means that the likelihood of a particular chart type being applied to it is more uncertain, which contains more information. Therefore, contrary to the application of conditional probability in the work [15], we apply the concept of information entropy and conditional entropy to represent the relationship: \[\begin{gather} H(type_{i} \mid type_{j}) = \\-\sum_{type_{i}, type_{j}}P(type_{i}, type_{j})log P(type_{i} \mid type_{j}) \end{gather}\] where the value represents the entropy of \(type_{i}\) appears in dashboard when there exist \(view_{j}\) in the screen. Therefore, the lower \(Relation_{ij}\) implies the higher the correlation between \(type_{i}\) and \(view_{j}\) since it represents the lower the uncertainty in the emergence of the chart \(type_{i}\). We define the relationships between a view and node that may contain multiple views from the layout tree at level-1 as: \[Relation_{ij}: = \min_{type_{k} \in node_{j}}H(type_{i} \mid type_{k})\] where \(type_{k}\) represents the various chart types included in \(node_{j}\). Based on the above two calculations, we propose the layout tree algorithm for layout construction for the following arrangement. We present the algorithmic pseudocode for constructing the layout tree in the supplementary materials.

Layout Arrangement: After constructing the layout tree, the tool arranges views in three steps: The tool identifies the most representative view, which holds the largest proportional size, and models the relative position of each view on the screen \(S\) as \(S(view_{i}):=\{p_{i1}S_{1},...,p_{in}S_{n}\}\), where \(p_{ik}\) represents the proportion of overlapping space \(S_{k}\). By comparing positions, the module selects row-based or column-based layouts for level-1 nodes. View sizes are adjusted to reflect their importance, calculated by accumulating the values of the \(p_{i}\) set for all views within a node. The space of each view is allocated proportionally, ensuring size reflects significance. Finally, the tool constructs level-2 nodes by slicing the underlayer orthogonally to the level-1 split, highlighting the hierarchical relationships within the layout tree and reducing the cognitive load on users when interpreting the dashboard. View sizes within level-2 nodes are determined based on common dimensions and task-specific requirements, ensuring visual coherence.

Small Multiple Generation: To support complex analytical narratives, the tool supports small multiples, which are individual views containing multiple visualizations for specific analysis tasks. Based on our survey, we defined four common analytical patterns (Fig. 4 (a)): (i) Comparison views use side-by-side charts with consistent types but distinct colors. (ii) Highlight views incorporate components like number flops or gauges to emphasize key metrics. (iii) Overview views employ tables or comprehensive charts to present a complete dataset. (iv) Decomposition views combine multiple chart types to facilitate hierarchical data exploration.

5.3.3 Stylization↩︎

The Stylization agent enhances the aesthetic appeal and functional clarity of dashboards using color patterns and embellishments. We apply LLMs for color recommendations and Image Generation Models (IGMs) for embellishments, to ensure visually engaging and contextually consistent designs (R4).

Color Recommendation: The agent applies a global color palette, dynamically recommended by the LLM based on user input or thematic keywords. It distinguishes between Categorical Palettes for discrete data and Sequential Palettes for continuous data.

Embellishment Generation: The tool extracts a theme color, translates it into natural language prompts and uses IGMs (Stable Diffusion v3 [82]) to generate design elements including borders and icons. To improve output quality, we fine-tuned the IGM on a custom dataset of dashboard components, creating specialized LoRa models for specific element types (see supplementary materials).

5.4 Result Evaluation and Iterative Adjustment↩︎

This module serves as the final checkpoint in the pipeline, ensuring that the generated performance dashboard prototype meets the expectations and adheres to the quality standards. Here we integrate proactive constraints with post-hoc validation, addressing the difficulty of the creation of factually incorrect or nonsensical content from LLM.

Result Evaluation: As described in the Composition agent, the use of a strict DSL prevents the LLM from generating arbitrary structures or commands. In addition, by separating schema definition from data generation, we prevent the LLM from directly inventing numerical data, a primary source of hallucination. In this agent, before rendering the prototype, it applies automated evaluation on the abstract structure: (i) It validates the generated JSON structure against its schema, ensuring all required fields are present and correctly typed. (ii) The tool (Fig. 5 (F)) checks if the chosen chart type is appropriate for the characteristics of data (e.g., if a line chart is used for non-temporal categorical data). If the evaluation identifies deficiencies, including structural errors or semantic inconsistencies, the pipeline automatically triggers a refinement loop by returning the task to the Planning module with specific feedback for correction.

Prototype Construction (Fig. 5 (G)): Once passing the evaluation, the abstract structure is translated into a real performance dashboard prototype. This tool bridges abstract structure and practical implementation, providing users with a realistic preview of the final product.

Human-in-the-Loop Validation and Iterative Adjustment: The pipeline is designed for iterative collaboration, not one-shot generation (Fig. 5 (H)). The generated prototype is a proposal for the designer to inspect. The designer, as a domain expert, is best equipped to identify subtle hallucinations, such as fabricated metrics that look plausible but are contextually meaningless, or inappropriate visual metaphors. The iterative adjustment process (R6) empowers the user to correct these issues conversationally, ensuring the final prototype is both structurally sound and semantically valid.

6 Interface and Case Study↩︎

DashChat is designed to facilitate seamless interaction between users and the system, enabling efficient and intuitive generation and modification of prototypes. The interface consists of two primary panels: Chat Panel and History Panel. Each view is crafted to support a natural language-driven workflow, provide real-time feedback, and allow users to iteratively refine their designs (R6). The interface is written in TypeScript. To illustrate the practical application and workflow of DashChat, this section presents a detailed case study following a designer, Nemo, as he undertakes the task of creating a high-fidelity prototype for a manufacturing monitoring dashboard. The scenario involves a tight deadline and evolving requirements, conditions under which the efficiency and iterative capabilities of DashChat are particularly valuable.

6.1 Phase 1: Initial Prototype Generation↩︎

Upon initiating a new project in DashChat, Nemo is presented with the Chat Panel. Faced with the initial “blank canvas” problem, he immediately benefits from the Content Suggestion Bubbles (Fig. 7 (a1)) displayed above the chat input area. Instead of formulating a complex initial prompt, Nemo selects a predefined option that aligns with his project domain: “Tobacco Industry Supply Chain”.

This single click serves as the initial command. The system processes the request and, within seconds, generates a Prototype Display Bubble in the chat interface (Fig. 7 (a2)). This bubble contains a foundational dashboard prototype, complete with relevant data visualization components such as Key Performance Indicator cards, a line chart for production trends, and a status overview for different assembly lines. This initial step effectively provides a structured starting point, allowing Nemo to bypass the time-consuming process of building a basic layout from scratch (R3).

6.2 Phase 2: Iterative Design Refinement↩︎

With a baseline prototype established, Nemo proceeds with a series of iterative refinements using a combination of natural language commands and guided interactive elements (R1, R6).

6.2.1 Layout Modification:↩︎

The default two-column layout is not optimal for the intended display environment. Nemo seeks to change this. He clicks on the “Modify Layout” interactive button located within Prototype Display Bubble (Fig. 7 (a2)). This action triggers the appearance of a Layout Selection Bubble (Fig. 7 (a3)), which presents several predefined grid and column templates. Nemo selects a three-column layout. The system instantly re-renders the prototype, automatically and intelligently repositioning the existing chart views into the new layout structure, demonstrating the capability for rapid structural adaptation (R3).

6.2.2 Aesthetic Enhancement:↩︎

To align the prototype with the client’s branding, which calls for a dark, high-tech aesthetic, Nemo interacts with the “Modify Design Style” button. This opens the Design Style Recommendation Bubble (Fig. 7 (a4)). He is presented with a gallery of visual style previews, such as “Serene Mountain Blue,” “Alpine Cloud White,” and “Gemstone Green.” After previewing the options, he selects “Serene Mountain Blue.” The system applies the style globally, updating the background color, font styles, and chart color patterns across the entire prototype in real-time, showcasing an efficient method for exploring and applying visual themes (R4).

6.2.3 Content Specificity and Addition:↩︎

Nemo thinks that the prototype now requires more specific content. First, a “Production Volume Pie Chart” needs to be changed to represent “Defect Rate by Production Line”. Nemo clicks the “Modify Content” button, which activates the Content Modification Bubble (Fig. 7 (a5)). This bubble displays an interactive outline of all chart views in the prototype. Nemo locates the node corresponding to the pie chart and directly edits its title to “Defect Rate by Production Line as a bar chart”. The system interprets this change, replacing the pie chart with a bar chart populated with placeholder data relevant to defect rates. This demonstrates a structured and error-reducing method for content editing (R2).

Second, Nemo need to add a new metric, “Overall Equipment Effectiveness”. For this, Nemo utilizes the Chat Box (Fig. 7 (b)), typing the natural language command: “Add a radar chart in the bottom right section showing Overall Equipment Effectiveness”. The system parses this instruction, identifies the target location and chart type, and generates a new Prototype Display Bubble with the updated design, which now includes the requested radar chart. This free-form input highlights the conversational interaction capabilities of our system (R1, R6).

6.3 Phase 3: Managing Design History and Reversion↩︎

During a review, a stakeholder expresses a preference for a layout that was explored earlier in the session. Instead of attempting to recall the exact previous configuration, Nemo utilizes the Additional Interaction Options in the top-right corner (Fig. 7 (d)) to toggle the visibility of the History Panel.

The History Panel displays a chronological sequence of Prototype Generation Record Cards (Fig. 7 (e)). Each card contains a thumbnail preview of the prototype at that stage, a timestamp, and the specific modification that was performed (e.g.,“Modified layout”, “Text-Based Generation”). Nemo quickly scrolls through this visual timeline, identifies the card corresponding to the desired layout, and clicks on it. This action allows him to either revert to this previous state or use it as a new starting point for a different design branch. This feature provides a comprehensive audit trail and robust version control, critical for a non-linear design process (R6).

This case study of Nemo’s workflow demonstrates how DashChat integrates multiple interaction modalities into a cohesive and efficient design experience. The system successfully guided the user from a vague initial concept to a detailed prototype through a continuous dialogue. By combining guided interactions via suggestion bubbles and interactive tags with the flexibility of natural language commands, DashChat significantly streamlines the generation and modification of dashboard prototypes. The History Panel further enhances this process by providing a safety net and a clear record of the design’s evolution, ultimately transforming a traditionally fragmented process into a fluid and iterative conversation between the designer and the system.

7 Evaluation↩︎

In this section, we conduct a quantitative experiment to evaluate our pipeline and a user study to assess the usability.

7.1 Pipeline Evaluation↩︎

The evaluation quantitatively measures the accuracy and robustness of the DashChat pipeline, with a specific highlight on system Accuracy and hallucination mitigation. Our primary goal is to demonstrate the value of our multi-agent architecture compared to simpler, more direct LLM-based approaches.

7.1.1 Experiment Setup↩︎

We utilized a manually curated test set of 50 prompts, which is distinct from any examples used in few-shot prompting mentioned in 5, to evaluate the performance of DashChat and the baseline methods. The test set comprises 50 standard prompts covering various assigned intent tasks. For each prompt, a corresponding ground truth DSL output was manually authored by a human expert. We compare our full DashChat pipeline against two baselines:

• Zero-Shot LLM: A large language model (Qwen2.5-Max) prompted with only the task description and DSL format, without our structured prompting, RAG, or iterative pipeline. This baseline isolates the contribution of our agentic architecture.

• Tool Call LLM: A baseline where the LLM (Qwen2.5-Max) utilizes a single, monolithic function calling API to populate all DSL attributes. The task for LLM is to parse the prompt requirement from users and fill in the arguments for this single tool call. This baseline represents a more advanced but still non-agentic approach.

7.1.2 Evaluation Metrics↩︎

To provide a comprehensive assessment, we measured metrics across three categories: viability, consistency, and efficiency.

Viability: We first report the Execution Success Rate, as the measure of viability. This metric measures the percentage of generated DSL specifications that can be successfully parsed and rendered without any runtime errors. This is a standard practice in evaluating code generation systems [83] and serves as a prerequisite for further consistency analysis.

Consistency: We measured Consistency to assess strict accuracy of standard prompt generation. A generated result is defined as consistent if it is identical to the ground truth specification across all key design attributes, including the chart types, their detailed axis characteristics and a detailed layout structure or color pattern suggestions. This metric serves as the most stringent measure of correctness, rewarding only those outputs that perfectly match the single, expert-authored ground truth. While it provides a clear signal of precision, its strictness may not fully account for the inherent ambiguity in natural language.

Efficiency: To directly evaluate our design requirement for fast prototyping, we measured Efficiency. This was quantified as the generation speed, defined as the end-to-end wall-clock latency in seconds, from prompt submission to the return of the final validated DSL specification.

7.1.3 Results↩︎

Our evaluation results are presented in Fig. 8, which indicates the performance of DashChat against the Zero-Shot LLM and Single Tool Call LLM baselines across our three metrics. The results indicate that DashChat significantly outperforms both baseline approaches in terms of viability, consistency, and efficiency. Notably, DashChat achieved perfect viability (100%), the highest consistency (94%), and the fastest generation speed (41.4s), demonstrating the effectiveness of our agentic architecture.

Comparison with the Baselines. Looking through the tested cases, two key factors stemming from our system’s architecture account for the performance differences. The first is superior semantic understanding and reasoning. Unlike the single-pass baselines, our multi-agent pipeline decomposes the problem. The structured intent parsing and the agent planning RAG-infused process enable a more robust interpretation of user utterances, especially those with abstract or omitted information (e.g., inferring an appropriate chart type from a task description). This directly contributes to DashChat’s higher consistency score (94%) compared to the Tool Call LLM (80%) and the Zero-Shot LLM (76%). The second factor is the explicit validation loop. DashChat’s 100% viability rate (50/50 successful executions) is a direct result of its Evaluation agent, which catches and corrects syntactically invalid or unexecutable specifications before finalizing the output. The baselines, lacking this mechanism, frequently failed, with the Zero-Shot LLM’s viability dropping to as low as 72%.

Metric Difference and Efficiency Analysis. An important observation is the relationship between the metrics. For DashChat, the consistency metric (94%) is slightly lower than its perfect viability score (100%), a difference that can be attributed to two main factors. First, ambiguity in abstract utterances often results in multiple reasonable answers. For instance, a prompt like "show sales distribution" might have a bar chart as its ground truth, but a generated pie chart would still be a viable and functionally correct response, thus passing the viability check but failing the stricter consistency test. Second, partially correct inferences occur when the model correctly generates the core visualization but misses a subtle, non-critical detail from the ground truth, such as a specific sorting order or a minor wording difference in a title. In terms of efficiency, DashChat’s superior generation speed (41.4s) compared to the Tool Call (62.9s) and especially the Zero-Shot LLM (134.7s) may seem counterintuitive for a multi-step pipeline. We attribute this to the fact that the baselines’ single, monolithic prompts are significantly more complex and computationally demanding for the LLM to process in one go. In contrast, DashChat’s smaller, focused prompts for each agent are individually more efficient, and the overall orchestration results in a faster end-to-end response time.

7.2 User Study↩︎

To comprehensively evaluate our system, we conducted a user study involving two distinct groups of participants: domain professionals and designers. It allowed us to assess both the system’s effectiveness in producing contextually relevant prototypes and usability as an authoring tool.

7.2.1 Participants↩︎

We recruited qualified experts by sending invitations via email and advertised the study on social media platforms for additional participants. Our user study adhered to ethical standards, ensuring informed consent and data privacy while paying special attention to participants’ well-being.

Group of Professionals (P). A total of 17 participants aged between 21 and 36 years old (\(M = 26.12,~SD = 3.35\)) were recruited, representing a diverse range of professional backgrounds (denoted as P1-P17). Among them, 6 identified themselves as male and 11 as female. Their background included applied statistics (3), biomedical engineering (2), finance (1), linguistics (1), communication engineering (2), manufacture (2), aviation (1), microelectronics (2), and international politics (3). They are highly experienced in their respective domains with over two years of work experience (\(M = 4.12,~SD = 2.18\)) and possess a certain level of knowledge in data visualization.

Group of Designers (D). We recruited 11 experienced designers (7 females and 4 males), our target users, to operate our system (D1-D11). The participants ranged in age from 24 to 41 (\(M = 31.18,~SD = 5.23\)) and had more than two years of professional experience (\(M = 6.73,~SD = 4.49\)). 7 participants with prior experience in data visualization authoring tools (e.g., Tableau [12], Flourish [13]) and 4 without any relevant experience.

7.2.2 Procedure↩︎

We conducted an in-person experiment for each participant. The study began with a 15-minute session where we introduced the system’s concept, functionalities, and interaction methods. Following the introduction, participants engaged in tasks tailored to their respective groups:

Tasks for Group P: Participants were asked to propose a set of requirements for a performance dashboard based on their own domain knowledge. They then used our system to generate a prototype by providing textual prompts. The process was iterative, simulating a real-world negotiation scenario where they provided feedback to refine the prototype until they were satisfied. Their primary goal was to achieve a satisfactory and useful visual dashboard for their domain needs.

Tasks for Group D: Participants were given a series of structured tasks. The structured tasks required them to (1) generate a prototype for a given scenario (e.g., tobacco industry supply chain), and (2) systematically modify a prototype’s views, layout, and style using both text and button interactions. In addition, they were asked to perform a within-study on the usability and effectiveness of the system against a state-of-the-art commercial tool, Tableau [12]. After a 30-minute tutorial on Tableau, each designer was asked to complete the same task of creating a dashboard prototype. The order of the tools was counterbalanced to mitigate learning effects.

After completing their tasks, all participants were required to complete the questionnaire and participate in a semi-structured interview to provide detailed feedback.

7.2.3 Measurements↩︎

Our evaluation aims to capture a holistic view of the user experience. A unified questionnaire (Q1-6), administered to all 28 participants, was first used to quantitatively assess the system against six fundamental metrics on a 5-point Likert scale (1 for “strongly disagree”, 5 for “strongly agree”). Especially, the designers need to assess both DashChat and Tableau:

Usefulness: The degree to which the system is perceived as being able to help users achieve their goals.

Effectiveness: The accuracy and completeness with which users can achieve their goals.

Satisfaction: The overall contentment and pleasantness of the user experience.

Desirability: The aesthetic appeal and attractiveness of the system and its output.

Ease of Use: The degree to which the system can be used without unnecessary effort.

Ease of Learning: The ease with which users can become proficient with the system.

While these core metrics provided a common baseline for all participants, our evaluation was further differentiated based on user type through tailored questions. The details of our questionnaire can be found in the supplementary material.

For Group P: The evaluation centered on the outcome and communicative value of the generated prototypes (metrics for prototype generation). We focused on how well the system understood domain-specific requirements and translated them into meaningful visuals (related to R2, R3, R4). 6 interview questions (Q7-12) probed the quality of data simulation, the relevance of chart choices, and the prototype’s potential to accelerate communication in their professional context.

For Group D: The evaluation centered on the authoring process and creative workflow (metrics for interface usability). We focused on the system’s efficiency, flexibility, and its integration into a designer’s toolkit (related to R1, R5, R6). 6 interview questions (Q13-18) probed the efficiency of the text-to-prototype pipeline, the intuitiveness of the hybrid interaction model, and how the tool compared to traditional design software.

7.2.4 Result and Findings↩︎

Fig. 9 and Fig. 10 depict the quantitative results of the questionnaire, which indicate that our system was regarded favorably by participants. We organize the feedback in detail.

Overall Positive User Experience (N=28). Across all 28 participants, the system scored highly on the first six metrics, confirming its usability. It received exceptionally high ratings for Effectiveness (\(M=4.68, SD=0.47\)) and Ease of Learning (\(M=4.46, SD=0.51\)), indicating that users not only successfully achieved their goals but also found the system quick to master. Usefulness (\(M=4.18, SD=0.72\)), Ease of Use (\(M=4.21, SD=0.80\)), and Desirability (\(M=4.14, SD=0.76\)) also received strong positive feedback, demonstrating that participants found the tool valuable, effortless to operate, and aesthetically pleasing. The overall Satisfaction (\(M=3.79, SD=0.96\)) was also positive, though exhibiting slightly more variance, reflecting the nuanced and specific feedback provided by different user groups.

Superior Usability and Effectiveness. As shown in Fig. 10, a paired t-test revealed that DashChat received significantly higher scores than Tableau. The advantages were most pronounced for Effectiveness (\(M=4.82\) vs. \(M=2.64\); \(p < .001\)), Ease of Learning (\(M=4.64\) vs. \(M=3.36\); \(p < .001\)), and Ease of Use (\(M=4.36\) vs. \(M=3.27\); \(p < .01\)). DashChat also scored significantly higher on Desirability (\(p < .05\)) and Usefulness (\(p < .05\)). No significant difference was found for Satisfaction (\(p = .31\)), suggesting that while both tools can produce a satisfactory final product, the process to get there is markedly different. The interview feedback clearly explained the result. D5 noted, “With DashChat, I described the dashboard and got a full, multi-chart layout in under a minute. With Tableau, I had to start from a blank canvas, manually drag each data field, and build each chart one by one.” This directly explains the dramatic difference in Effectiveness. Similarly, D8 highlighted the gentler learning curve: “Tableau forced me to think about dimensions versus measures. DashChat just understood ‘sales trends’ and did the right thing. It felt like a conversation, not a technical task,” which accounts for the higher Ease of Use and Ease of Learning scores.

High-quality generated prototypes. We found that participants were generally positive about the generated prototypes. First, participants expressed general satisfaction with the chart generation (R2). P9 commented that “the simulated data was surprisingly realistic.” P9, from the manufacturing domain, requested a prototype focused on production efficiency across factories. In response, the system automatically simulated output data for morning, afternoon, and night shifts at each factory, information not explicitly mentioned in the input. “This shift-based data is exactly what we monitor in our daily operations,” indicating that the system’s proactive data simulation aligned closely with real-world industry practices and user expectations. However, P3 and P17 felt this functionality could be further improved, particularly when LLMs failed to interpret certain domain-specific acronyms correctly. P17 further suggested adding the web search to enhance data simulation accuracy and relevance. Moreover, reasonable chart-type selections greatly supported effective data communication. P16 remarked, “I was surprised that the charts matched what I had in mind. It used a line chart to show trends over time, saving me from explaining every detail.”

Regarding layout (R3), most participants appreciated the system to organize multiple views coherently and informally. P6 stated, “The layout looks clean and pleasing, balancing visual density and whitespace.” P2, P3, and P11 favored how the system automatically prioritized key views based on their intent. They remarked that “the layout of the dashboard views was well-organized, allowing me to quickly locate the key metrics without visual clutter.” P11, a communication engineering professional, requested emphasis on base station counts across Chinese provinces. The system responded with a geographic heatmap, centrally positioned and occupying the largest screen area. P15 suggested improving the layout by placing semantically related views closer to enhance interpretation and comparison. Aesthetically, the visual style was also well received (R4), with P8 and P9 commenting that “the design looks professional and polished, and the color scheme enhances readability without being overwhelming.” The incorporated stylistic elements enhanced the aesthetic sophistication, “I liked how the system added subtle design touches, like borders, icons, and animations.”

Accelerated negotiation process. Many participants emphasized that the system substantially enhances communication efficiency, particularly in the early design stages. “Seeing the visuals update instantly based on my feedback makes me feel like we were all on the same page much faster.” By quickly transforming high-level textual requirements into concrete visual prototypes, our system served as a shared reference point that reduced ambiguity and accelerated the alignment of expectations. P9 remarked, “It helped me communicate my ideas without needing to draw anything or explain every detail. I could just react to what I saw.” P6 added, “Normally, it takes multiple meetings to get the visual right, but with this system, we aligned in just a few iterations.”

Effective and efficient authoring process. Participants consistently praised the system for significantly streamlining the construction of prototypes, reducing both design complexity and turnaround time (R1, R5). D2 and D5 noted, “Compared to traditional performance dashboard construction, the 1-minute generation speed is significantly faster.”, illustrating a substantial speed improvement compared to traditional methods. D6 also expressed satisfaction, “It allows me to quickly build a prototype, enabling face-to-face communication with clients to showcase design details. If clients request sudden changes, the system responds promptly with updated outputs, streamlining communication and reducing turnaround time.” It also aligns with the findings from our previous user study. Besides that, participants are positive that the system bridges the gap between abstract textual input and concrete visual output. D7 commented, “The generated prototype closely aligned with the textual requirements, with minimal discrepancies.” The simulated data functionality was appreciated by participants, “it helps designers without a strong data background to save the time that would otherwise be spent creating placeholder data manually.”. Additionally, D8 noted that the prototypes demonstrated a high level of visual fidelity, including well-balanced layouts, appropriate color schemes, and realistic chart compositions, “The generated output closely resembles real-world performance dashboards in terms of visual appearance”

User-friendly interactions. Our system received positive feedback for its interactive flexibility, particularly in supporting both iterative refinement and direct manipulation (R6). The combination of the Q&A system and in-bubble button allows users to freely choose their preferred interaction mode, enhancing overall flexibility and user autonomy. D4 commented, “The chat bubble interaction design is particularly appealing, as it offers a more efficient way to operate.” The ability to provide shortcuts for common modifications was also highly valued. D3 and D6 noted that shortcuts facilitate faster text input and additionally serve as a source of creative inspiration. Additionally, some participants provided detailed suggestions for refining our interaction design. D4 suggested, “Sometimes I just want to tweak one color without changing the whole palette. Having that option would make the system feel more customizable.” D9 and D10 suggested, “Direct on-prototype editing would further improve usability.”

8 Discussion↩︎

Implications and generalization of DashChat. Our work demonstrates how multi-agent interaction powered by LLMs can be effectively harnessed to support performance dashboard authoring, a task that traditionally requires both domain expertise and design proficiency. DashChat supports a more inclusive and collaborative design process between humans and AI by enabling users to construct high-quality dashboard prototypes through natural language. By distributing sub-tasks across multiple specialized agents running in parallel, our pipeline offers a new approach to improving the responsiveness of LLM-powered multi-agent workflows. This parallelism mitigates the latency commonly associated with sequential LLM pipelines, which provides a scalable solution for complex generation tasks. Beyond the domain, our system’s underlying architecture and interaction paradigm exhibit strong potential for generalization across broader domains of LLM-assisted visual design. We structure the design space as a knowledge base to guide LLM-based generation, define task-specific automation pipelines based on user intent, and design interaction mechanisms that support user-centered refinement. The methodology offers a generalizable blueprint for combining LLMs with domain knowledge and interaction design, applicable to a wide range of visual communication tasks.

Creativity and robustness of LLMs. LLMs are powerful tools for bridging natural language and visual representations, allowing even non-experts to generate meaningful visualizations through text-based interaction. This capability lowers the barrier to entry for data storytelling and democratizes access to visualization tools by translating human intent into structured visual output. However, the application of LLMs in visualization reveals a fundamental tension between creativity (generative diversity) and robustness (output stability). Emphasizing creativity allows for diverse and potentially novel visual representations, opening new avenues for expressive and personalized design, but unconstrained generation may violate core visualization design principles. Conversely, overly deterministic strategies (e.g., relying solely on fixed templates) may preserve correctness but suppress innovation, which results in homogeneous outputs that lack adaptability to specific user contexts or domain narratives. Because LLMs generate visual descriptions based on probabilistic language patterns rather than grounded reasoning or formal semantics, they often lack the logical rigor required in data visualization. Effective generation must integrate knowledge from data science, visual design, and domain expertise. In this work, we constrained the generative space with a validated knowledge base, leveraging trusted templates and refining them based on user preferences. The conflict can be mitigated by designing such hybrid frameworks carefully in the short term. Resolving this tension in the long term may require paradigm-level shifts of LLMs, such as unifying statistical learning with logical reasoning [84], [85].

Limitations and future work. First, our system has room for improvement in task composition and data simulation. The first user study shows this limitation may cause mismatches between simulated data and user intent, reducing the output’s utility and credibility. The ability could be enhanced by integrating external knowledge sources via techniques such as web search or RAG [80]. These approaches enable LLMs to access up-to-date, domain-specific knowledge beyond their pre-trained parameters, allowing for more accurate and context-aware responses. Second, we mainly focus on the global palette recommendation and embellishment generation centering on a color-related design style. Broader design dimensions, such as typography and iconography, should be considered to enhance stylistic richness. Incorporating the additional elements would allow for richer stylistic expression, better alignment with user preferences and domain contexts, and greater adaptability across different application scenarios.

9 Conclusion↩︎

This work presents DashChat, an interactive system that leverages LLMs to generate performance dashboard design prototypes from natural language. We first summarized their common design patterns by analyzing high-quality performance dashboards. Next, we built a multi-agent pipeline powered by LLMs to generate practical, aesthetic prototypes. Then, we developed a user-friendly interface that supports text-based interaction. Last, we evaluated DashChat via a quantitative experiment and a user study, which showed that DashChat can generate high-quality prototypes rapidly while offering a user-friendly interactive experience.

References↩︎

---

1. Ziyue Lin, Siming Chen, Xingyu Lan are with Fudan University, China. (e-mail: ziyuelin917@gmail.com; simingchen@fudan.edu.cn; xingyulan96@gmail.com).↩︎

2. Siqi Shen, Honghui Mei, Wanchen Liu, Chengye Xin, Wenzhuo Dai, Xiao Wen are with DataV Lab, Alibaba Group, Hangzhou, China. (e-mail: shensiqi.ssq@alibaba-inc.com; honghui.mhh@alibaba-inc.com; liuwanchen.lwc@alibaba-inc.com; mier.xcy@alibaba-inc.com; basi.dwz@alibaba-inc.com; ninglang.wx@alibaba-inc.com).↩︎

3. Ziyue Lin and Siqi Shen contributed equally to this research. Xiao Wen and Xingyu Lan are the corresponding authors.↩︎