1 Introduction↩︎

Artificial Intelligence (AI) -powered tools, and AI coding assistants specifically, are rapidly becoming part of everyday software development. From smart autocompletion and code generation to conversational tools that help with debugging, refactoring, or documentation, developers now use these systems at nearly every stage of the workflow: from early ideation to testing and maintenance. Surveys show that most developers have already integrated AI tools into their daily work [1], [2]. As these tools become more embedded in development practice, expectations are changing. Rather than serving as one-size-fits-all helpers, there is a growing expectation for personalization and configurability [3]. Developers now want tools that reflect their own working styles, preferred technologies, team practices, and even communication tone. Some care about how autonomous or directive the assistant feels, others prefer a particular level of verbosity or humanized behavior.

While interest in configurable coding assistants continues to grow, the actual configuration options available today remain limited. Mainstream tools like GitHub Copilot [4] and Cursor [5], for instance, offer impressive code completion capabilities but provide little control over aspects such as interaction style. More broadly, discussions around configurable AI suggest that major vendors are beginning to treat it as a strategic priority. Yet when it comes to coding assistants in particular, we still know relatively little about what kinds of configurations developers actually want and how they prefer to control them.

Our paper seeks to address the current gap through structured user research with experienced software developers. We compile a list of 33 configurations (aka "controls") that currently exist in the most popular coding AI assistants, or were suggested in personalization and coding personalization research. We categorize the identified configurations into broader themes based on what they characterize: Code suggestions, System & policies, Human-Assistant interaction, and Users & their personal context. This categorization helps us to uncover which aspects of configurability developers value most and what motivates these preferences, whether it is trust calibration, workflow alignment, or organizational policy. We also explore how developers wish to express control, by examining preferred interface forms such as toggles, checklists, configuration files, or natural language commands.

Specifically, we aim at answering the following Research Questions (RQs):

• What configurations do experienced developers want in AI-assisted coding tools?

• How do developers prioritize different types of configuration needs?

• What types of configuration interfaces do developers prefer to control their AI-assisted coding tools?

To answer these questions, we conduct a two-stage study. First, we survey developers to identify and rank the configuration options they care about most. Then, we run 7 co-design sessions and interviews to dig deeper into developers’ preferred settings, interface ideas, motivations, and constraints. Our findings suggest that developers primarily value configurations that shape what the assistant outputs and how it behaves in context, rather than those that set up rich personal profiles. They also clearly distinguish between stable, project-level defaults and quick, task-specific tweaks, and that distinction influences both how often they adjust a setting and where they expect to find it in the IDE.

2 Background↩︎

2.1 Industry overview↩︎

AI-powered coding assistants are now a routine part of modern software development [1]. Tools such as GitHub Copilot [4], OpenAI ChatGPT [6], Google Gemini [7], and JetBrains AI Assistant [8] are widely used for code completion, documentation, testing, debugging, and other tasks. Newer platforms, such as Cursor [5], are also emerging, following the direction of agent-mode workflows and offering more integrated AI support across the software engineering lifecycle. In this section, we overview the current state of configurability in five of the most popular coding AI tools according to the State of Developer Ecosystem in 2025 Report [1].

GitHub Copilot. [4] Copilot offers configuration options spread across GitHub account settings, IDE settings, and project artifacts such as instruction and prompt files. At the account level, users and administrators can enable or disable features (e.g., chat, coding agent, code review, web search), select models, and control data usage. Within the IDE, developers can select a preferred language and adjust how aggressively it edits code, while project-, path-, and agent-specific instruction files configure coding conventions, frameworks, testing practices, and security rules. Organizations can add higher-level instructions and content-exclusion rules that apply across repositories, and reusable prompt templates and custom instructions shape how Copilot responds in chat. Overall, Copilot offers many controls over its behavior, but they are spread across multiple configuration interfaces and are mostly presented as simple toggles or textual prompts.

OpenAI ChatGPT [6]. ChatGPT offers configuration options across account-level settings, per-chat controls, project workspaces, custom GPT definitions, and workspace administration panels. Users can define global custom instructions, enable or disable long-term memory and chat history, and set data-usage and training preferences. Within a chat, they select models and modes and provide chat-specific instructions, while projects group related chats and reference files under project-specific instructions. Custom GPTs combine instructions, knowledge files, tools, and sharing settings into reusable assistant profiles, and organizational plans add controls for feature availability, connectors, and data retention. Overall, ChatGPT’s configurations are relatively focused and easier to locate, since global options are centralized and scope-specific options are attached to the corresponding level, but it offers a more limited set of configurations and does not provide code-specific settings.

Google Gemini [7]. Gemini Code Assist offers configuration options across IDE extension settings, repository-level configuration files, and Google Cloud project settings. In the IDE, developers can enable or disable inline suggestions, choose supported languages, adjust how and when suggestions appear, and configure local codebase awareness and agent behavior, while repository-level artifacts control which files and patterns should be ignored. In the Enterprise edition, administrators can connect customization indexes to private repositories so that suggestions draw on internal libraries and standards. Within the chat interface developers can define custom commands and rules for common workflows. Overall, Gemini’s coding assistant distributes its configurations across several interfaces and, although it provides a range of code- and system-level configurations, offers little control over end-user personalization and interaction style.

JetBrains AI Assistant [8]. JetBrains AI Assistant offers configuration options across per-project and per-chat IDE settings, repository-level files, and organization-level AI Enterprise controls. In the IDE, developers can specify where Assistant may modify files without confirmation, choose languages and technologies, adjust disruptive behavior, and connect MCP servers, while quick settings in the chat window expose Ask/Code/Auto modes, a Brave Mode checkbox (autonomy in command execution and file modification), and a model picker. At the project level, guidelines files provide reusable context about coding conventions, frameworks, testing practices, and examples, while ignore files restrict Assistant’s freedom of action. Organizational deployments configure which providers and models are available, whether detailed interaction data is collected, and which users can access AI features. Overall, Assistant’s configurations are concentrated in IDE and project configuration and, while it offers rich options for project scope and code-style guidance, it provides limited support for user-level personalization or cross-project personas.

Cursor [5]. Cursor’s AI-enabled editor offers configuration options across IDE settings, project-level rule and ignore files, and an online team dashboard. In the IDE, developers can define global user rules for preferred languages, frameworks, code structure, and documentation style, which are applied to chat and inline edits across projects. Project-level configuration files define repository-specific standards and workflows, while ignore files control which files are indexed and used as context. System-level controls handle model selection, context size, privacy modes, and across-session memorization, and team dashboards manage enforced privacy settings. Overall, Cursor has notable code- and system-level control settings but more limited user-personal and task-context controls that are mostly configured indirectly through prompts.

In conclusion. Despite growing adoption, the configuration options these tools offer remain fairly limited. Some allow developers to specify preferred languages or frameworks, while others provide basic system settings, e.g., chat memory depth and output size. However, few offer fine-grained control over the whole range of personalization, including aspects such as behavior, interaction style, company policies, or even color schemes.

Another challenge in today’s assistants’ configuration approach is that the settings that do exist are scattered across multiple poorly connected interfaces. Developers might find some settings at the account level (e.g., data retention, memory preferences), others buried in IDE-specific menus (e.g., toggles for inline completions and language activation), and still others in project files (e.g., ignore lists that define what the assistant can access) or ad hoc textual prompts and instruction templates.

The result is a fragmented and often confusing configuration experience. In our survey, a number of developers report being unaware that certain configurations exist (3.3). This reflects a wider trend in modern AI systems, where essential functionality becomes distributed across heterogeneous interfaces, making it difficult to discover or manage in a coherent way [9]. Configuring the tool becomes cognitively demanding and error-prone rather than straightforward, and this scattered setup makes it hard to build a clear mental model of how the assistant works, which in turn makes it harder to trust or predict its behavior [10].

Given these challenges, it becomes increasingly important to offer developers configuration mechanisms that are clear, unified, and easy to find. Prior work in human-AI interaction highlights that user control, predictability, and transparency are key for building and maintaining appropriate trust [10], [11]. As AI assistants take on a greater share of daily development work [1], [2], [11], developers need reliable ways to influence how these systems behave, what constraints they follow, and how they integrate into personal and organizational workflows.

2.2 Research overview↩︎

Personalized coding assistants hold substantial potential to increase productivity and trust by aligning with developers needs and expectations [12], [13]. However, implementing automatic personalization poses notable challenges. According to  [14], it requires access to sensitive information, such as private repositories, usage history, or organizational resources, raising concerns about privacy, data retention, and security. Besides that, complex development workflows, distributed teams, and interdependent codebases might complicate context modeling, limiting the assistant’s ability to reliably generate context-aware suggestions. Personalization techniques such as content-based modeling or collaborative filtering may fail in the presence of cold-start scenarios or diverse preference profiles. That is why many current LLMs mostly rely on prompt-controlled configuration or user-provided feedback for adaptation.

As we described in the previous subsection, most existing assistants offer limited and non-obvious mechanisms for adjusting how they operate, what conventions they follow, or how they interact with the user. This one-size-fits-all design has well-documented drawbacks. Prior work shows that misalignment between a developer’s expectations and the assistant’s behavior can reduce trust, slow adoption, and introduce friction into the development workflow [12], [13], [15]. Research in human–AI interaction consistently highlights the importance of adaptability, arguing that AI systems must support users’ goals, context, boundaries, and need for control in order to function effectively [3], [16].

Prompting is currently one of the most common forms of control over assistant behavior. According to  [17], developers are moving from short-lived, task-specific instructions toward persistent, project-level prompts that are expected to be followed across interactions with the same codebase. They illustrate this evolution by analyzing 401 open-source repositories with Cursor rules and developing a taxonomy of developer-provided context and directives. Their findings suggest that developers already treat such rules as a new configuration surface, using them to encode project norms and constrain assistant behavior beyond individual queries.

For developers, adaptability includes not only technical considerations, such as preferred libraries, frameworks, and code style, but also workflow habits, communication preferences, privacy expectations, and organizational constraints. Having the ability to configure stylistic conventions such as naming schemes, formatting rules, testing structures, or documentation style can help ensure that AI-generated code fits seamlessly into existing codebases and speed up the AI-generation process [13], [15], [16]. Similarly, studies of goal-setting and intent-specification interfaces show that allowing users to establish constraints or objectives improves predictability and mutual understanding between humans and AI systems [18], [19]. Developers also value control over their interaction with AI, such as timing, verbosity, colors and on-screen positioning,  [3], [15].

Despite these insights, existing research stops short of identifying which specific configurations developers actually want, why they want them, or how they want to set them up. This gap is especially noticeable when we look at the general picture and not specific modality, e.g. generated code properties. In this work, we want to address the gap through structured user research with experienced software developers and outline design and research opportunities for Configurable Coding AI.

2.3 Compiling a list of configurations↩︎

To study developers’ configuration needs in a structured way, we first need an initial, reasonably comprehensive list of configuration options for AI coding assistants. This list serves as the basis for our survey and co-design activities, which participants are encouraged to extend with their own suggestions.

We create this list in two steps. First, we systematically review the configuration settings offered by existing coding assistants, including GitHub Copilot, ChatGPT, Google Gemini, JetBrains AI Assistant, and Cursor (2.1). Second, we extend these practical configurations with ideas inspired by prior research on configurable and personalized AI, and trust in AI-powered developer tools (2.2). This allows us to include not only currently implemented options, but also plausible forward-looking configuration mechanisms that have been proposed in the literature.

We organize our list of configurations into four higher-level groups that reflect what each setting primarily characterizes. These groups are built in line with prior work on trust in AI-assisted development. Brown et al. [12] identify three types of characteristics that influence trust in AI code completion tools: properties of the suggestion itself, characteristics of the developer, and aspects of the development context. The PICSE framework for trust in software tools by Johnson et al. [16] groups relevant factors into five categories: Personal, Interaction, Control, System, and Expectations. Inspired by these perspectives, we organize our list of configurations into the following groups: Characteristics of Code suggestions, Characteristics of System & policies, Characteristics of Human–Assistant interaction, and Characteristics of Users & their personal context.

Our proposed grouping serves two purposes. Analytically, it provides a manageable structure for comparing how developers prioritize different kinds of configurations, e.g., whether they care more about shaping code suggestions or aligning the assistant with their personal context. Methodologically, it helps us design clear survey blocks, so that participants would think about related configuration options together rather than confronting an undifferentiated list. Below, we briefly define each group and list the 33 configuration options that we consider in our study.

Characteristics of Code suggestions (CS).

These settings are about structure, style and technologies used in the code the assistant suggests: from formatting and language choice to the libraries it uses and how much detail it includes.

This includes Preferred code style and formatting (e.g., tabs or spaces, naming conventions); Restricted APIs, libraries, and files; Preferred programming language(s), versions, and environment constraints (e.g., use React with TypeScript rather than plain JS, generate code compatible with Python 3.8); Framework or architecture conventions (e.g., functional or OOP, preferred design patterns); Preferred testing style (e.g., include unit tests by default, use Jest); Preferred documentation style (e.g., generate docstrings, inline comments, or no comments unless asked); and Optimization priorities (e.g., readability, performance, or memory).

Characteristics of System & policies (SP).

These settings let the user define preferences related to memory, data handling, execution boundaries, and compliance with policies.

This includes Model selection, Chat memory depth; Memory management and cleanup (e.g., delete memorized chats); On-premise vs. off-premise execution; Data retention and privacy preferences (e.g., opt-out of logging conversations for training); Suggestion validation (e.g., legal license, static analysis, organization style guideline); Organizational governance integration (e.g., connect to internal policy engines or shared style guides and linters); User role-based access or permissions (e.g., interns’ assistants are not allowed to change codebase and access specific files); and Network or data access controls (e.g., disable web search, access to internal databases, or external APIs).

Characteristics of Human–assistant interaction (HAI).

These settings focus on how the assistant communicates with the user: how it responds, when it responds, and how it looks.

Namely, Preferred response length; Custom shortcuts; Response timing and triggering behavior (e.g., only when prompted, frequent auto-completions, snoozed autocompletions); Appearance of code completions (e.g., position in editor, opacity or background shading); Showing confidence or quality scores for suggestions; Minimum confidence threshold to show completions; Preferred form of address (e.g., by first name, nickname, honorifics, or anonymous); Assigned persona (e.g., work buddy, senior expert, review bot, code generation tool); and Preferred communication style (e.g., concise or detailed, formal or casual, suggestive or directive).

Characteristics of Users & their personal context (UPC).

These settings focus on the user, their personality, abilities, and work context, so the assistant can adapt to the user’s individual needs and the professional context.

Options here are Gender identity or pronoun preferences; Custom job title or role specification to adjust guidance and expectations (e.g., Senior Frontend Engineer, Tech Lead, QA Intern); Accessibility settings (e.g., screen reader compatibility, text size, contrast options); Familiarity with AI assistant (e.g., novice, experienced user); Business context integration (e.g., connect to project management tools like YouTrack, Jira, or GitHub Issues for task-aware assistance); Personal goals or development focus (e.g., learning React, improving test coverage habits); Workstyle profiling (e.g., through a lightweight personality test or user-selected archetype such as strategist, hacker, careful editor); and Preferred learning style (e.g., hands-on guidance or high-level suggestions, step-by-step explanations or summary results).

3 Survey↩︎

To answer RQ1 and RQ2, we conducted a survey to understand whether and in what ways experienced developers care about configuring AI coding assistants, and to get an initial sense of which configuration options they consider most important. In this section we report the design of the survey and our findings.

3.1 Survey Participants↩︎

We recruited participants through an internal research panel at JetBrains, selecting developers who reported at least three years of professional experience and one year of active use of AI coding assistants. We intentionally focused on experienced users rather than novices, as they are more likely to have developed stable preferences around assistant behavior and to have encountered concrete configuration needs in real-world practice. We sent 3,000 invitation emails, receiving 165 link clicks and 56 completed responses. Participants could enter a draw for USD 50 Amazon eGift Cards or an equivalent-value JetBrains product pack.

The participants reported residing in 28 countries, mainly based in Europe (\(66\%\)). \(79\%\) are full-time software engineers with substantial professional experience: 15 with 3–5 years, 18 with 6–10 years, 9 with 11–15 years, and 14 with 16+ years. Most had 1–2 years of AI tool experience (about \(70\%\)), while others had used AI tools for over 2 years. AI tool use is frequent (\(79\%\) daily) and typically IDE-integrated (\(89\%\)). The most commonly used tools are ChatGPT (\(77\%\)), Claude (\(48\%\)), GitHub Copilot (\(46\%\)), and JetBrains AI assistant (\(38\%\)). Among employed respondents, their company’s stance toward AI tools is mixed: 30% reported that their company encourages use, 27% allow it with minimal restrictions, 25% allow it with significant restrictions, and 7% are uncertain.

3.2 Survey Outline↩︎

The survey is structured around the list of 33 configurations introduced in  2.3 and split into four configuration groups: CS, SP, HAI, and UPC. The full survey text is available at the Appendix A.1¹.

The survey starts with background questions that are meant to collect necessary information about the participants and filter out unsuitable candidates. The filtering criteria are: currently working, at least 3 years of professional experience, a role that involves coding (see question 4 in the survey), and at least 1 year of AI experience. In the rest of the survey, the suitable candidates are shown one group at a time and asked to evaluate the configurations within that group. For each configuration option, respondents indicate (1) whether the setting exists in the AI tools they currently use (binary: It exists or It doesn’t exist) and (2) how useful the setting is or would be for their workflow (3-point scale: Not useful, Moderately useful, Very useful). They can optionally add comments and suggest their own configuration ideas that they feel are missing.

3.3 Survey Findings↩︎

In total, 56 participants evaluated 33 configuration options, resulting in 1848 individual ratings. In general, developers expressed a strong interest in configurability. \(72.6\%\) of all ratings evaluate offered configurations as "Moderately" or "Very useful", which further we call positive usefulness. When we map these responses onto an ordinal scale ("Not useful" as 0, "Moderately" as 1, "Very" as 2), the overall mean usefulness score is \(1.1\) out of 2. We will further call mean score calculated over such mapping an "ordinal mean".

By contrast, perceived availability lags behind: only \(35.6\%\) of ratings indicate that a given control exists in participants’ current tools. Together, these results point to a clear configuration gap: developers frequently value controls that they do not believe are currently offered in mainstream AI coding assistants.

Most respondents also reported using configuration to some extent. \(67.9\%\) said that they customize AI tool controls at least occasionally: \(17.9\%\) actively customize settings, while another \(50\%\) tweak a few options. At the same time, \(8.9\%\) of respondents said that they were not aware that AI assistants can be configured at all, reinforcing concerns about discoverability.

3.3.1 Demand & Supply↩︎

We calculated per-configuration summaries and ranked settings to identify the most useful settings, the least useful settings, and settings with the highest unmet demand. Here, we only provide the top results, the table with all statistics for each configuration can be found in the Appendix A.2.

3.3.1.1 Highest unmet demand

We define unmet demand as the difference between positive usefulness rate and existence rate, where larger gaps indicate controls participants value, but do not have or are not aware of their existence.

Showing confidence or quality scores for suggestions and Minimum confidence threshold to show completions: \(67.9\%\) demand gap. In comments, some participants explained that, depending on a task, they might want to see more suggestions, even those with lower quality scores. They mentioned brainstorming as an example of such a task.

Preferred response length: \(62.5\%\) demand gap. Similarly to the confidence threshold, participants explained that they want to change the verbosity depending on the task context (e.g., quick edits vs brainstorming).

Suggestion validation: \(55.4\%\) demand gap. In the comments, participants mentioned that they want AI tools to "self-check" their output, based on the predefined rules and context.

Overall, unmet demand is highest for the HAI group, especially for task-specific controls that influence user trust.

3.3.1.2 Configurations most frequently perceived useful

Model selection: \(94.6\%\) positively useful with \(82.1\%\) as "Very useful". It was also one of the few settings that the participants reported as already available (\(91.1\%\))

Both Preferred programming language(s), versions, and environment constraints and Preferred response length were rated with \(91.1\%\) of positive usefulness. However, less than half of the participants identified these controls as existing in their tools.

Data retention and privacy preferences, Preferred testing style, Showing confidence or quality scores for suggestions, and Minimum confidence threshold to show completions all received \(87.5\%\) of positive usefulness score. Interestingly, participants identified only Data retention and privacy preferences as known to them in their IDEs (\(55.4\%\)).

3.3.1.3 Configurations least frequently perceived useful

Gender identity or pronoun preferences: \(21.4\%\) positively useful and \(12.5\%\) knew about this control in their tools.

Custom job title or role specification: \(35.7\%\) positively useful and known to \(16.1\%\).

Familiarity with AI assistant: \(46.4\%\) positively useful and known for \(17.9\%\).

These results suggest that respondents prioritize configurations that affect correctness, constraints, and trust over purely social or identity-style controls. Moreover, in the comments one of the participants expressed their discomfort with the Gender identity configuration, saying that it is too controversial and unnecessary for work.

3.3.2 Contextual patterns↩︎

Finally, we analyzed whether participants’ backgrounds have any correlation with how they evaluate configurations. This analysis was intended to surface trends rather than make causal claims.

3.3.2.1 AI experience

We examined whether the perceived usefulness of configurations differs by respondents’ experience using AI tools for coding. Participants with 1–2 years (\(n=39\)) of AI use reported a positive usefulness rate of \(69.2\%\) and an ordinal mean of \(1.02/2\). Those with more than 2 years (\(n=17\)) reported \(80.6\%\) positive usefulness and a higher ordinal mean of \(1.28/2\). It might suggest that configurability tends to be valued more as AI tools become more embedded in day-to-day work, or that early (2023) adopters are generally more positive towards AI.

1 breaks down the positive usefulness and ordinal means within each of the four configuration groups. Interestingly, the rates in the CS are very similar between two experience brackets. Positive usefulness in SP is also similar, however, the ordinal means are noticeably higher for more experienced participants, since they voted more configuration options in this group to be "Very useful". Interestingly, significantly more experienced AI users voted positively in the HAI and UPC groups, both in terms of positive usefulness and ordinal means.

3.3.2.2 Company policy

Respondents in organizations where AI use is actively encouraged (\(n=17\)) reported a higher overall usefulness (positive \(79.9\%\), ordinal mean \(1.24\)) than those in organizations with significant restrictions (positive \(68.4\%\), ordinal mean \(0.99\)) and minimal restrictions (positive \(70.7\%\), ordinal mean \(1.11\)). The per-group statistics are shown in  2.

4 Design Sessions↩︎

To gain more insight into RQ2 and answer RQ3, we conducted design sessions with 7 survey respondents who had indicated that they were open to a follow-up interview. The goal of these sessions was to move beyond 0-2 rating of individual configuration options and instead explore how experienced developers would actually organize and control settings in an ideal coding assistant: which configurations they would choose to have, where they would place them, which control widgets they would use, and how they reason about priorities and trade-offs.

4.1 Design Sessions Participants↩︎

We conducted 7 individual design sessions with professional software developers. These people were invited as more experienced AI and configuration users out of the survey participants who agreed to be approached further for additional interviews. All participants reported at least one year of active use of AI coding assistants and five have been using such tools for two years or more. All participants have substantial professional experience with at least five years of professional software development. They regularly use tools such as ChatGPT, GitHub Copilot, JetBrains AI Assistant, Gemini, and Claude, typically on a daily basis for tasks like debugging, refactoring, documentation, and learning new technologies.

4.2 Design Sessions Outline↩︎

All 7 design sessions were conducted in English, lasted between 60 and 90 minutes, and happened via video call with a shared Miro board. ² The board contained a warm-up task based on familiar IDE settings and the main design task focused on configuring an AI coding assistant. Participants were asked to think aloud throughout the tasks.

4.2.1 Warm-up↩︎

We began with a 5-minute task to familiarize participants with the Miro interface and the idea of prioritizing settings. On the Miro board, participants saw an empty settings screen for PyCharm and a set of cards representing common IDE settings sections: Python, Appearance & Behavior, Version Control, Languages & Frameworks, Editor. Each card had a brief description and examples of the specific controls in that section, extracted from PyCharm.

Participants were asked to drag these cards onto the settings screen and arrange them from most to least important according to any criteria they feel relevant. We asked them to think aloud about why some settings matter more than others and how they typically approach configuration in their everyday tools.

4.2.2 Main design task↩︎

After the warm-up, we introduced the main task focused on AI coding assistants (see 1 for the finished state of the main task). The Miro board had two central panels: a larger "settings window" representing a traditional IDE settings dialog, and a narrower "assistant chat window" representing a quick-access area embedded in the assistant’s chat interface.

On the right side, participants saw a list of configuration options, each represented as a draggable card. These cards corresponded to the configurations from our compiled list (2.3), presented as a flat list without group labels to avoid biasing participants towards placing configurations from one group close to each other. The order of items in the list was randomized for each participant. Participants were instructed to choose which configurations they would want in their ideal coding AI assistant. We emphasized that they do not have to use each offered option. If participants felt that an important configuration is missing, they could create a new card using a blank template and label it with their own name and description. Then they placed each selected configuration in either the main settings window or the quick-access chat panel, depending on where they would prefer to control it in practice. Participants also attached a control widget to each selected configuration by copying one of the available widgets (toggle, slider, text box, checklist, radio button) and placing it next to the setting card, or by describing another control type they would prefer.

Participants were encouraged to comment on why they place each card in one panel or the other and a certain control widget was chosen, or why they skipped certain configurations and did not place them anywhere. They were also asked to tell any stories about when they needed such configurations in their work, and voice any ideas on the usage scenarios.

4.2.3 Post-session interviews.↩︎

After completing the design task, we conducted a short semi-structured interview to find out more about the reasons behind participants’ layouts and control widget choices. We asked follow-up questions about why certain settings were prioritized or ignored, which controls they expected to adjust most often, and what additional configuration capabilities they wished existed in current tools. You can see the interview script, together with task explanations, in the Appendix B³.

4.3 Design Sessions Findings↩︎

4.3.1 How much control?↩︎

In total, participants made 231 choices across 7 sessions and 33 configurations to choose from. Most of the chosen configurations were placed in the main settings window rather than quick access. Overall, about \(66\%\) of placements assigned configurations to general settings, \(14\%\) to quick access, and \(20\%\) were left out. When we break this down by configuration group, clear patterns emerge.

Code Suggestions. Configurations that directly affect output were treated as baseline preferences. Participants almost always placed these in the main settings window (\(76\%\)), with only a few quick-access placements or leaving them out (\(12\%\) each). All the participants explicitly said that they want to set these up once per project and not change them afterwards. They also often mentioned configuration files, such as .yaml, .ignore, to be a good alternative for the main settings window,

System & Policies. System controls turned out to be more debatable in terms of placement (only \(62\%\) of placements in main settings). On-premise vs. off-premise execution, Data retention and privacy preferences, Suggestion validation, Organizational governance integration, Network or data access were generally placed in the main settings window, and named as helpful and mostly stable controls that rarely change. Only two people out of 7 said that User role-based access could be potentially useful to them, others chose to leave this configuration out. Six participants placed Model selection into a quick-access window, explaining that they all have this configuration in their tools and change it depending on a task they are working on. The other one participant mentioned always having the cheapest model on, but expressed interest in a feature that would automatically pick a model based on their task and budget, and three other participants mentioned a similar idea. Both Chat memory depth and Memory management and cleanup were split equally between main settings and quick-access windows, with some participants explaining that they would like to control chat message visibility to avoid biasing models with previous dead-end or incorrect reasoning.

Human–Assistant Interaction. While most interaction configuration placements ended up in the main settings window (\(72\%\)), there were several opinion clashes. Most notably, half of the participants wanted Response length control in main settings, while others requested quick access because, according to one person, AI can be very "bossy" and repetitive, and sometimes needs to be forcefully refocused. Also, Preferred form of address was mostly described by participants as unnecessary.

Users & Personal Context. User- and personality-centered configurations were rejected the most (\(41\%\)). Business context integration and Accessibility settings were the only controls that got one rejection, with all the others being rejected by at least half of the participants. When configurations from this group were used, they were always assigned to the main settings window, with the exception of Business context integration which was requested as quick-access by two participants in the context of "turning relevant Jira tickets on and off".

All together, these findings show a clear and recurring distinction. Developers tend to separate consistent project- or organization-specific configurations from task-specific ones. The former are generally thought to belong in a dedicated settings interface, while the latter are expected to be adjusted more fluidly within the assistant’s interface during everyday use. One participant suggested that users should be able to build their "control panels" on their own and adjust them to their current needs and tasks.

4.3.2 How to control?↩︎

One of the design sessions’ goals was to see whether particular control widgets would naturally emerge as better fits for specific types of settings. However, there was rarely a single universally agreed-upon choice. For most configurations, participants suggested a mix of controls rather than settling on one "correct" widget. Checkboxes and free-text boxes were the most common choices overall and appeared at least once for nearly every setting. In many cases, the same configuration was represented with either checkboxes or text boxes by different participants, reflecting a broader split between those who preferred more structured controls and those who favored open-ended input. We also noticed a strong tendency: most developers implicitly picked a side by leaning either toward check boxes or text box, and then stayed with that choice across most of their configuration decisions.

Still, a certain number of settings attracted near-consensus around specific widget types. Numeric or continuous parameters, such as Minimum confidence threshold to show completions and Preferred response length, were uniformly mapped to sliders. Showing confidence or quality scores for suggestions and On-premise vs. off-premise execution were almost always represented as toggles. Model selection was unanimously chosen as a radio button. Other than this configuration, radio buttons were not a popular choice and were mostly mixed with check boxes and toggles. The most prominent pattern that we observed was the fact that most UPC settings, such as Personal goals or development focus, were represented with text boxes (when picked).

4.3.3 Additional configuration ideas.↩︎

Finally, several participants took advantage of the "add your own" option to suggest configurations that were not in our initial list:

• Create task-specific profiles (e.g., reviewer, explainer, or quick-search mode) for the assistant that could be switched on demand and would have predefined configurations.

• Switch for agentic mode

• Switch for vibe coding mode

• Switch for incognito mode that will not save chats into history or use them for training.

• Automatic model selection depending on prompt, resources and overall task

• Resource precalculation of the prompt, such as price or CO2 production

5 Discussion↩︎

RQ1: What configurations do experienced developers want?

Taken together, the survey and design sessions show a clear interest for configurability in AI coding tools. Across all 33 configurations, almost \(75\%\) of survey ratings mark options as at least moderately useful, yet just over a third are believed to exist, revealing a clear configuration gap. The demand is highest for controls that directly affect trust and task fit. Minimum confidence thresholds, Visibility of confidence scores, Response length, Suggestion validation all combine high usefulness with low perceived availability. In contrast, social or identity-oriented options, such as Gender identity, Job titles, Workstyle profiling, are consistently rated as less important. Some of the most valued settings, such as Model selection and Data retention preferences, are already implemented in mainstream tools, but some participants still described them as hard to find. Our findings highlight that not only are valued configurations missing, but even existing ones are often fragmented across account pages, IDE panels, project files, and implicit prompting, to the point where developers are unaware they can configure their assistants at all.

RQ2: How do developers prioritize different types of configuration needs? In the survey, CS and SP configuration groups stand out as the most important. We also see from design sessions that participants placed output- and system-related configurations into the main settings window or imagined them living in configuration files, describing them as project or workspace defaults that should be set once and then rarely changed. The HAI group is close behind and, more interestingly, it contains configurations with the highest unmet demand. Interaction-level controls were also much more likely to be placed in the assistant’s quick-access area, reflecting their task-specific and situational nature. UPC options were often rejected entirely, with multiple participants explicitly questioning their relevance to work. In design sessions, when used at all, they were treated as infrequently touched background settings.

RQ3: What types of configuration interfaces do developers prefer? The design sessions did not reveal a single dominant control widget, but rather a set of conventions. Across configurations, checkboxes and text boxes were the most frequently chosen widgets, and many configurations attracted both, reflecting different preferences for structured versus open-ended control. Individual participants tended to stick with one of them, indicating that widget preferences may be as much about personal interaction style as about the configuration itself.

At the same time, some configurations showed near-unanimous choices. For example, Minimum confidence threshold to show completions or Response length were consistently represented as sliders. These choices align closely with how current tools already implement similar controls, suggesting that existing interfaces strongly shape developers’ expectations. In the future, it might prove difficult to move developers from one configuration setup to another, even if the new version proves to be "superior".

Future directions Our participants described some configuration ideas that go beyond the initially proposed list. We see them as promising directions for future work, particularly ideas about (1) task-specific assistant profiles, (2) agentic, vibe and incognito mode switches, (3) controls for estimating and constraining per-prompt cost or CO2 footprint.

Finally, our observations hint that configuration needs may interact with user personality and working style. Some developers wanted access to as many controls as possible but were happy to bury them deep inside general settings. Other participants, who also seemed to be more eloquent than others, preferred to set up their preferences through text boxes and configuration files. Future research could explicitly investigate how traits such as desired autonomy, risk tolerance, or introversion shape configuration preferences, and how assistants might adapt not only to project context but also to different "configuration personas" among developers.

5.1 Threats to validity↩︎

Internal validity. Our survey relies on self-reported experience, habits, and perceptions of what configurations exist in current tools. Participants may misremember which configurations they have seen or used, or respond in ways that are influenced by social desirability. We partly mitigate this by focusing on experienced assistant users, but we cannot rule out recall or response biases. In the design sessions, facilitator explanations, example demonstrations, and the structure of the Miro board may also have nudged participants toward particular placements or widget types. To reduce this risk, we randomized the order of configuration items, avoided showing our four-category grouping, and encouraged participants to add their own settings and controls.

External validity. Our participants were experienced professional developers who already use AI coding assistants regularly. As a result, the findings may not generalize to novice developers, occasional AI users, or teams working under very different regulatory or organizational constraints. We see our results as most directly applicable to teams and organizations that are actively (with possible restrictions) integrating AI assistants into development workflows.

Construct validity. We evaluate usefulness via Likert-style ratings but some options may be rated as useful in theory but rarely used in practice. Our configuration list and four-group structure are also grounded in current tools and prior literature, and therefore cannot cover the full design space. Finally, the design tasks used a simplified two-panel layout (main settings window and quick-access) and a fixed set of control widgets, which may have constrained how participants expressed their preferences. We attempted to mitigate this by allowing participants to add free-form settings and to propose alternative controls.

6 Conclusion↩︎

AI coding assistants are rapidly becoming part of everyday software development, yet their configurability remains limited, fragmented, and often invisible to users. In this paper, we compiled a list of 33 configuration options for coding assistants, grounded in existing tools and prior work, and organized them into four groups: Code suggestions, System & policies, Human–assistant interaction, and Users & their personal context. We then studied how 56 experienced developers and 7 design session participants evaluate and structure these options.

Our results show a clear demand gap: most configurations are considered useful in principle, but only a minority are believed to exist in current tools. Developers particularly want more control over interaction-related aspects such as confidence thresholds, showing confidence scores, and response length, while being skeptical of many persona-related configurations. They also implicitly distinguish between stable, project-level controls and more fluid task-specific controls that are expected in quick-access areas. Together, these findings outline design opportunities for configurable coding assistants and point to the scope for more unified, discoverable, and trustworthy configuration mechanisms.

Acknowledgements↩︎

We thank all study participants for generously sharing their time, experiences, and insights into their daily development work. Portions of this manuscript were revised using OpenAI’s ChatGPT for grammar and clarity improvements. The authors reviewed and edited all suggestions and take full responsibility for the content.

References↩︎

---

1. https://github.com/KatyaKos/configurableAI↩︎

2. https://miro.com/↩︎

3. http://github.com/KatyaKos/configurableAI↩︎