1 Introduction↩︎

Owing to recent advances and the widespread deployment of LLMs, they are now integrated into a broad range of daily tasks, including workflow planning [1], [2], decision-making processes [3], [4], and reasoning [5], [6]. Consequently, users now anticipate responses that are not only accurate but also personalized to individual needs [7], goals [8], and backgrounds [9]. To address these expectations, recent studies have focused on personalizing LLM responses by integrating user-specific text-based information, including personas [10], [11], preferences [12], and long-term user histories [13]. However, these static profile-based approaches [10], [11] often fail to capture per-query dynamics, where user context can shift depending on domain expertise, task demands, or intent [9], [14].

To support query-wise dynamic personalization, we focus on user domain expertise—an essential factor that determines the appropriate level of detail, terminology, and conceptual complexity in LLM-generated responses [15]–[18]. Notably, a user’s expertise can vary significantly across queries, even within the same subject, making it a crucial signal for generating personalized responses [19], [20]. Yet, users’ domain expertise is difficult to infer from query text alone—short or ambiguous prompts may obscure actual expertise level [21]; for instance, experts may simplify their question for efficiency, while novices may adopt technical terms without fully understanding them. These challenges motivate us to consider combining textual cues with behavioral signals to infer expertise robustly at the query level.

To this end, we propose ExPerT (Expertise-aware Personalized Text generation), a query-wise response personalization framework that dynamically adapts LLM responses to a user’s query domain expertise by leveraging both semantic and behavioral cues (Figure 1). Specifically, ExPerT introduces an expertise inference module that jointly interprets (i) query semantics, such as phrasing and domain-specific terminology, and (ii) keystroke dynamics reflecting fluency and confidence during query composition to estimate user expertise. To achieve this, we design an input feature extraction pipeline that combines query text with keystroke-derived word-level features, capturing timing and fluency patterns, to provide input features for the LLM. The inferred expertise level is subsequently injected into our expertise-conditioned generation prompt that encodes explicit instructions.

We conducted experiments to validate two key assumptions of this work: (i) user expertise can be accurately inferred from semantic and behavioral cues, and (ii) personalizing LLM responses based on user expertise improves response satisfaction. We recruited 40 participants from three domains (chemistry, computer science, and business) who interacted through a custom chatbot interface that recorded query text, keystroke dynamics, self-reported expertise, and satisfaction ratings, yielding a dataset of 1270 annotated queries. Using this dataset, we first evaluated whether semantic-behavioral cues enable accurate query-level expertise inference. Our approach achieved the lowest mean absolute error (MAE) of 0.398, representing a 65.7% reduction compared to IDL [14], the strongest baseline (MAE \(=\) 1.162). We then examined whether expertise-conditioned responses improve satisfaction and observed a 17.52% increase (from 3.71 to 4.36) on a 5‑point Likert scale.

These results demonstrate that ExPerT not only infers situational expertise accurately but also generates responses that better align with users’ knowledge, improving perceived quality.

We summarize our key contributions as follows:

The preprocessing pipeline and an anonymized version of the dataset are available.¹

2 Method↩︎

We present ExPerT (Expertise-aware Personalized Text generation), a query-wise adaptive response generation framework that personalizes LLM responses based on the user’s domain expertise level. As shown in Figure 2, ExPerT consists of two main modules: (i) an expertise inference module that estimates expertise from query text and keystroke dynamics via in-context learning (Section 2.2), and (ii) a response generator that adjusts content and style using expertise-conditioned prompts (Section 2.3). Full prompt templates and examples are provided in Appendix 8 and  9.

2.1 Problem formulation↩︎

Given a user query \(q\), our objective is to generate a personalized response \(r_q\) tailored to the user’s domain expertise level, denoted as \(e_q\). This is achieved through a two-stage: (i) semantic-behavioral expertise inference, where expertise is estimated from the user’s semantic features derived from the query \(q\) and behavioral features \(\mathbf{X}_q\) such as a summary of keystroke dynamics, which are collectively represented as input \(I_q\), and (ii) expertise-conditioned generation, where the response is personalized based on the inferred expertise level. Formally: \[\begin{align} \text{Input features:} \quad & I_q = \{q, \mathbf{X}_q\}, \\ \text{Inferred expertise level:} \quad & \hat{e}_q = f(I_q),\\ \text{Personalized response:} \quad & r_q = g\bigl(q, \hat{e}_q\bigr), \end{align}\] where \(f\) denotes the expertise inference function, and \(g\) is the expertise-conditioned response generation function.

2.2 Expertise inference via semantic-behavioral cues↩︎

We estimate expertise levels using in-context learning, taking into account both the semantic context of the user query and behavioral signals such as keystroke dynamics during query composition. Our method includes two key components: (i) a feature extraction pipeline that captures both the user’s semantic query context and behavioral context, converting key-level keystroke events into interpretable temporal statistics at the word level; and (ii) a prompt-based inference mechanism that uses feature representations, along with few-shot examples, to classify the user’s query domain expertise.

2.2.0.1 Feature extraction.

We use both semantic and behavioral signals as key features, motivated by the intuition that both signals provide complementary evidence of a user’s domain expertise. On the semantic side, the content and phrasing of a query reflect the user’s conceptual understanding and familiarity with domain-specific terminology [15]. On the behavioral side, keystroke dynamics capture how confidently and fluently a user expresses their query [22]. We hypothesize that the synergetic integration of both signals enables more accurate expertise inference. To capture semantic indicators of expertise, we include the raw query text \(q\) as part of the inference input. Rather than explicitly extracting linguistic features, we allow the LLM to interpret the semantic content of the query.

For capturing behavioral cues, illustrated in Figure 3, we encode keystroke dynamics using seven temporal and fluency features, adapted from prior work on user state and input behavior modeling [23]–[25]. Specifically, let \(K_q = \{k_0, k_1, \dots, k_m\}\) denote the sequence of keystroke events for a given query, where each event \(k_i = (c_i, a_i, t_i)\) consists of key character \(c_i\), action type \(a_i \in \{\texttt{down}, \texttt{up}\}\), and timestamp \(t_i\). From this sequence, we compute five temporal features: Duration, which captures the hold time between key press and release for individual keys; and inter-key interval features, Down-Down, Up-Down, Up-Up, and Down-Up, which measure temporal intervals between consecutive key transitions. Additionally, we incorporate two global indicators of fluency: the total number of Backspace keys, and Typing Speed, defined as the number of keystrokes per minute.

While key-level features capture fine-grained timing, segmenting them at the word level yields stable behavioral units and aligns typing patterns with semantic content for expertise inference. To achieve this, we segment the raw keystroke stream into words using space and punctuation boundaries, identifying word starts by detecting characters preceded by a space key. For each word \(w\), we compute a word-level feature vector from the corresponding keystroke events \(K_w\). Specifically, we calculate the mean and standard deviation for one duration feature and four inter-key interval features, resulting in \(2 \times 5 = 10\) dimensions, and include two additional fluency features. This yields a 12-dimensional representation defined as: \[\mathbf{x}_{K_w} = \phi(K_w) \in \mathbb{R}^{12},\] where \(\phi\) aggregates the above statistics into the 12-dimensional feature space.

To account for typing behaviors that occur before the user begins inputting the first character of a query, we introduce a dummy group represented by \(\mathbf{x}_0 = \phi(K_0)\), where \(K_0\) includes all keystrokes that occur prior to the first typed character and where the immediately preceding keystroke is a backspace. While the dummy group is not included in the natural language query passed to the LLM, it is treated identically during feature extraction to preserve early behavioral signals.

In addition, we define a navigation group for keystroke events related to cursor movement (e.g., arrow keys) during query formulation, which captures revision behavior and is retained as part of the behavioral signal.

Therefore, given a query \(q\) consisting of a dummy group, word-level groups, and navigation groups, the complete behavioral representation is: \[\mathbf{X}_q = \{ \mathbf{x}_i \}_{i=1}^{|\mathcal{G}_q| },\]

where each \(\mathbf{x}_i = \phi(K_i)\) corresponds to a behavioral group (dummy, word-level, or navigation) extracted from the keystroke sequence, and \(|\mathcal{G}_q|\) denotes the total number of groups for query \(q\).

Finally, the input pair used for the expertise inference prompt is defined as: \[I_q = \{q, \mathbf{X}_q\}.\] Additional details and examples are in Appendix 10.

2.2.0.2 Expertise inference prompt.

We frame the expertise inference task as an in-context classification problem [26], [27], allowing the LLM to infer user expertise directly from structured semantic-behavioral features. To support this, we design a system prompt composed of four key components: (i) a role and task definition, including an explanation of the five-level expertise taxonomy; (ii) natural language descriptions of temporal features extracted from keystroke logs; (iii) explicit multi-step classification procedure that guide the model’s decision-making process; and (iv) a set of few-shot examples comprising input-output pairs.

To enable expertise inference, we construct a few-shot example. Let \(Q = \{q_0, q_1, \dots, q_M\}\) denote the full set of queries. We then select a subset of \(N\) queries \(\{q_0, q_1, \dots, q_N\} \subset Q\), where each query is represented as an input pair \(I_{q_i} = \{q_i, \mathbf{X}_{q_i}\}\) and annotated with a self‑reported expertise label \(e_{q_i} \in \{\text{Novice}, \dots, \text{Expert}\}\). Each example in the prompt is formatted as a structured input-output pair: \[\mathcal{D}_i = (I_{q_i},\;e_{q_i}).\] The final in-context support set is: \[\mathcal{S} = \{ \mathcal{D}_1, \dots, \mathcal{D}_N \}.\] where \(N = 10\) in our default configuration, a design choice made to balance context richness with prompt length in the absence of strong prior constraints [28], [29].

Finally, we append the user prompt—comprising the current query and keystroke features, enabling inference of the user’s expertise.

2.3 Expertise-conditioned response generation↩︎

To personalize LLM responses, we design an expertise injection prompt that encodes the user’s query domain expertise within the prompt input. This framework guides the model to align the level of detail, terminology, and conceptual complexity of its responses with the user’s expertise, enabling more reliable personalization of tone and depth.

2.3.0.1 Expertise injection prompt.

We construct a system prompt with two main components: (i) structured metadata specifying the user’s expertise level on a five-point taxonomy—Novice, Beginner, Intermediate, Proficient, and Expert  [20]; and (ii) an expertise-level behavioral profile with generation guidance for calibrating response complexity, terminology, and abstraction. Additionally, the user query is included as a user prompt.

3 Data Collection↩︎

To construct a query-level user behavior dataset for expertise-conditioned response generation and satisfaction evaluation, we conducted a user study that recorded keystroke dynamics, query text, self-reported expertise levels, and response satisfaction. More details are provided in Appendix 11.

3.0.0.1 System.

We developed a custom web-based chatbot that supports querying an LLM, collecting query text, satisfaction ratings, and fine-grained keystroke dynamics. Responses were generated using chatgpt-4o-latest [30].

3.0.0.2 Study design and procedure.

Participants completed three domain-specific tasks in chemistry, computer science, and business, chosen to span varying levels of domain familiarity and conceptual complexity. For each domain, a unique task was designed, and participants used our chatbot to investigate the task. For each query, they reported perceived expertise (used for response generation) and rated response satisfaction on a 5-point Likert scale. After each task, participants completed a post-task survey assessing self-reported measures, followed by a short multiple-choice quiz.

3.0.0.3 Participants.

We recruited 40 participants (20 males, 20 females, mean age of 23.10 (SD 1.61)), targeting students with different academic backgrounds. To ensure sufficient expertise, participants were selected from each domain: 13 in chemistry, 14 in computer science, and 13 in business.

3.0.0.4 Dataset summary.

For each query, we recorded the text by the user, the self‑reported expertise level, and two LLM responses (baseline and expertise-conditioned) along with corresponding satisfaction ratings, and raw keystroke data logged as sequential key presses and releases with timestamps. In total, the study yielded 1270 queries spanning the three domains, forming a dataset of textual, behavioral, and subjective user feedback. The dataset comprised 651 Novice, 245 Beginner, 209 Intermediate, 120 Proficient, and 45 Expert queries.

4 Expertise Level Inference↩︎

We evaluate our expertise inference method, which estimates user expertise per query from semantic content and keystroke features without long-term interaction history or explicit user profiles.

4.1 Setup↩︎

4.1.0.1 Baselines.

As a theoretical lower bound, we consider a Random Guess baseline that is a random predictor uniformly sampling labels from the five-point expertise scale. We then compare our semantic-behavioral inference method with prior approaches. We include a Persona-based Classifier baseline based on Hu et al. [10], which uses zero-shot prompting with static user attributes (e.g., demographics and attitudes) to predict subjective labels, serving as a reference for profile-based personalization without dynamic or behavior-aware adaptation. Second, we implement a Session-level Classifier inspired by prior work [20], which aggregates users’ past queries and model responses within a session to predict expertise, capturing long-term interaction context without per-query or behavior-aware adaptation.

Lastly, we implement two Dynamic Adaptation baselines that evolve with changes in the user-system dialogue. In-Dialogue Learning (IDL) [14] extracts persona-related sentences from user dialogue for fine-tuning and DPO-based adaptation, while AI Persona [9] dynamically updates persona attributes during conversation to enable real-time profile adjustment. More details are provided in Appendix 12.

4.1.0.2 Metrics.

We evaluated performance based on the mean absolute error (MAE) and mean squared error (MSE) between the predicted and self-reported expertise levels. Expertise labels were mapped to a five-point scale (Novice=1, Expert=5).

4.2 Results↩︎

We evaluated keystroke-based expertise inference by comparing model predictions to users’ self-reported expertise levels. Inference was conducted in the within-user setting, using 10-shot examples drawn from the same user, unless specified, with GPT-4.1 [31]. As shown in Table 1, ExPerT achieved the best overall performance with a mean absolute error (MAE) of 0.398 and a mean squared error (MSE) of 0.698.

ExPerT outperformed the strongest baseline, IDL [14], reducing the MAE by 65.7% (from 1.162 to 0.398).

As shown in Figure 4, we further compare ExPerT with the session-level baseline, which explicitly predicts expertise. While the session-level baseline predictions were heavily concentrated on the intermediate expertise level, our method produced a more balanced distribution that better captured the full range of user expertise.

4.2.0.1 Complementary advantages of semantic and behavioral features.

Beyond aggregate performance improvements, a fine-grained analysis is provided to clarify how semantic and behavioral cues are complementary for expertise inference. Semantic cues indicate what technical terms appear and how they are used in context (e.g., definition-seeking vs.conceptual use), while keystroke behavioral cues capture how fluently users produce those same terminologies during composition.

Figure 6 shows systematic expertise-aligned trends across multiple inter-key intervals: as expertise increases, both mean intervals and word-level variability decrease on domain-specific terms. Figure 5 illustrates this effect for each example, showing that experts type domain-specific terms faster and more consistently, whereas beginners exhibit slower and more variable timing when composing them. These patterns collectively show how timing-based fluency provides an orthogonal signal that complements the semantic interpretation of term usage, thereby reducing ambiguity in expertise inference. Statistical analyses supporting these observations are provided in Appendix 14.1.

Crucially, ExPerT infers expertise by jointly leveraging semantic and behavioral cues, which provide complementary evidence. As shown in Table 2, left example, stable keystroke behaviors on segments (e.g., low backspace rate and consistent inter-key timing) support an expert-level interpretation for the domain-specific terms ("priority scheduling", "multilevel queue scheduling"), leading ExPerT to classify as Expert. In contrast, semantic-only classifies the same query as Novice by treating the broad phrasing ("tell me everything") as a low-expertise overview request. In the right example, a query about spectroscopic properties is retained as Beginner despite fluent typing, because the domain-specific terms (e.g., "IR" and "Raman activity") are used only to request an explanation rather than to apply or integrate concepts. In contrast, behavior-only inference labels it as Proficient due to low error correction and stable timing.

These cases demonstrate complementarity: combining semantic and behavioral cues corrects semantic-only misinterpretations of open-ended phrasing and behavior-only overestimation driven by fluent typing. (See Appendix 15 for full cases.)

4.2.0.2 Model generalizability.

To assess generalizability beyond a single model, we evaluate ExPerT’s performance across multiple LLMs, including GPT-4.1 [31], Claude Sonnet 4 [32], and Gemini 2.5 Pro [33]. As shown in Figure 7, ExPerT consistently outperforms semantic-only and behavioral-only across all models, indicating model-agnostic effectiveness.

4.2.0.3 Dummy group.

We conduct an ablation study to examine the effect of the dummy group. As shown in Table 3, removing the dummy group degrades performance, with MAE increasing by 1.3% (from 0.398 to 0.403). It suggests that the dummy group provides a stabilizing reference during inference.

4.2.0.4 Keystroke representation methods.

To evaluate how keystroke representations affect expertise inference in ExPerT, we compared three approaches: (i) raw key-level features without aggregation, (ii) query-level features aggregated over the entire query, and (iii) word-level features aggregated per word (ExPerT). As shown in Table 4, word-level aggregation achieves the best performance, suggesting that it better preserves behavioral patterns than either unstructured raw keystrokes or coarse query-level statistics (see Appendix 10).

4.2.0.5 Few-shot examples.

To assess few-shot configuration effects, we vary the number of examples (1, 2, 5, and 10) and the sampling source (within-user vs. cross-user). ExPerT supports both settings: within-user personalization when prior interactions from the same user are available, and cross-user inference for cold-start settings where such history is unavailable. As shown in Figure 8, performance improves with more examples, with the within-user setting achieving the best results at 10-shot, which suggests that even a modest number of examples is sufficient for accurate inference. In contrast, the cross‑user setting, where all few‑shot examples were drawn from different users, performed worse, likely due to misaligned behavioral patterns [34]. This gap highlights the importance of user-specific cues, as typing dynamics tend to vary significantly across individuals [35]–[38]. Additional statistical analyses of user-dependency are reported in Appendix 14.2.

5 Expertise-Conditioned Response Generation↩︎

In this section, we report two response-generation evaluations. Together, these evaluations assess whether conditioning responses on user expertise improves user satisfaction (Section 5.1) and whether these benefits persist when expertise is inferred rather than self-reported (Section 5.2).

5.1 Self-Reported Expertise Conditioned User Satisfaction↩︎

We analyze the expertise-conditioned responses collected during the data-collection stage, stratified by participants’ self-reported expertise. Because the data violated the normality assumption, we use the Wilcoxon signed-rank test for paired comparisons.

5.1.0.1 User satisfaction

As illustrated in Figure 9, expertise-conditioned responses received significantly higher satisfaction ratings than baseline responses. The mean score increased from \(3.71\) (\(SD = 0.64\)) to \(4.36\) (\(SD = 0.40\)), corresponding to an average improvement of \(0.65\) (\(SD = 0.56\)) (\(p<0.001\), \(W = 51824.0\), \(Z=-26.94\), \(r=0.76\)). Most participants rated ExPerT higher than the baseline, with higher satisfaction across all self-reported expertise levels.

5.1.0.2 User expertise dynamics

Participants’ self-reported expertise varied substantially across and within domains, depending on subtopic and recent exposure, highlighting its context-dependent nature. Moreover, expertise assessments fluctuated within sessions, as participants revised their perceived expertise upward with growing familiarity or downward when encountering unfamiliar concepts or unexpected (p1, p11, p12, p13, p17). These findings underscore the dynamic nature of expertise and motivate adaptive LLMs that tailor responses to users’ evolving cognitive states. (See Appendix 11.5.)

5.2 Follow-up Evaluation with Predicted Expertise↩︎

We present a follow-up study with returning participants, who evaluated three responses to queries they had originally submitted: a response without expertise conditioning, one conditioned on self-reported expertise, and one conditioned on  ExPerT-predicted expertise (See Appendix  11.6).

5.2.0.1 Study and participants.

For the follow-up study, we recruited 16 returning participants (5 males, 11 females, mean age of 22.63 (SD = 1.45)) from the original data collection stage. Participants were selected from each domain: 7 in chemistry, 7 in computer science, and 2 in business. All responses were generated using gpt-5.3-chat-latest [39]. For each query, participants were shown three responses and independently rated their satisfaction with each response on a 5-point Likert scale.

5.2.0.2 User satisfaction

In this study, we collected 522 satisfaction queries. As illustrated in Table 5, expertise-informed responses increased user satisfaction: self-reported expertise received the highest ratings (\(M = 4.010, SD = 0.947\)), predicted expertise showed similar ratings (\(M = 3.975, SD = 0.982\)), and no-expertise received the lowest ratings (\(M = 3.586, SD = 0.994\)). A repeated-measures ANOVA showed a significant effect of response type (\(F(2, 30) = 28.39, p < .001, \eta^2 = .208\)). Holm-corrected post hoc comparisons showed that both the self-reported expertise condition (\(t(15) = 6.09, p < .001\)) and the predicted expertise condition (\(t(15) = 5.79, p < .001\)) were rated significantly higher than the no-expertise condition, with no significant difference between the self-reported and predicted expertise conditions (\(t(15) = 1.22, p = .241\)). These results indicate that ExPerT-predicted expertise achieved user satisfaction comparable to self-reported expertise.

6 Conclusion↩︎

We introduce ExPerT, a query-wise personalization framework that adapts LLM responses to users’ expertise via semantic and behavioral cues. Our inference method achieves an MAE of 0.398, surpassing both session-level and persona-based baselines. and improves user satisfaction.

Acknowledgments↩︎

This work was supported by Institute of Information & communications Technology Planning & Evaluation(IITP) grant funded by the Korea government(MSIT)(No.RS-2020-II201336, Artificial Intelligence graduate school support(UNIST)), (RS-2025-25442824, AI Star Fellowship Program(Ulsan National Institute of Science and Technology)), and under the Leading Generative AI Human Resources Development(IITP-2026-RS-2024-00360227). This work was also supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (RS-2025-00553241). In addition, this research was supported by the "Advanced GPU Utilization Support Program" funded by the Government of the Republic of Korea (Ministry of Science and ICT).

We also thank the UAI Lab members, especially Changmin, Jaemin, and Jiseung, for their feedback and participation in the pilot study.

Limitation↩︎

While our in-context learning-based approach enables fine-grained personalization, it has several limitations. First, the method remains sensitive to prompt design and few-shot exemplar composition. In addition, because inference benefits from within-user demonstrations, its robustness may decrease in cross-user or cold-start settings where user-specific interaction history is unavailable.

Second, keystroke dynamics are informative but inherently noisy behavioral signals. Features such as typing speed, backspace rate, and temporal variation may reflect confounding factors including fatigue, keyboard familiarity, or device differences rather than expertise alone [23], [24]. This may reduce robustness across heterogeneous users, devices, and usage contexts. In addition, behavioral signals such as keystroke dynamics raise privacy, fairness, and accessibility concerns, including residual identifiability risk and potential bias related to motor impairments, device familiarity, or input conditions. Future work should therefore investigate privacy-preserving deployment strategies such as on-device personalization, secure communication, and minimizing cross-user sharing of behavioral data.

Finally, our study was conducted with a relatively small and homogeneous participant pool, primarily university students. As a result, the generalizability of  ExPerT to broader populations and interaction settings remains to be validated. Also, our labels rely on self-reported expertise, which serves as a practical proxy for users’ perceived query-wise expertise but may not always align with objective proficiency or task performance.

7 Related Work↩︎

7.0.0.1 LLMs for personalized response generation.

Most existing LLM personalization studies rely on static approaches, such as persona-conditioned prompting [10], retrieval-augmented generation (RAG) [11], [21], or reinforcement learning from human feedback (RLHF) based fine-tuning [40], [41], which assume persistent profiles and fixed preferences. Persona and RAG-based methods depend on long-term user data, yet often face context overflow or cold-start issues [11], [21], while RLHF-based approaches tend to optimize for average user preferences, missing individual variation [40], [41]. In contrast, our approach enables query-wise personalized responses to the target user’s expertise without relying on static profiles or retraining.

7.0.0.2 Semantic and behavioral features for user interaction.

Recent work tried to personalize LLMs using either semantic features, such as few-shot semantic prompts [42] and reasoning over emotion and intent in dialogue [7], or behavioral features, including mobile sensing for cognitive context [43] and keystroke logging to capture writing behaviors [44]. However, semantic methods rarely address domain expertise, and behavioral approaches often require specialized hardware or focus on aggregate patterns. Inspired by these studies, our approach infers and adapts to users’ expertise levels via the synergetic use of semantic and behavioral signals, achieving accurate and personalized responses.

7.0.0.3 Impact of expertise on LLM interactions.

The expertise reversal effect [45] shows that instructional strategies effective for novices can hinder experts, who prefer succinct, information-dense content [46]. For instance, novices benefit from contextual explanations and step-by-step support, while experts prefer concise text and quick access to definitions [15], [46]. Misalignment between system output and user expertise significantly reduces user satisfaction and task success [20]. While aligning LLMs with user proficiency improves engagement [20], [47], prior work relies on explicit expertise cues [10] or post-hoc classification [13], [20], rather than instantaneous expertise-level inference. Our approach addresses this gap by dynamically adapting responses to each query.

8 Expertise Inference↩︎

8.1 Implementation details↩︎

All inference experiments used gpt-4.1 [31] with temperature=0.0, top_p=1.0, and a maximum output length of 512 tokens to prevent response truncation. In our experiments, few‑shot samples were randomly selected using a fixed random seed of 0.

8.2 Prompt structure for expertise inference↩︎

We provide the core prompt structure used for expertise inference. This prompt guides the model to classify user expertise based on semantic and behavioral cues, including query text and keystroke-derived features. It specifies the model’s role, expertise level taxonomy, reasoning steps, and required output format. To support consistent reasoning, we prepend a task instruction that decomposes the classification process into a series of sub-steps inspired by Chain-of-Thought prompting [5]. These steps explicitly instruct the LLM to (i) infer a baseline expertise level from within-user support examples that reflects the user’s overall topic familiarity, (ii) analyze the intent and conceptual characteristics of the current query to compute a bounded adjustment, (iii) identify technical terms in the query, (iv) use keystroke features such as hesitation, corrections, and stability only to modulate this adjustment on technical-term segments, and (v) combine the baseline and the bounded adjustment to select the final category within the five-level expertise taxonomy. This structured reasoning encourages stable, user-consistent predictions grounded in behavioral evidence. The following boxes present the exact system and user prompt templates used in our experiments.

8.3 User prompt format↩︎

8.4 Output examples↩︎

8.5 Keystroke representation methods↩︎

The description of input features provided to the model varies depending on the chosen keystroke representation method. As described in Appendix 10, we prepare multiple instruction of semantic‑behavioral input, including word‑level, query‑level, and raw‑level keystroke summaries.

8.6 Few-shot examples↩︎

We include few‑shot examples to provide context for expertise inference, enabling the model to ground its reasoning in realistic typing and query behaviors. Two configurations are considered: (i) within‑user examples, which include queries and typing features from the same user, and (ii) cross‑user examples, which are drawn from other users. The within‑user setting captures user‑specific typing dynamics and lexical preferences, while the cross‑user setting provides more diverse but potentially less aligned behavioral patterns. These examples help the model learn both individual behavioral cues and generalizable expertise‑related patterns. Here, each few‑shot example corresponds to the input representation shown in Appendix 10 augmented with the associated expertise label, forming a set of \(\{I, \hat{e}\}\) pairs. We sample these exemplars uniformly at random from eligible prior interactions using a fixed random seed of 0, excluding the current test query.

8.7 Semantic-only & behavioral-only method prompt↩︎

In addition to semantic‑behavioral representations, we also consider two ablation variants: (i) a semantic‑only method, which uses only the raw query text as input without any keystroke features, and (ii) a behavioral‑only method, which uses only the keystroke features without the semantic content of the query. In these settings, the input set was adjusted accordingly: the semantic‑only method received only the query text, while the behavioral‑only method received only the keystroke summary. Few‑shot examples were also constructed to match each input type, ensuring consistency between the example demonstrations and the target prompt.

9 Expertise-conditioned response generation↩︎

9.1 Implementation details↩︎

We used OpenAI’s chatgpt-4o-latest model [30] for all generation experiments, with temperature=1.0, top_p=1.0, and a maximum output length of 2048 tokens to avoid truncation. This model was chosen because it closely reflects the behavior of widely available commercial chatbots, ensuring that our findings generalize to realistic user-LLM interaction settings.

9.2 Expertise-conditioned response prompt↩︎

To ensure unbiased ratings by preventing participants from recognizing whether the answer was personalized, an instruction explicitly stated that the response should never mention the user’s expertise.

10 Feature Extraction↩︎

To support expertise inference from keystroke dynamics, we construct explicit semantic-behavioral input feature representations that are directly fed into our expertise inference prompt as user prompt (Appendix 8). These inputs integrate (i) the raw query text, capturing semantic indicators of the user’s domain knowledge, and (ii) numerical feature vectors derived from keystroke dynamics, capturing behavioral cues of typing fluency and confidence. Below, we provide concrete examples for each feature format to illustrate the exact values passed to the inference model.

10.1 Word-level input examples↩︎

Word-level features capture fine-grained typing behavior by segmenting the query into individual words and computing a feature vector for each. For every word segment, we compute five timing-based metrics—key hold duration, Down-Down (DD), Up-Down (UD), Up-Up (UU), and Down-Up (DU) intervals—and aggregate each by its mean and standard deviation. We additionally record two global control signals: typing speed (keystrokes per minute) and the number of backspace keystrokes used during that word. These features are paired with the corresponding word tokens to align behavioral patterns with semantic context.

10.2 Query-level input examples↩︎

Query-level features summarize the entire query as a single behavioral profile. Instead of computing features for each word separately, all timing intervals, typing speed, and backspace counts are aggregated across the full query.

10.3 Raw-level input examples↩︎

Raw-level features retain keystroke dynamics for each key event rather than aggregating across the entire query. For every key pressed, we record five timing metrics in milliseconds: key hold duration (Duration_key1), time between consecutive key presses (DD-time), time between a key release and the next key press (UD-time), time between consecutive key releases (UU-time), and time between the press of one key and the release of the next (DU-time).

11 User Study: Task and Interface Information↩︎

Before starting the study, the protocol was approved by the local Institutional Review Board (IRB). All participants provided informed consent, approved by the local Institutional Review Board (IRB), which permitted the collection and analysis of their interaction data. All users were anonymized, and all data were stored and used strictly for research purposes. All participants were compensated 22 USD, and to enhance engagement, the top five participants, based on question quality and completion, received an additional 15 USD reward.

11.1 User study interface↩︎

We implemented a custom web-based interface to capture detailed user-LLM interaction data while maintaining experimental control and minimizing distractions. Each session, which lasted approximately 2 to 2.5 hours in total, was conducted on a lab-provided computer under the supervision of a study operator. For each query, participants were presented with two responses and asked to comparatively evaluate them. To reduce presentation-order bias, the responses were displayed in randomized left–right order. Participants were blind to the condition and were not informed which response came from the baseline system or the expertise-conditioned system. To further ensure a fair comparison, all responses were generated using the same decoding settings across conditions, thereby controlling for verbosity and other generation-related differences. The interface and study flow were structured as follows:

• Task Execution. Each participant performed three distinct information-seeking tasks, each lasting 25 minutes, using the chatbot interface. Participants were given detailed written instructions for each task on printed handouts. The chatbot’s conversational state was reset between tasks to avoid carryover effects. In the experiment, participants completed three domain-specific tasks, with the task order balanced using a partial counterbalancing method.

• Query Submission and Feedback. Before submitting each query, participants rated their familiarity with the topic using a five-level scale (Novice-Expert). A tooltip-based help system clarified the meaning of each level. After receiving the LLM’s response, participants rated their satisfaction on a mandatory five-point Likert scale (1: very unsatisfied, 5: very satisfied) using a star-rating widget.

• Interaction Logging. The interface automatically logged all user interactions, including timestamps, queries, self-reported expertise, model responses, and satisfaction ratings for subsequent analysis.

• Experimental Controls and Compliance. To ensure engagement and control for external aids, copy-paste functionality was disabled, and participants were required to submit a minimum of ten queries per task. The chatbot was stateless across tasks and did not retain prior interactions, which was explicitly communicated to participants to discourage meta-queries (e.g., “continue from earlier”).

After completing the three tasks, participants filled out a brief interaction survey (approximately two minutes) regarding perceived difficulty and usability, followed by a nine-item multiple-choice quiz assessing knowledge acquisition based on the investigated topics.

11.2 User study tasks↩︎

Participants completed three distinct information-seeking tasks, each designed to reflect a different academic domain and to evaluate the LLM’s effectiveness in supporting domain-specific knowledge acquisition. Each task lasted 25 minutes and was accompanied by detailed written instructions displayed within the study interface. The tasks were as follows:

• Chemistry Participants investigated the principles of molecular symmetry and the classification of molecules into point groups. Subtasks included: (i) identifying symmetry elements such as rotation axes, centers of inversion, and mirror planes; (ii) explaining the criteria for point group classification; and (iii) discussing correlations between point groups and spectroscopic properties, including infrared (IR) and Raman activity.

• Computer Science Participants explored two core process scheduling algorithms used in operating systems: priority scheduling and multilevel queue scheduling. Subtasks included: (i) analyzing the design and structure of priority scheduling; (ii) investigating the implementation of multilevel queue scheduling; and (iii) examining supporting mechanisms such as data structure design, time allocation methods, and priority handling logic.

• Business Participants studied key accounting issues encountered during the preparation of consolidated financial statements. Focus areas included: (i) elimination of investments in subsidiaries against equity; (ii) recognition and treatment of goodwill or bargain purchase gains; and (iii) accounting for non-controlling interests.

11.3 Participant demographics↩︎

Table 6 summarizes the demographic and academic background information of the study participants. The collected data include age (in years), gender, nationality, major or primary academic field, and current academic year.

11.4 Collected query expertise results↩︎

For each collected query, we recorded the query text, the user’s self‑reported expertise level, two LLM responses (baseline and expertise‑conditioned), and satisfaction ratings, along with raw keystroke logs capturing key press and release events with timestamps. In total, the dataset comprised 1270 queries spanning Chemistry, Computer Science, and Business, covering a range of expertise levels (651 Novice, 245 Beginner, 209 Intermediate, 120 Proficient, and 45 Expert queries). Table 7 shows how queries were distributed across expertise levels within each domain, indicating that most queries came from novice users, with decreasing numbers at higher expertise levels and some variation between domains.

11.5 Survey results↩︎

11.5.1 Quiz performance by self-assessed expertise↩︎

To analyze participants’ objective knowledge across domain-specific tasks, we examined quiz scores (out of 9) stratified by self-assessed expertise level for each task. Participants provided their overall expertise per task on a scale: Novice, Beginner, Intermediate, Proficient, and Expert. Given the relatively few Expert self-assessments, we combined Proficient and Expert into a higher proficiency group for analysis.

Overall, the results show a strong positive association between self-assessed domain expertise and objective knowledge, as measured by quiz scores. Across all three tasks, mean quiz performance increased monotonically with expertise level. For example, participants identifying as Proficient or Expert consistently scored above 7.0, whereas Novices averaged between 5.1 and 5.8 points. Notably, the magnitude of expertise-related differences varied by domain. The largest score gap was observed in Chemistry (Task 1), where Proficient/Expert participants averaged 7.3 compared to 5.1 among Novices (a 2.2-point difference). Computer Science (Task 2) exhibited the highest overall scores, with even Novices achieving a mean of 5.7, while Business (Task 3) showed intermediate patterns. Although the sample sizes for higher proficiency levels were small, the trend remains consistent across tasks. These results indicate that self-perceived expertise reliably predicts task-specific knowledge, while also revealing substantial domain-specific variability in baseline knowledge levels among participants.

11.5.2 Task experience and cognitive load by self-assessed expertise↩︎

Participants rated multiple aspects of their task experience using a combination of standardized and custom measures. Table 8 details the full survey questions, their scale types, and the abbreviated labels used in reporting, while Table 9 summarizes mean ratings for each measure stratified by self-assessed expertise group. The measures included perceptions of task appropriateness, difficulty, and immersion, as well as the perceived helpfulness and relevance of LLM assistance, all rated on 7-point Likert scales. Additionally, six NASA Task Load Index (NASA-TLX) subscales were administered, capturing cognitive and emotional workload dimensions (e.g., mental demand, effort, and negative affect) on 0–20 scales.

Consistent with expectations, self-assessed expertise was systematically associated with participants’ subjective task experiences (Table 9). Participants with higher expertise generally perceived the tasks as better matched to their abilities and reported lower levels of cognitive workload. In particular, Novice participants reported higher mental demand than Proficient/Expert participants (11.6 vs.). Negative affect (e.g., stress and frustration) likewise decreased with increasing expertise (7.8 vs.), indicating a more positive emotional experience among more experienced users. Perceived task success showed a modest increase with expertise (9.2 for Novices vs. for Proficient/Expert participants). In contrast, effort ratings were comparable across expertise levels, with Proficient/Expert participants reporting slightly higher effort (13.8) than Novices (12.2), despite lower reported mental demand. Ratings of LLM helpfulness and relevance showed relatively small differences across expertise levels(5.8–6.4), suggesting broadly consistent perceptions of system support. Overall, these results highlight systematic expertise-related differences in cognitive workload and affect, illustrating how participants’ perceived proficiency shaped their subjective task experience.

11.6 Follow-up Evaluation↩︎

For the follow-up study, we recontacted participants from the original study who had agreed to be contacted again for future research. Among those we were able to reach, 16 returning participants completed the follow-up evaluation and were included in the analysis. Participants received an additional compensation of 15 USD for completing the study.

In this follow-up evaluation, participants rated responses to queries they had originally written in the main study. For each query, they were shown responses from three conditions: (1) a baseline response without expertise conditioning, (2) a response conditioned on self-reported expertise, and (3) a response conditioned on  ExPerT-predicted expertise. The prompts used for expertise-conditioned generation were identical to those described in Appendix  9, with the predicted-expertise condition using the expertise level inferred by  ExPerT instead of self-reported expertise. The order of the conditions was randomized, and the identities of the conditions were blinded to participants. Participants rated each response on a 5-point satisfaction scale. This design allowed us to assess whether the benefits of expertise-conditioned generation persisted when the expertise signal was inferred by the model rather than explicitly provided by the user.

As shown in Figure  11, most participants preferred responses that incorporated expertise information over those that did not. At the individual level, both the reported-expertise and predicted-expertise conditions generally received higher satisfaction scores than the no-expertise condition, indicating that expertise-aware responses were more positively evaluated overall. While some participant-level variation remained, the advantage of the expertise-based conditions was consistently observed across a broad set of users rather than concentrated in only a few cases. Furthermore, the reported-expertise and predicted-expertise conditions were competitive, with many participants assigning similarly high ratings to both. This pattern suggests that predicted expertise can approximate the benefit of reported expertise in supporting user satisfaction. Taken together, these findings indicate that incorporating expertise information is beneficial at the participant level, and that inferred expertise may provide a practical alternative when self-reported expertise is unavailable.

12 Baselines for expertise inference↩︎

To benchmark our proposed approach, we implemented random guess baseline and two baseline expertise classification systems following prior work on persona-aware LLM prompting [10], [20]: a persona-based baseline and a session-based baseline. We additionally report a conventional supervised classifier baseline for completeness, although it is intended primarily as a supplementary reference point.

12.1 Baseline Details↩︎

12.1.0.1 Random guess baseline

For reference, we compute the expected performance of a random predictor that samples labels uniformly from the five-point ordinal expertise scale \(\mathcal{Y} = \{1, 2, 3, 4, 5\}\). Assuming that the true label \(Y\) is fixed and the prediction \(\hat{Y}\) is drawn uniformly at random from \(\mathcal{Y}\), the expected mean squared error (MSE) and mean absolute error (MAE) are given by \[\mathbb{E}\!\left[(Y - \hat{Y})^2\right] = \frac{1}{5} \sum_{y=1}^{5}\frac{1}{5}\sum_{\hat{y}=1}^{5} (y-\hat{y})^2 = 4.0,\] \[\mathbb{E}\!\left[|Y - \hat{Y}|\right] = \frac{1}{5} \sum_{y=1}^{5}\frac{1}{5}\sum_{\hat{y}=1}^{5} |y-\hat{y}| = 1.6.\] These values represent a random guess baseline, which any model leveraging semantic or behavioral information should outperform.

12.1.0.2 Persona-based baseline

We implement a persona-aware classifier that predicts user expertise using both query text and structured persona metadata (age, gender, nationality, native and foreign languages, academic major, study year, and self-reported LLM usage). For each query, the persona attributes are presented in the system prompt and the query text in the user prompt (see Box [box:persona-prompt]). The model outputs one of five ordered labels (Novice, Beginner, Intermediate, Proficient, Expert) along with a concise justification.

All experiments used OpenAI’s gpt-4.1 with temperature=0.0, top_p=1.0, and a maximum output length of 512 tokens to ensure deterministic outputs without truncation.

12.1.0.3 Session-based baseline

We implement a session-level classifier that predicts user expertise using only the linguistic content of all queries entered by a participant for a given task. Each session (grouped by user_id and task) is concatenated into a single text block and classified using GPT-4.1 with temperature=0.0, top_p=1.0, and a 512-token output cap to ensure deterministic responses. We adopt the session-level prompt template from Palta et al. [20] without modification, where the model infers an overall expertise label (Novice-Expert) based solely on aggregated query text.

Predictions are evaluated against ground-truth expertise labels obtained by aggregating participants’ self-assessed query-level ratings within each session.

12.1.0.4 IDL baseline

We implement an IDL-based baseline that adapts the core principles of In-Dialogue Learning (IDL) [14] to expertise inference, where stable user characteristics are induced from accumulated dialogue history without relying on predefined profiles. The baseline uses LLaMA-2-7B-Chat as the backbone model and operates on query histories spanning multiple turns and multiple dialogue sessions for each user-task pair. For each user, dialogue sessions are embedded using a sentence-level encoder and clustered via \(k\)-means to identify persona-relevant session groups. Within each cluster, reference sessions are dynamically reordered by minimizing inter-session conversational edit distance, producing a coherent dialogue sequence. The model is then trained using a Mutual Supervised Learning (MSL) objective following the IDL framework, with LoRA-based fine-tuning, and subsequently aligned via DPOC-based preference optimization to mitigate persona inconsistency and hallucination. At inference time, the LoRA- and DPOC-aligned model is provided with the concatenated dialogue history and prompted to infer the user’s overall expertise level on a five-point ordinal scale (Novice-Expert).

12.1.0.5 AI Persona baseline

We implement an AI Persona baseline following the life-long semantic personalization framework proposed in AI Persona [9]. This approach models user characteristics as an explicit persona state that is incrementally updated from accumulated interaction history, under the assumption that stable user attributes can be inferred through repeated dialogue with a language model. Concretely, for each user, we maintain a structured persona representation initialized from demographic and background information. Persona updates are performed by aggregating interaction histories over multiple sessions and prompting GPT-4.1 to revise the persona fields accordingly. Following the original AI Persona protocol, persona updates are triggered every \(k=3\) sessions, where all query—response pairs within the update window are concatenated and provided to the model for persona revision. For expertise prediction, the current persona state is combined with the user’s query and provided to GPT-4.1, which is prompted to classify the user’s expertise level on a five-point ordinal scale (Novice to Expert). All persona updates and expertise predictions are performed using GPT-4.1 with temperature set to 0.0, top_p set to 1.0, and a maximum output length of 512 tokens to ensure deterministic behavior. This baseline relies exclusively on semantic information encoded in dialogue content and persona representations, and does not incorporate query-level behavioral signals or fine-grained temporal features.

12.2 Conventional Supervised Classifier Baselines↩︎

To further examine whether  ExPerT’s gains stem from its in-context semantic-behavioral inference rather than from multimodal input availability alone, we additionally implemented three conventional supervised classifier baselines using the same query text and keystroke-derived inputs: (i) BiLSTM(text) + FFN(keystroke_flat)  [48], [49], (ii) Word2Vec(text) + FFN(keystroke_flat)  [50], and (iii) BiLSTM(text) + BiLSTM(keystroke_seq)  [51] with attention pooling and late fusion (concatenation followed by an MLP). These baselines are included as additional supervised reference points.

12.2.0.1 Inputs and architectures.

For all baselines, the text modality is constructed from the query text, and the behavioral modality is constructed from the same keystroke-derived signals used in our main task. The BiLSTM(text) + FFN(keystroke_flat) baseline encodes the query text with a BiLSTM and flattened keystroke features with a feed-forward network, followed by fusion for prediction. The Word2Vec(text) + FFN(keystroke_flat) baseline represents the query text using averaged Word2Vec embeddings and combines it with an FFN encoding of flattened keystroke features. The BiLSTM(text) + BiLSTM(keystroke_seq) baseline encodes the query text and keystroke sequence with separate BiLSTMs, applies attention pooling, and performs late fusion via concatenation followed by an MLP. In all cases, the final prediction target is our 5-level expertise label.

12.2.0.2 Evaluation setting.

We evaluate these supervised baselines using the individual split, where each user’s samples are divided into train/validation/test sets with a 70/15/15 ratio. Since this setting is the closest supervised counterpart to our within-user evaluation, we compare these results against  ExPerT under the within-user 10-shot setting.

12.2.0.3 Results.

Table 10 reports the results in terms of mean absolute error (MAE) and mean squared error (MSE), where lower values indicate better performance. Among the supervised baselines, BiLSTM + FFN achieves the best MAE, while Word2Vec + FFN achieves the best MSE. However, all three supervised baselines remain substantially behind  ExPerT, indicating that  ExPerT’s gains are not explained solely by access to both semantic and behavioral inputs, but are associated with its in-context LLM-based inference mechanism.

13 Per-class metrics↩︎

We additionally report per-class F1 scores with 95% bootstrap confidence intervals (1,000 resamples with replacement) in Table 11. The overall macro-F1 is 0.6102 with 95% CI [0.5682, 0.6479].

Performance is highest in the Novice class and lowest in the Expert class. The relatively lower F1 score and wider confidence interval for the Expert class may be partly explained by its underrepresentation in the dataset (n=45), which is substantially smaller than the other classes.

8pt

14 In-depth Statistical Analyses of User Keystroke Behavior↩︎

14.1 Expertise-level Differences↩︎

To assess whether keystroke-based behavioral features differ statistically across expertise levels, we conducted a Kruskal-Wallis test [52], a non-parametric method suitable for comparing multiple groups with potentially unequal sample sizes and non-normal distributions. To ensure consistency with the features used by ExPerTand to mitigate dependence among raw keystroke events, we performed this analysis on word-level aggregated behavioral features, treating each typed word as a single behavioral unit. The analysis was conducted across all expertise levels defined in our study.

We examined a range of keystroke metrics, including key duration, Down-Down (DD), Up-Down (UD), Up-Up (UU), and Down-Up (DU) inter-key intervals, as well as typing speed and backspace count. The results show robust and highly significant differences across expertise groups for nearly all behavioral features (all \(p < 0.001\)). Corresponding H-statistics indicate clear separation between expertise-level distributions, with larger values reflecting stronger group-wise differentiation.

These results provide statistical evidence that keystroke dynamics systematically vary with user expertise. In particular, higher expertise levels are associated with faster, more stable typing patterns and fewer correction behaviors, consistent with increased fluency and conceptual familiarity. This expertise-aligned structure of behavioral signals supports our claim that behavioral cues complement semantic information by mitigating ambiguity in query text and enabling more accurate expertise inference.

14.2 User-level Differences↩︎

6pt

To further investigate individual variability in keystroke behavior, we analyzed whether keystroke features differ significantly across users. We conducted a one-way ANOVA using a calibration dataset in which each participant typed a standardized paragraph before the main study, allowing for controlled comparison of typing behavior independent of task content.

The analysis included multiple keystroke features, such as key duration and DD, UD, UU, and DU inter-key intervals. The results reveal significant between-user differences for all examined features (all \(p < 0.05\)), with F-statistics indicating substantial separation among users. These findings confirm that keystroke behaviors are highly user-specific, consistent with prior studies on typing dynamics [36]–[38].

This strong user-level variability explains our empirical observation that within-user few-shot inference consistently outperforms cross-user settings. Because each user’s typing rhythm and fluency form statistically distinct distributions, models trained across users tend to average away meaningful individual differences, leading to degraded performance. In contrast, within-user examples preserve user-specific behavioral patterns, enabling more effective personalization.

15 Full Examples of Model Predicted Expertise and Reasoning↩︎

This appendix provides full qualitative examples illustrating how  ExPerT predicts user expertise and reasons over semantic and behavioral cues at the query level (Table 14. For each example, we present the original user input, the predicted expertise level, and the corresponding reasoning process that integrates semantic signals from the query text with behavioral signals derived from keystroke interactions. These examples complement the quantitative and statistical analyses in the main paper by offering concrete, case-level evidence of how  ExPerT adapts its inference to different users and contexts. Together, they demonstrate how the proposed framework resolves semantic ambiguity, leverages user-specific behavioral patterns, and produces expertise-aware predictions in practice.

16 Qualitative Comparison of Baseline and  ExPerT Responses↩︎

Tables 15 to 19 show examples of baseline and  ExPerT responses at different expertise levels.

As shown in Table 15, the  ExPerT response demonstrates a deliberate restructuring of the explanation to match the cognitive constraints of novice users. Rather than merely simplifying terminology,  ExPerT decomposes the target concept into a sequence of prerequisite notions (e.g., molecule → symmetry → molecular symmetry), thereby explicitly scaffolding understanding. This layered progression contrasts with the baseline response, which presents a coherent but largely narrative explanation without clearly isolating conceptual dependencies. The insight here is that  ExPerT does not only simplify content for novices, but actively reorganizes knowledge to minimize abstraction and reduce cognitive load, enabling comprehension without prior domain assumptions.

As shown in Table 16, the  ExPerT response shifts the explanatory focus from outcome reporting to perspective-oriented reasoning. Specifically, the  ExPerT response foregrounds the distinction between separate and consolidated financial statements before addressing the treatment of goodwill, making the underlying accounting logic explicit. By contrast, the baseline response primarily states where goodwill is recorded, offering limited insight into why that treatment arises. This indicates that, at the beginner level,  ExPerT emphasizes conceptual framing and viewpoint alignment, helping users internalize domain-specific reasoning patterns rather than memorizing isolated results.

As shown in Table 17, the  ExPerT response transitions from procedural illustration to rule-based abstraction. While the baseline response relies on a detailed scheduling scenario to demonstrate how Round Robin operates within priority and multilevel queue scheduling,  ExPerT extracts general principles (e.g., Round Robin as an intra-queue fairness mechanism or as a tie-breaking strategy among equal-priority processes). The key insight is that  ExPerT adapts to intermediate users by prioritizing transferable reasoning: explanations are structured so that the user can apply the same logic to unseen system configurations, rather than reproducing a single execution trace.

As shown in Table 18, the  ExPerT response emphasizes conceptual taxonomy and awareness of the design space. Instead of extending procedural examples,  ExPerT explicitly categorizes scheduling policies along orthogonal dimensions such as static versus dynamic priorities and preemptive versus non-preemptive execution. The baseline response, although comprehensive, remains example-heavy and less explicit about these structural distinctions. This suggests that  ExPerT aligns with proficient users’ expectations by highlighting comparative frameworks and architectural trade-offs, supporting analytical reasoning rather than operational understanding alone.

As shown in Table 19, the  ExPerT response demonstrates expert-level adaptation by embedding explanations within formal theoretical structures. The  ExPerT response connects IR and Raman activity to group-theoretic constructs such as irreducible representations, polarizability tensors, and selection rules derived from character tables. In contrast, the baseline response remains descriptive, focusing on physical conditions without explicitly linking them to symmetry formalism. The insight here is that  ExPerT does not merely add detail, but shifts the explanatory goal toward theoretical completeness and disciplinary rigor, reflecting how experts evaluate correctness and depth through formal frameworks rather than phenomenological description.

17 License for Artifacts↩︎

We follow the terms of use released by OpenAI (https://openai.com/policies/row-terms-of-use/).

18 Potential Risks↩︎

The proposed framework dynamically adapts large language model responses based on inferred user expertise. While this personalization is designed to improve user experience, it introduces potential societal risks. Expertise misclassifications could systematically disadvantage particular user groups or amplify pre-existing biases, especially for users with non-standard interaction patterns. Additionally, malicious actors could attempt to manipulate the system to receive inappropriate or misleading personalized outputs.

19 Implementation Details and Evaluation Packages↩︎

We used standard Python libraries for evaluation and statistical analysis. Evaluation metrics, including mean absolute error (MAE), mean squared error (MSE), and confusion matrices, were computed using the sklearn.metrics package with default parameter settings. Statistical significance testing was conducted using standard functions from the scipy.stats package. Specifically, the Wilcoxon signed-rank test using scipy.stats.wilcoxon, one-way ANOVA was performed using scipy.stats.f_oneway, and the Kruskal–Wallis test using scipy.stats.kruskal. All tests were applied following their conventional assumptions and default configurations.

References↩︎

---

1. https://github.com/UbiquitousAILab/ExPerT↩︎