1 Introduction↩︎

Large Language Models (LLMs) have fundamentally altered the landscape of software engineering, demonstrating exceptional capability in tasks ranging from automated code generation to bug localization. These models, typically based on the Transformer architecture, learn to predict tokens by minimizing loss over internet-scale repositories [1]. However, this process relies on “machine attention,” a statistical mechanism that treats context uniformly based on data correlations. In contrast, human developers employ distinct cognitive strategies characterized by program comprehension [2], [3]. Eye-tracking research consistently demonstrates that experts rely on intuition to form mental models. They fixate selectively on semantically critical elements, such as control flow conditions and method signatures, while skimming over syntactic sugar and boilerplate [4], [5].

The dominant paradigm for adapting CodeLLMs relies heavily on minimizing prediction error over vast datasets or aligning models via high-level instruction tuning. While these methods improve general adherence to user intent, they do not fundamentally alter the model’s underlying attention mechanism, which remains tethered to statistical co-occurrences rather than semantic reasoning. Existing attempts to bridge this gap, such as retrieval-augmented generation (RAG), provide external context but fail to teach the model how to process that context like an expert. Consequently, even state-of-the-art models continue to distribute attention uniformly across boilerplate and logic, missing the nuanced, selective focus that characterizes human program comprehension.

To bridge the critical gap between statistical machine attention and selective human focus, we propose EyeMulator, a methodology to ground CodeLLM training in cognitive visual patterns. We posit that while purely data-driven models excel at surface-level pattern matching, they fundamentally lack the ability to prioritize the logical dependencies, such as variable data flow and execution paths, that characterize expert comprehension. By explicitly aligning model attention with human scan paths, we enable the system to process code with a semantic salience mirroring that of a developer. Crucially, unlike prior gaze-aware approaches that necessitate specialized architectures or expensive pre-training from scratch [6], we aim to distill these cognitive insights into a modular, model-agnostic signal that can be seamlessly integrated into standard supervised fine-tuning pipelines.

The core innovation of EyeMulator is a generative attention mechanism that allows standard CodeLLMs to learn human cognitive patterns directly from text. We achieve this by first distilling raw eye-tracking data into two portable artifacts: Semantic Salience Priors, which model the static importance of token types (e.g., variables vs. keywords), and Transition Probabilities, which capture the dynamic flow of expert reading (e.g., jumping from definition to usage). These artifacts enable us to synthesize “pseudo-scan paths” for standard training data where no eye-tracking exists. We then align the model to these paths using a composite objective: a re-weighted Supervised Fine-Tuning (SFT) loss to emphasize salient tokens, and a token-level Direct Preference Optimization (DPO) loss that explicitly rewards the model for prioritizing human-preferred context over irrelevant boilerplate.

We evaluate EyeMulator on three diverse CodeXGlue tasks: code completion, Java-to-C# translation, and Java-to-English summarization. To test robustness, we run six small CodeLLM backbones across low-data and full-data regimes, with seed 42 and seed 12345 used for robustness checks. The cleaned results show positive quality-safe gains in every model–task–setting cell. Completion and translation provide the strongest evidence, with consistent Exact Match improvements across all six models and both data regimes; summarization is more recipe-sensitive but remains positive under METEOR after targeted tuning and DeepSeek tokenizer correction. Session-mode, component-ablation, and attention diagnostics further support the mechanism behind these gains.

In summary, this paper contributes: (1) a portable pipeline for converting raw human gaze traces into reusable semantic salience and sequential transition priors over code tokens; (2) a model-agnostic fine-tuning objective that incorporates these priors through token-level weighting and preference alignment without requiring architectural changes; and (3) a six-backbone evaluation across multiple CodeXGlue tasks and data regimes, showing that gaze-derived signals provide consistent quality-safe gains while revealing when human-attention cues are robust, task-sensitive, and complementary.

2 Approach↩︎

As illustrated in Figure 1, EyeMulator consists of three stages: processing raw gaze data, extracting statistical attention priors, and fine-tuning the model using a dual-objective loss.

2.1 Data Transformation and Metrics↩︎

We leverage the EyeTrans corpus [6], which contains 120Hz gaze recordings from 27 programmers reading and writing Java code. To transform raw sensor data into a training signal, we first apply a dispersion-threshold (I-DT) algorithm to identify Fixations (\(\mathcal{F}\)), defined as stable gaze points lasting at least 100 ms. These fixations serve as our primary metric of cognitive interest. We then spatially map these fixations to Abstract Syntax Tree (AST) leaf tokens using bounding boxes captured during the study (Figure 2). Unmapped points, which account for approximately 3% of the data, are discarded. This process yields 1,565 Scan Paths (\(\mathcal{P}\)), representing the chronological sequence of attended tokens aligned with the code structure.

2.2 Attention Pattern Extraction↩︎

We distill the token-aligned fixations into two statistical artifacts: semantic salience priors and sequential transition models.

2.2.0.1 Semantic Salience Priors.

To model the inherent importance of different token types (e.g., ForStatement vs. Identifier), we employ a Bayesian approach. For each semantic class \(s\), we count the number of fixations \(c_1(s)\) relative to the total number of available tokens \(n_s^{\mathrm{tok}}\). We model the probability of attention \(\theta_s\) using a Beta distribution \(\mathrm{Beta}(\alpha_s, \beta_s)\), which serves as a conjugate prior to the binomial likelihood of fixation. We set \(\alpha_s = c_1(s) + 1\) and \(\beta_s = n_s^{\mathrm{tok}} - c_1(s) + 1\). The posterior mean \(\mathbb{E}[\theta_s] = \alpha_s / (\alpha_s + \beta_s)\) provides a robust estimate of salience, smoothing out noise in rare token classes.

2.2.0.2 Sequential Transition Models.

To capture the flow of expert reading, we count bigrams and trigrams within the scan paths. We compute conditional probabilities \(P(s_b|s_a)\) and \(P(s_c|s_a,s_b)\), which represent the likelihood of transitioning between specific semantic states (e.g., from a variable definition to its usage). We prune n-grams with fewer than 5 occurrences to reduce noise.

2.3 Pseudo-Attention Path Generation↩︎

To simulate human attention for standard training data where no eye-tracking exists, we generate “pseudo scan paths” \(\tilde{\mathcal{P}}\). For an input sequence of length \(n\), we proceed in three steps: (1) We sample a saliency ratio \(\rho\) from the aggregate Beta distribution of the corpus. (2) We determine the total count of attended tokens \(m = \lfloor \rho n \rfloor\) and allocate quotas to specific token classes based on their posterior means \(\mathbb{E}[\theta_s]\). (3) We construct the path \(\tilde{\mathcal{P}}\) by greedily matching masked tokens into valid n-grams (prioritizing Trigrams, then Bigrams, then Monograms) that satisfy a line-span constraint \(L\). This procedure (Figure 3) synthesizes sequences that preserve both the local structure and the semantic selectivity of human attention.

2.4 Gaze-Informed Fine-Tuning↩︎

We seamlessly integrate these artifacts into standard LLM training using a novel weighting scheme and a composite loss function.

2.4.0.1 Weight Calculation.

Since LLMs typically use subword tokenizers (e.g., Byte-Pair Encoding), we project our AST-level weights by assigning the parent token’s weight to all its constituent subword shards. We compute a final training weight \(w_j\) for each token \(j\) by combining a base importance term, an inverse frequency term to upweight rare code constructs, and the semantic probability: \(w_j = w_{\text{base}} + (\log(\text{freq}(g_j)+2))^{-1} + \mathbb{E}[\theta_{s_j}]\).

2.4.0.2 Composite Objective.

We fine-tune the model using a loss function \(\mathcal{L}(\phi) = \mathcal{L}_{\mathrm{SFT}}(\phi) + \gamma \mathcal{L}_{\mathrm{DPO}}(\phi)\). The Weighted SFT Loss is a modification of the standard categorical cross-entropy, scaled by the calculated weights: \(\mathcal{L}_{\mathrm{SFT}}(\phi) = - \sum_{j} w_j \log P_\phi(x_j | x_{<j})\). The Token-Level DPO Loss adapts the Direct Preference Optimization framework [7] to our setting. DPO typically optimizes a policy to prefer a winning sample \(y_w\) over a losing sample \(y_l\). We define our “winning” trajectory as the generated pseudo-scan path (high salience) and the “losing” trajectory as the complement (low salience background tokens). This term explicitly rewards the model for assigning higher probability to the semantically salient tokens that humans prioritize.

2.5 Integrated Training Procedure↩︎

We synthesize the artifact extraction, path generation, and loss computation into a unified training loop. Algorithm 4 details the complete fine-tuning procedure.

The process begins by initializing the model \(\phi\) and loading the distilled EyeTrans artifacts (Semantic Priors \(\mathbb{E}[\theta]\) and Transition Probabilities \(P_{\mathrm{trans}}\)). For every batch of code sequences, we dynamically generate pseudo-scan paths \(\tilde{\mathcal{P}}\) that mimic human visual attention. These paths serve as the ground truth for calculating token-specific importance weights \(W\). Finally, the model parameters are updated via gradient descent on the composite objective \(\mathcal{L}_{\mathrm{total}}\), which balances the reconstruction of salient tokens (\(\mathcal{L}_{\mathrm{SFT}}\)) with the preference alignment of attention trajectories (\(\mathcal{L}_{\mathrm{DPO}}\)).

3 Experimental Setup↩︎

We design our experiments to evaluate the efficacy of gaze-informed training across different code intelligence tasks and model architectures. We specifically address five research questions: (RQ1) How faithfully do our distilled artifacts capture human attention patterns? (RQ2) Does mimicking human attention improve performance on downstream code-intelligence tasks? (RQ3) How do reading-derived versus writing-derived priors differ in their impact across tasks? (RQ4) How do the individual components of EyeMulator contribute to overall gains? (RQ5) Does the fine-tuned model actually learn to attend to semantically salient regions?

3.0.0.1 Datasets and Benchmarks.

We utilize the EyeTrans corpus [6] solely for extracting attention artifacts as detailed in Section 2. For downstream evaluation, we employ CodeXGlue [8], selecting three tasks to test generalization. For Code Translation (Java to C#), we use the standard Java-to-C# split. For Code Summarization (Java to English), we use 16,490 training, 518 validation, and 1,095 test examples. For Code Completion, we sample 20% of the CodeXGlue Java completion corpus, yielding 1,633 training, 1,005 validation, and 1,318 test examples. We report two data regimes: a low-data setting with 200 training examples and a full-data setting using the available training split. Completion and translation are evaluated with Exact Match, while summarization is evaluated with METEOR. Appendix 10 provides the metric definitions.

3.0.0.2 Models and Baselines.

To demonstrate model agnosticism, we apply EyeMulator to six backbones: StarCoderBase-1B, Llama-3.2-1B, DeepSeek-Coder-1.3B, Qwen2.5-Coder-1.5B, SmolLM3-3B, and Granite-Code-2B. Each baseline is fine-tuned on the same task data without gaze weights. EyeMulator uses task-level LoRA/QLoRA recipes, with code completion and translation using a rank-32, weight-4 recipe and summarization using lower-rank recipes selected from completed quality-safe variants. For robustness, we run the clean recipe on seeds 42 and 12345 and report best quality-safe results only when generated outputs pass qualitative sanity checks.

3.0.0.3 Implementation Details.

All experiments use HuggingFace transformers with parameter-efficient LoRA/QLoRA adapters. Input sequences are processed with a maximum length of 1024 tokens. Completion and translation use rank-32 LoRA with gaze weight 4 as the clean recipe; summarization uses lower-rank recipes selected from quality-safe sweeps. We use AdamW-style optimization, task-specific learning rates, and seeds 42 and 12345 for robustness checks. Detailed training configurations and attention-prior alignment steps are documented in Appendix 12.

4 Result Analysis↩︎

We organize the results from broadest to most diagnostic. We first establish the main effect across backbones, tasks, and data regimes (RQ2), then use session-mode and component-ablation analyses to explain which gaze signals matter (RQ3–RQ4). Finally, we retain the attention-map analysis as an appendix diagnostic rather than as the primary evidence (RQ5).

4.1 RQ1: Artifact Distillation↩︎

We assess how well distilled artifacts capture human attention by examining n-gram fixation patterns, fitted Beta parameters for semantic labels, and the resulting attention distributions across reading, writing, and combined tasks.

4.1.0.1 N-gram Analysis.

To assess the consistency and structure of human attention, we mapped programmers’ fixation sequences to semantic categories and extracted the most frequent transitions. As shown in Table 1, the resulting patterns reveal a non-linear yet highly organized reading strategy. The high count of function declaration \(\to\) variable declaration bigrams (8,399) indicates that developers frequently switch focus between interface definitions and their implementation details. Similarly, the conditional statement \(\to\) loop pattern (6,026) reflects an iterative inspection of branching logic and control flow. These recurring transition motifs demonstrate that expert attention is driven by logical dependencies rather than linear scanning.

4.1.0.2 Beta Parameter Estimation.

Figure 5 presents the fitted Beta parameters (\(\alpha_s=\) gaze hits, \(\beta_s=\) gaze misses) for each semantic label. Consistent with the n-gram findings, variable declaration exhibits the highest \(\alpha_{s}\) in both reading and combined tasks, reinforcing its role as a primary cognitive anchor for developers. In contrast, control-flow labels like loop show more balanced ratios (\(\alpha_{s}=7,876\), \(\beta_{s}=4,876\)), indicating that attention to these constructs is more context-dependent, shifting between cursory scanning and deeper inspection depending on the complexity of the logic.

4.1.0.3 Density Function Visualization.

Figure 6 illustrates the Beta distribution density functions derived from these parameters. Reading-only curves are sharply peaked; for instance, variable declaration centers tightly around a high probability, signifying focused and reliable inspection. Conversely, the distributions for conditional statement in the combined task are broader and even bimodal. This variance suggests divergent reading strategies, where developers may either skim standard conditions quickly or fixate heavily on complex predicates, confirming that our artifacts capture the nuance of cognitive load.

4.2 RQ2: Main Cross-Task Results↩︎

RQ2 asks whether gaze-derived priors improve downstream CodeLLM fine-tuning across tasks, data regimes, and backbones. The main result is that EyeMulator improves every evaluated cell under the matched task metric: 36 positive gains across six backbones, three tasks, and two data settings. Table 2 reports the selected quality-safe run for each cell, and Figure 7 summarizes the same evidence as a model-by-task profile.

4.2.0.1 Structure-Preserving Code Tasks.

The strongest evidence comes from completion and translation, where the target output preserves executable code structure. Completion gains are largest for code-specialized backbones after targeted recipe selection, reaching +34.4% relative improvement while remaining positive for all other models. Translation is less peaked but more uniformly stable, with relative gains up to +18.1% in the low-data setting and +10.8% in the full-data setting. These patterns suggest that gaze-derived priors are especially useful when semantic attention can directly constrain valid code tokens.

4.2.0.2 Natural-Language Summarization.

Summarization is more sensitive to decoding, tokenizer behavior, and LoRA recipe choice than the two code-structure tasks, because the target output no longer preserves the input program structure token by token. After excluding pathological outputs and applying the DeepSeek tokenizer fix, EyeMulator improves every summarization cell under METEOR, suggesting that gaze priors still provide useful semantic guidance even when code understanding must be expressed in natural language.

4.2.0.3 Backbone-Level View.

Figure 7 shows why evaluating multiple backbones is more informative than reporting a larger table alone. The average relative gain is similar for StarCoderBase, Qwen2.5-Coder, and SmolLM3 (about 16%), while Llama-3.2 and Granite-Code show smaller but still consistent gains (about 7–8%). This spread supports a model-agnostic interpretation without hiding backbone sensitivity: code-specialized backbones benefit most on completion, whereas Granite and Llama indicate that the signal is not limited to the strongest completion-tuned models.

4.2.0.4 Model and Data-Regime Robustness.

Across Table 2, the selected EyeMulator runs improve every task, backbone, and data regime under the matched metric. The results support the main claim under a conservative interpretation: human-attention priors provide consistent positive signal across architectures and data regimes, with the strongest effect on structure-preserving code generation and a smaller but positive effect on natural-language summarization. The breadth of this pattern is important because it rules out a single-model or single-task explanation. It also motivates the next two analyses, which ask which session source and which gaze-derived components account for the observed gains.

Taken together, RQ2 establishes the outcome-level effect. RQ3 and RQ4 then unpack this effect mechanistically: RQ3 varies the source of the human-attention prior, while RQ4 removes individual signal components from the full recipe.

1.8pt

4.3 RQ3: Session-Mode Analysis↩︎

Figure 8 shows that semantic categories vary across tasks, motivating a comparison of reading-derived (EyeMulator(R)) and writing-derived (EyeMulator(W)) priors. We rerun this analysis on current Llama-3.2-1B full-data artifacts, using Exact Match for completion and translation and METEOR for summarization. Table 3 reports all baseline and prior variants by group.

Completion subcategories. Writing-derived and full priors are strongest for code-production cases. The full variant leads on Concurrency/Lambda (+13.1%), JUnit Test Fixture (+3.1%), and completion (+3.9%), while writing ties on Control Flow and Boilerplate. This suggests that writing traces capture generation-oriented dependencies, with the cleaned runs giving a more conservative estimate than earlier validation results.

Translation and summarization subcategories. Reading-derived priors are strongest for translation overall (+5.0%) and for Multi-statement Control Flow, Inheritance, and Primitives & Operators. The full variant is strongest for Exceptions, and writing is strongest for Generics/Collections. Summarization is more mixed under METEOR: reading leads Async/Concurrency, Annotations, Return/Exception Semantics, and the overall aggregate, while writing leads API Contract and the full model leads Parameter Docs. These results indicate that comprehension-oriented reading priors help many input-understanding cases, but no single session mode dominates all natural-language summary categories.

Combined session effect. The full-prior variant remains robust, especially for completion, but the current Llama results show a more nuanced pattern than the previous draft: specialized session priors often win individual rows. We therefore treat session mode as a task- and category-dependent design choice rather than as a universally monotonic improvement from combining reading and writing statistics.

4.4 RQ4: Component Ablation↩︎

RQ4 examines whether the full gaze signal matters beyond any single component. Table ¿tbl:tab:current95rq495ablation? reports complete component-removal sweeps spanning all three tasks: completion on Llama-3.2-1B, and translation and summarization on DeepSeek-Coder and StarCoder. Removing either AST salience or rarity/transition flow reduces performance relative to the full recipe, and randomizing the weights also underperforms Full. This pattern supports the full EyeMulator recipe: semantic salience and sequential gaze flow are complementary rather than interchangeable.

4pt

4.5 RQ5: Attention Distribution↩︎

We treat attention distribution as diagnostic evidence rather than as part of the current six-backbone confirmation grid. Table 4 recomputes the earlier attention-quality view on the selected current Llama-3.2-1B full-data adapters, using 24 held-out examples per task. The clearest current signal is improved generation confidence, especially for summarization; distributional attention measures are smaller and task-dependent, so we use them to interpret behavior rather than to replace the main quality results in Table 2.

4.5.0.1 Qualitative Analysis.

Figure 9 visualizes final-layer attention from generated tokens back to input tokens. We keep this qualitative view as an illustrative diagnostic rather than as final quantitative evidence. A broader selection of qualitative case studies is presented in Appendix 13.

5 Discussion↩︎

EyeMulator builds on a simple observation: developers do not process code uniformly. They revisit declarations, control-flow predicates, API contracts, and other semantic anchors while skimming boilerplate. Our results suggest that compact CodeLLMs benefit when this selectivity is distilled into training-time priors rather than architecture changes.

5.0.0.1 Why the gains are task-dependent.

The largest improvements appear in completion and translation, where outputs preserve program structure and exact token choices matter. In these settings, gaze-derived weights help the model prioritize variables, signatures, and control-flow anchors that constrain valid code. Summarization is more diffuse because the output is natural language, making gains more dependent on recipe selection and tokenizer behavior. This helps explain why summarization gains are smaller yet still positive under METEOR.

5.0.0.2 Why modular priors matter.

Because EyeMulator changes only the training objective and data weights, it can be applied across heterogeneous backbones without requiring a custom transformer. The component ablation further shows that semantic salience and transition/rarity flow are complementary: removing either signal consistently weakens the full recipe. This supports the view that human attention is useful not as a single heuristic, but as a structured prior combining what developers inspect and how they move through code.

5.0.0.3 Interpreting diagnostic evidence.

The attention diagnostics serve as behavioral support, not the primary evaluation target. They show that gaze-informed fine-tuning can shift model behavior in interpretable ways, while downstream task quality remains the central evidence. Thus, EyeMulator’s main claim rests on improvements across the evaluation grid, with attention analyses explaining why those gains are plausible.

6 Related Work↩︎

Prior work in code intelligence has advanced attention-based Transformers, yet often omits human cognitive cues about how developers actually inspect code. EyeMulator unifies these directions by integrating distilled attention priors, n-gram rarity weighting, and sequential gaze modeling into a single framework that can guide standard CodeLLM fine-tuning.

6.1 Human-centered AI for Software Engineering↩︎

Human-centered AI integrates cognitive insights to align models with developer workflows [9]–[12]. Eye-tracking has historically illuminated how programmers manage cognitive load [4], [13]–[15], identifying key segments to improve automated summarization [13], [16], [17]. Recent work deepens this integration by correlating mouse interactions with neural attention [18], training predictive gaze models [19], and incorporating gaze into transformer architectures [6] or program repair [20]. Unlike these approaches, EyeMulator distills gaze artifacts into modular, task-agnostic priors that can be injected into any pre-trained model without architectural changes, preserving sample efficiency.

6.2 Large Language Models for Code Intelligence↩︎

LLMs such as StarCoder [21], Llama-3.2 [22], [23], and DeepSeek-Coder [24] have advanced code generation [25]–[28]. Strategies to refine performance include Retrieval-Augmented Generation (RAG) [29]–[32], instruction tuning [33], and reasoning frameworks like SemCoder [34]. However, models still struggle with deep semantic understanding [35]–[38], leading to hallucinations [39]–[41]. Existing feedback methods often lack token-level granularity [42], [43]. EyeMulator bridges this gap by injecting gaze-derived salience directly into self-attention, enhancing semantic grounding.

6.3 Preference Learning and Model Alignment↩︎

Preference learning aligns models with developer needs beyond simple correctness [44]–[47]. While Reinforcement Learning from Human Feedback (RLHF) [33], [48]–[50] is standard, direct optimization methods like DPO [7], SimPO [51], and KTO [52] offer efficient alignment. Techniques such as Group Relative Policy Optimization (GRPO) [53] further stabilize training. EyeMulator extends this landscape by incorporating gaze-derived salience priors as token-level feedback within DPO, enabling precise alignment with human cognitive processes.

7 Conclusion↩︎

We present EyeMulator, a lightweight, model-agnostic framework that injects human gaze signals into CodeLLM fine-tuning. By mapping eye-tracking data from 27 programmers onto AST tokens, we derive semantic salience and gaze-transition priors, then incorporate them into parameter-efficient fine-tuning. Across six backbones, three tasks, and two data regimes, EyeMulator yields positive quality-safe gains in every evaluated cell, with the strongest effects on completion and translation and smaller but consistent gains on summarization. Session-mode and component-ablation analyses, together with appendix diagnostics, show that gaze-derived signals change model behavior in interpretable, task-dependent ways.

Acknowledgments↩︎

We thank the study participants from Vanderbilt University and the University of Notre Dame for contributing to the human gaze data that supports this work. We also thank our collaborators and lab members for their feedback on the eye-tracking setup, artifact preparation, and manuscript presentation.

8 Limitations↩︎

While EyeMulator demonstrates significant improvements in code intelligence tasks, we acknowledge several limitations in our current study, spanning model scale, language coverage, and participant demographics.

8.0.0.1 Model Scale and Compute.

Due to computational resource constraints, our evaluation focuses on small code language models from roughly 1B to 3B parameters. This range allows broad coverage across six backbones and two data regimes, but does not establish scaling behavior for 7B, 13B, or larger systems.

8.0.0.2 Language and Task Diversity.

Our priors are distilled from gaze recordings on Java code (EyeTrans) and evaluated on Java/C# static code comprehension tasks, so transfer to structurally distinct languages (e.g., Python, Haskell) or markup (HTML/CSS), as well as to dynamic, interactive editing environments that require temporal gaze modeling, remains unverified.

8.0.0.3 Participant Demographics.

The gaze patterns were distilled from a cohort of 27 verified programmers. While this sample size is consistent with prior eye-tracking research, it may not fully capture the cognitive diversity of the global developer population across different experience levels, neurodiverse traits, or cultural coding practices.

9 Ethical Considerations↩︎

9.0.0.1 Potential for Misuse.

Gaze analysis carries a dual-use risk in workplace surveillance. EyeMulator is designed only to distill aggregated cognitive patterns for model training, not to assess or track individual developers, and we discourage any use that blurs this boundary.

9.0.0.2 Data Privacy and Dataset Usage.

We use the publicly available EyeTrans dataset under its original IRB-approved consent and anonymization protocols, and further mitigate re-identification risk by releasing only aggregated statistics (Beta priors and n-gram counts), not per-participant traces.

9.0.0.3 Use of AI Assistants.

The authors used AI assistants (LLMs) for minor editorial assistance, including language polishing and LaTeX formatting. All research ideation, experimental design, implementation, analysis, and claims are the authors’ own.

Data Availability Statement↩︎

The distilled human-attention artifacts, including gaze priors, a schema-matching dataset sample, and reference implementations, are archived on Zenodo at https://zenodo.org/records/17205682; the underlying eye-tracking data originates from the EyeTrans corpus [6] and is redistributed under its original ethical-use terms.

10 Evaluation Metrics↩︎

To ensure reproducibility, we provide formal definitions for all metrics used to evaluate task performance (RQ2–RQ4).

10.1 Task Performance Metrics↩︎

10.1.1 Code Translation and Completion↩︎

For Java-to-C# translation and code completion, the main tables report Exact Match, with additional code metrics available for audit:

• Exact Match: A prediction receives credit when the whitespace-normalized prediction and reference are identical, or when one contains the other as a complete target span: \[\text{Exact} = \mathbb{I}(\hat{y} = y \lor \hat{y} \in y \lor y \in \hat{y})\] where \(y\) is the ground truth, \(\hat{y}\) is the generated code, and \(\mathbb{I}(\cdot)\) is the indicator function. This exact-style criterion is appropriate for code tasks where generations may include the target span plus surrounding syntactic context.

• Hybrid Exact Match (H-Exact): To account for minor formatting variations while rewarding precision, we calculate a weighted average of strict exact match and substring inclusion: \[\text{H-Exact} = 0.5 \times \mathbb{I}(y = \hat{y}) + 0.5 \times \mathbb{I}(\hat{y} \in y)\]

• CodeBLEU: Unlike standard BLEU, CodeBLEU integrates syntactic and semantic properties. It is computed as the weighted sum of four components: \[\begin{align} \text{CodeBLEU} &= w_1 \text{BLEU} + w_2 \text{BLEU}_{\text{weighted}} \\ &\quad + w_3 \text{Match}_{\text{ast}} + w_4 \text{Match}_{\text{df}} \end{align}\] where \(\text{BLEU}_{\text{weighted}}\) gives higher weight to keywords, \(\text{Match}_{\text{ast}}\) measures Abstract Syntax Tree similarity, and \(\text{Match}_{\text{df}}\) measures data-flow graph similarity.

• CrystalBLEU: A variant of BLEU optimized for code that filters out “trivially shared” \(n\)-grams (e.g., frequent syntax like public void) to prevent inflated scores. It calculates \(n\)-gram precision only on a distinct set of \(n\)-grams not appearing in the top-\(k\) most frequent occurrences in the training corpus.

10.1.2 Code Summarization↩︎

For Java-to-Natural Language summarization, the main tables report METEOR, with other standard text-generation metrics used for audit:

• METEOR: Computes the harmonic mean of precision and recall, incorporating stemming and synonym matching to capture semantic overlap.

• ROUGE-L: Measures the Longest Common Subsequence (LCS) between the candidate summary and the reference, capturing sentence-level structure.

• BERTScore: Computes the similarity between candidate and reference summaries using contextual embeddings from a pre-trained BERT model: \[R_{\text{BERT}} = \frac{1}{|x|} \sum_{x_i \in x} \max_{y_j \in y} \mathbf{x}_i^\top \mathbf{y}_j\] where \(\mathbf{x}_i\) and \(\mathbf{y}_j\) are the embedding vectors for tokens in the candidate and reference, respectively.

11 Initial Validation Results↩︎

Table 5 preserves the earlier result table. We include it as historical and diagnostic evidence that the method improved over matched baselines in the initial three-backbone study. The current main-text Table 2 should be read as the final confirmation: it uses the cleaned six-backbone evaluation, two data regimes, current quality-safe recipe selection, and the final metric choices described in Appendix 10. The qualitative attention map in Figure 9 comes from the same initial diagnostic package and is retained to illustrate the attention-shift intuition.

12 Implementation Details↩︎

To ensure reproducibility, we provide the configuration for EyeMulator and the corresponding baselines across the six-backbone evaluation.

12.1 Model and Loss Configuration↩︎

We apply the same gaze-weighting mechanism to six backbones: StarCoderBase-1B [21], Llama-3.2-1B [22], [23], DeepSeek-Coder-1.3B [24], Qwen2.5-Coder-1.5B, SmolLM3-3B, and Granite-Code-2B. Each baseline uses standard causal language modeling, while EyeMulator scales per-token loss by gaze-derived weights \(w_j\). The current experiments use LoRA/QLoRA adapters rather than full fine-tuning, which keeps the method lightweight and makes the six-model grid tractable.

12.2 Hyperparameters↩︎

We use task-level recipes rather than a single universal setting: completion and translation use rank-32 LoRA with weight 4, while summarization uses lower-rank variants selected from completed quality-safe sweeps. Table 6 summarizes the main shared settings.

12.3 Attention Prior Processing↩︎

Gaze-derived priors are injected into the loss function at the subword level. For each training example, we first run a language-specific AST parser over the source code and assign every leaf token a semantic label (e.g., MethodDeclaration, IfStatement, VariableDeclarator). Each label is then mapped to its posterior salience \(\mathbb{E}[\theta_s]\) via the Beta priors distilled in Section 2.2. When the tokenizer splits an AST token into multiple BPE subwords, the parent’s salience is propagated to each shard. The final per-token weight \(w_j\) combines this salience with a base term and an inverse-frequency correction (Section 2.4). At training time, these weights are materialized into a single tensor that is broadcast against the cross-entropy loss, so that the procedure adds negligible overhead relative to standard fine-tuning.

For the Reading-derived (EyeMulator(R)), Writing-derived (EyeMulator(W)), and Full variants in Table 3, the only change is the source of the Beta and n-gram statistics (reading sessions, writing sessions, or both); the model, optimizer, and training schedule remain identical across all configurations.

13 Qualitative Case Studies↩︎

We present specific cases where EyeMulator demonstrates superior logic and conciseness compared to the baseline model.

13.1 Case Study 1: Java-to-C# Translation↩︎

In this example, the baseline fails to terminate cleanly, producing a syntax error and unreachable code, while EyeMulator recognizes the high-level intent and uses a built-in C# method.

13.2 Case Study 2: Code Summarization↩︎

This case illustrates the “repetition loop” common in smaller LLMs: the baseline produces redundant Javadoc-style guarantees, whereas EyeMulator emits a single focused sentence.

14 Future Work↩︎

Several directions could extend EyeMulator beyond the current camera-ready scope. Future work should examine whether gaze-derived priors remain effective for stronger instruction-tuned and agentic models, transfer beyond Java-centered benchmarks to broader programming settings, capture richer temporal attention signals, and improve developer-facing coding workflows.

Scaling to larger models. Our experiments focus on compact 1B–3B CodeLLMs, where full cross-task sweeps are tractable. Applying the same priors to larger instruction-tuned and agentic coding models would test whether gaze-derived signals still add value when base models already encode stronger code structure and reasoning patterns.

Expanding language and task coverage. The present study uses Java-centered gaze artifacts and CodeXGlue-style tasks. Extending the pipeline to additional languages, multi-file contexts, and repository-level tasks would clarify whether the learned signal reflects Java-specific habits or broader program-structure priors.

Modeling richer gaze dynamics. Our current priors summarize fixation and transition patterns, but eye-tracking data also contains dwell duration, regressions, scan-path uncertainty, and temporal shifts in attention. Incorporating these signals could support softer, context-dependent token weights that better distinguish quick skimming from deeper semantic inspection.

Studying interactive use. Developer studies with EyeMulator-adapted models would complement offline metrics by evaluating practical effects in coding workflows. Such studies could measure whether gaze-informed outputs reduce review effort, debugging time, or cognitive load during real development tasks.

References↩︎