1 Introduction↩︎

In recent years, Software Engineering (SE) has experienced a significant transformation, largely driven by the integration of Artificial Intelligence (AI) techniques into development practices. Among these, Large Language Models (LLMs) have emerged as powerful tools for automating a wide range of SE-related tasks, enabling developers and practitioners to work more efficiently. Collectively referred to as Large Code Models (LCMs), these models have been successfully applied to various aspects of software development automation, including code generation [1], [2], code summarization [3], [4], code translation [5], and test case generation [6], showcasing their versatility and impact across the whole spectrum of activities characterizing the software development lifecycle.

However, the remarkable capabilities of LCMs come at a significant cost. Models with billions of parameters demand substantial computational resources for inference, and their deployment often requires expensive GPU infrastructure that is inaccessible to many practitioners – particularly those in smaller organizations or academic settings who wish to deploy models locally for privacy, customization, or cost reasons [7], [8]. The environmental implications are equally concerning: training and operating these models at scale produces a considerable carbon footprint, raising questions about the long-term sustainability of current development trajectories [9]–[11].

In this context, quantization [12], the process of reducing the numerical precision of model parameters from 16- or 32-bit floating point to compact representations such as 4-bit integers – has emerged as one of the most practical strategies for bridging the gap between model capability and deployment feasibility. Unlike other efficiency techniques such as knowledge distillation [13] or pruning [14], quantization operates directly on pre-trained model weights without requiring retraining, making it especially attractive for practitioners who need to deploy existing models on resource-constrained hardware.

The growing popularity of quantized models reflects this need in practice. Lightweight local inference frameworks such as Ollama [15] now enable developers to run large language models directly on personal machines, and recent analyses report hundreds of thousands of deployed Ollama instances worldwide, highlighting the rapid growth of local LLM deployments outside centralized cloud infrastructures. At the same time, model repositories such as Hugging Face host a rapidly expanding ecosystem of quantized checkpoints; for example, the widely used TheBloke repository alone provides thousands of quantized model variants across multiple architectures and quantization formats [16].

While this rapidly growing ecosystem makes quantized models increasingly accessible, it also introduces substantial variability in the techniques used to compress them. As a result, an emerging body of research has begun to investigate how different quantization strategies affect the behavior of large code models and in which scenarios of code generation this behavior drifts compared to its full-precision counterpart. Early investigations into this question have yielded encouraging but incomplete answers. Wei et al. [17] demonstrated that 8-bit quantization can preserve code generation performance with improved energy efficiency. Giagnorio et al. [18] pushed the frontier further, showing that 4-bit quantization reduces memory by approximately 70% without significant degradation, establishing 4-bit as the practical operating point. Nyamsuren [19] reinforced this finding across low-resource programming languages. More recently, Afrin et al. [20] took the first step beyond functional correctness by examining how quantization affects code quality attributes such as maintainability and structural complexity, finding that AWQ largely preserves these properties.

However, quantization itself is a fundamentally lossy compression: it reduces numerical precision and the model’s representational capacity, both of which support reasoning over syntax, semantics, and task-specific patterns [21]. This suggests that any loss in generated-code quality may be non-uniform, varying with both the quantization technique applied and the complexity of the input being processed. Existing experimental designs, by averaging across techniques and tasks, may therefore obscure such systematic effects rather than reveal their absence.

All prior work has established quantization as a viable compression strategy for code LLMs, though several important gaps remain. First, existing studies evaluate only a small subset of quantization techniques, despite the Post Training Quantization (PTQ) landscape now encompassing multiple algorithmic families with potentially different behaviors.

Second, most evaluations rely solely on pass@\(k\), overlooking code quality aspects such as maintainability and structural complexity [22], [23]. Third, prior work treats benchmark tasks uniformly and does not examine whether quantization effects vary with prompt complexity, which we approximate using observable properties of the input prompt such as token length and information density (i.e., Shannon Entropy [24]), potentially placing different demands on compressed model representations.

To address these gaps, we conduct an empirical study of six widely used weight-only quantization techniques – GPTQ, AWQ, QuIP#, AQLM, BitsAndBytes, and GGUF – applied at 4-bit precision to two large code model families. We evaluate their impact on functional correctness (pass@1) and code quality (maintainability, reliability, security, and structural complexity) across multiple benchmarks and programming languages, and further analyze how robustness varies with prompt complexity, approximated through token length and Shannon entropy.

Our results show that 4-bit quantization largely preserves functional correctness, but the choice of technique introduces meaningful variation. AQLM consistently matches or exceeds the full-precision baseline, whereas QuIP# shows the largest degradation on complex tasks. Code quality effects are limited but observable: security remains stable, AWQ increases maintainability issues in Java, and BitsAndBytes produces the largest degradation in Python complexity metrics. Robustness to prompt complexity is also strongly model-dependent. These findings provide practical guidance for selecting quantization strategies and highlight the importance of evaluating quantized models beyond functional correctness. All artifacts are released to support replication [25].

The remainder of this paper is organized as follows. Section 2 reviews background and related work. Section 3 describes the study design. Section 4 details the implementation and evaluation setup. Section 5 presents the empirical findings. Section 7 discusses threats to validity, and Section 8 concludes the paper.

2 Background and Related Work↩︎

2.1 Efficiency in Large Code Models↩︎

The deployment of billion-parameter large code models imposes substantial computational and energy demands, raising concerns about environmental sustainability [9]–[11], particularly for practitioners deploying models locally for privacy, customization, or cost reasons [7], [8]. Shi et al. [26] identified four key areas for improving sustainability—data reduction, model-centric, system-centric, and program-centric approaches—while Shi et al. [27] demonstrated that model compression can reduce energy usage by up to 184\(\times\) and carbon emissions by 157\(\times\) with negligible effectiveness loss.

Several directions address these challenges. Parameter-Efficient Fine-Tuning (PEFT) updates only a small parameter subset via adapters [28], LoRA [29], or prompt tuning [30], [31], with promising results on code tasks [32]–[35]. Knowledge Distillation (KD) trains smaller models to emulate larger ones [36]–[38], but still requires teacher model access. Pruning removes redundant parameters at the cost of potential performance degradation [14]. Among these, quantization—a model-centric approach [26]—is particularly practical: it directly reduces the precision of pre-trained parameters without retraining, making it well-suited for deploying code LLMs on resource-constrained hardware.

2.1.1 Quantization↩︎

Quantization reduces memory footprint and computational cost by representing model parameters in lower-precision formats (e.g., 4-bit integers) instead of standard 16/32-bit floating point [12], [39]. Quantization-Aware Training (QAT) [40] integrates quantization into training but requires full retraining—prohibitively expensive for billion-parameter LLMs [41], [42]. Post-Training Quantization (PTQ) [43] converts pre-trained models using only a small calibration dataset, making it the dominant approach for compressing large code models and the focus of our study.

We organize weight-only PTQ approaches into six categories (Table ¿tbl:tab:quant-techniques?). Four are algorithmic: second-order/Hessian-informed methods (GPTQ [44], OWQ [45], SpQR [46]) minimize quantization error using approximate Hessian information; activation-aware saliency methods (AWQ [47], SqueezeLLM [48], OmniQuant [49]) protect critical weight channels via activation-magnitude analysis; rotation/incoherence-based methods (QuIP [50], QuIP# [51], QuaRot [52], FlatQuant [53]) apply orthogonal transforms to spread error uniformly; and vector quantization methods (AQLM [54], VPTQ [55]) encode weight groups through learned codebooks. Two are ecosystem-level: library/kernel-based implementations (BitsAndBytes [56], HQQ [57]) provide efficient low-bit kernels with seamless framework integration, and format/runtime-based solutions offer standardized formats for CPU-friendly inference. Weight-and-activation quantization—compressing both weights and activations—is more challenging due to input-dependent outliers; methods such as ZeroQuant [58], SmoothQuant [59], LLM.int8() [60], RPTQ [61], and PB-LLM [62] remain less widely adopted for code tasks. Jin et al. [63] corroborated that 4-bit quantization retains performance comparable to full-precision counterparts across instruction-tuned LLMs (7B–72B).

For our study, we select one approach from each category: GPTQ [44] (second-order/Hessian-informed), AWQ [47] (activation-aware saliency), QuIP# [51] (rotation/incoherence-based), AQLM [54] (vector quantization), BitsAndBytes [56] (library/kernel-based), and GGUF (format/runtime-based), ensuring broad coverage across four algorithmic strategies and two dominant deployment pathways, all widely supported in major inference frameworks and with publicly available implementations. Detailed descriptions and configurations are in Section 3.

2.1.2 Quantization for Code Models and Code Quality↩︎

While quantization has been extensively studied for general NLP tasks, its application to code-related activities remains comparatively nascent. Wei et al. [17] conducted the first large-scale investigation, examining 8-bit quantization on PLBART [64], CodeT5 [65], InCoder [66], and CodeGen [67] for code generation and summarization, finding that quantized models achieve improved energy efficiency with only marginal performance loss. Giagnorio et al. [18] extended this to larger models (CodeLlama up to 34B, DeepSeek-Coder up to 33B) using AQLM at 2-bit precision, showing that 4-bit quantization reduces memory by approximately 70% without significant degradation, while more extreme levels (2–3 bits) incur notable drops that can be partially mitigated through code-specific calibration and post-quantization fine-tuning. Nyamsuren [19] evaluated five 7B models quantized in GGUF format for Lua code generation on consumer hardware, reinforcing the 4-bit trade-off while showing that degradation is more pronounced for low-resource languages. In another work [20] presented the first study to go beyond functional correctness and examine quantization’s impact on code quality, finding that AWQ at 4-bit largely preserves quality metrics captured by static analysis tools.

Beyond quantization, a growing body of research recognizes that functional correctness alone is insufficient to characterize the practical utility of generated code [23], [68]. Siddiq and Santos [22] found that LLM-generated code frequently contains quality issues despite being syntactically valid, while Liu et al. [69] and Kharma et al. [70] examined iterative refinement and security vulnerabilities, respectively. Yetiştiren et al. [23] evaluated code quality across multiple dimensions including maintainability and reliability.

Despite this progress, existing studies have predominantly focused on a single quantization technique per study, with pass@\(k\) as the primary evaluation criterion. Afrin et al. [20] took the first step toward examining quantization’s impact on code quality but were limited to AWQ alone. The present study addresses this gap by comparing six quantization techniques and assessing their differential effects on both functional correctness and code quality.

3 Study Methodology↩︎

We evaluate six widely adopted weight-only quantization approaches—GPTQ, AWQ, QuIP#, AQLM, BitsAndBytes, and GGUF—applied at 4-bit precision to Qwen2.5-Coder-7B and CodeLlama-7B, and structure our investigation around the following research questions:

• RQ\(_1\): How do different quantization techniques impact the functional correctness of code generated by large code models?

  In this RQ, we assess functional correctness using the pass@1 metric across two benchmarks (McEval [71] and CoderEval [72]) and two programming languages (Python and Java). By contrasting each quantized variant against its full-precision baseline, we aim to determine whether certain quantization approaches preserve correctness more effectively than others, or whether code generation capabilities degrade uniformly across compression strategies.

• RQ\(_2\): How do different quantization techniques impact the quality attributes of automatically generated code?

  Beyond correctness, RQ\(_2\) examines maintainability, reliability, security, and structural complexity using SonarCloud [73] static analysis. Code that passes test cases may still exhibit excessive complexity or subtle defects that diminish its practical value [22]; we compare quality attributes across all six approaches to identify whether certain methods introduce more pronounced degradation.

• RQ\(_3\): Does input prompt complexity influence quantization-induced correctness degradation?

  We hypothesize that more complex prompts—longer inputs with higher information density—may be disproportionately affected by weight compression. To test this, we evaluate all six approaches on three benchmarks forming a natural complexity gradient: McEval [71] (short, simple prompts), CoderEval [72] (mid-length, context-dependent), and BigCodeBench [74] (long, multi-library orchestration). This analysis is conducted on Python only. We characterize complexity using token length, and Shannon entropy [24], and examine whether correctness degradation correlates with these measures.

The remainder of this section describes our experimental setup, covering the selection of code generation models (Section 3.1.1), the evaluation benchmarks (Section 3.1.2), the quantization approaches under investigation (Section 3.2), and the static analysis tools employed for quality assessment (Section 3.3).

3.1 Code Models and Dataset↩︎

This section outlines the code model families used in our experiments and the benchmarks employed to evaluate code generation across multiple programming languages.

3.1.1 Code Models↩︎

We employ two well-established code model families: Qwen2.5-Coder [75] and CodeLlama [76]. Both have been extensively adopted in code generation and quantization research [18], [20], [77]–[80]. Following established practices [18], [20], [80], [81], we use the instruction-tuned variant of each model.

Qwen2.5-Coder [75] is a code-specialized adaptation of the Qwen2.5 architecture, pre-trained on over 5.5 trillion tokens comprising 70% code, 20% natural language, and 10% mathematical content [75]. The family spans 0.5B to 32B parameters and has demonstrated strong performance across diverse code tasks [74], [82].

CodeLlama [76] is built on Llama-2 [83] and further trained on 500 billion tokens of natural language and source code. It offers general-purpose and instruction-tuned variants ranging from 7B to 70B parameters.

For both families, we restrict experiments to the 7B variant. Prior research shows that correctness and quality improve with model size but with diminishing returns [17], [18], [20]; the 7B configuration is representative of the resource-constrained settings where quantization is most commonly applied. Constraining to a single size also keeps the combinatorial experimental space (six approaches \(\times\) two benchmarks \(\times\) two languages, plus a third benchmark for RQ\(_3\)) manageable while enabling deeper cross-technique analysis.

3.1.2 Evaluation Benchmarks↩︎

We employ three code generation benchmarks. McEval [71] and CoderEval [72] are used in both Python and Java for RQ\(_1\) and RQ\(_2\), and in Python for RQ\(_3\). BigCodeBench [74] is used in Python only and contributes to all three RQs, additionally spanning a gradient of input complexity (Python only) in RQ\(_3\).

McEval [71] provides human-annotated, multilingual coding tasks with function signatures, docstrings, and test cases at varying difficulty levels. We use the Python (42 tasks) and Java (53 tasks) subsets. McEval represents the simplest end of our complexity spectrum, with concise function-level prompts.

CoderEval [72] comprises 230 Python and 230 Java tasks from real-world open-source projects, spanning six levels of context dependency—from self-contained functions to project-level contexts. Unlike standalone-only benchmarks, it reflects pragmatic generation scenarios and represents mid-length input complexity with richer contextual information. We use the filtered CoderEval subset of Crupi et al. [84] (190 Python and 184 Java tasks), which removes tasks with unreliable test suites. While CoderEval can include file- or project-level context, the natural-language descriptions themselves are typically concise—the contextual code is supplied separately—explaining the relatively low mean prompt length in Table 1.

3.2 Selected Quantization Approaches↩︎

As described in Section 2.1.1, we select six weight-only PTQ approaches—one from each category in our taxonomy: GPTQ [44] (second-order/Hessian-informed), AWQ [47] (activation-aware saliency), QuIP# [51] (rotation/incoherence-based), AQLM [54] (vector quantization), BitsAndBytes [56] (library/kernel-based), and GGUF (format/runtime-based). All models are quantized to 4-bit precision, the most widely adopted operating point for practical LLM deployment [63]. Below, we provide a brief description of each approach and its configuration in our experiments.

GPTQ [44] performs layer-wise weight quantization using approximate second-order (Hessian) information. After each column of the weight matrix is quantized, the remaining weights are adjusted to minimize the overall output reconstruction error. We use 4-bit quantization with a group size of 128.

AWQ [47] identifies salient weight channels by analyzing activation magnitudes and applies per-channel scaling to protect them before quantization. This activation-aware approach preserves the weight channels that contribute most to model accuracy. We apply 4-bit quantization with the default AWQ configuration.

QuIP# [51] uses randomized Hadamard transforms to make the weight matrices incoherent, spreading quantization error uniformly across dimensions, and employs E8 lattice codebooks for efficient encoding.

AQLM [54] employs multi-codebook additive quantization, where groups of weights are jointly encoded through learned codebooks optimized across entire layer blocks. We use the 4-bit configuration with two codebooks.

BitsAndBytes [56] provides efficient low-bit quantization kernels with seamless integration into the Hugging Face ecosystem. We use 4-bit NormalFloat (NF4) quantization with double quantization enabled.

GGUF [85] is a standardized format for distributing quantized model weights, enabling CPU-friendly inference via the llama.cpp ecosystem. We use the Q4_K_M quantization variant, which applies mixed-precision 4-bit quantization with medium-sized k-quant blocks.

For consistency, we utilize pre-quantized model checkpoints available on Hugging Face where possible, and quantize models locally using official implementations when pre-quantized versions are unavailable.

3.3 Static Analysis–Based Evaluation of Code Quality and Quantization Robustness↩︎

Assessing the quality of generated code spans multiple dimensions—syntax validity, coding style, code smells, reliability, maintainability, and security [23], [68]. While various static analysis tools target these attributes, not all are necessary when their outputs converge on consistent findings.

For this study, we adopt SonarCloud [73] as the sole static analysis tool for both Python and Java. SonarCloud identifies bugs, code smells, security vulnerabilities, and code duplications within a single, language-agnostic platform [70]—well suited to our cross-language evaluation.

Rather than supplementing SonarCloud with language-specific tools—such as Pylint [86] and Flake8 [87] for Python or PMD pmd? for Java—we adopt a consolidated strategy. This is grounded in empirical evidence from Afrin et al. [20], who found strong consistency between quality trends reported by SonarCloud and those captured by Pylint, Flake8, and PMD, with no meaningful divergence in conclusions. Given this alignment, additional tools would contribute redundancy without different insights.

To better understand how robustly quantized models maintain their code generation capabilities as task demands increase, we complement the quality assessment with an analysis of quantization robustness across input complexity levels. We characterize each benchmark’s prompts using token length, and Shannon entropy [24]—two complementary, content-agnostic measures motivated by prior code-LLM evaluation work [88] and by NLP literature on entropy as a measure of textual information density [89]—and examine whether quantization-induced correctness degradation, measured as the pass@1 difference between full-precision and quantized models—correlates with these complexity measures.

4 Implementation and Evaluation↩︎

4.1 Experimental Setup and Environment↩︎

For each model family (Qwen2.5-Coder-7B-Instruct and CodeLlama-7B-Instruct), we prepare seven configurations: one full-precision (FP16) baseline and six 4-bit quantized variants (GPTQ, AWQ, QuIP#, AQLM, BitsAndBytes, GGUF). Each configuration generates solutions for all benchmark tasks across the three RQs—McEval and CoderEval are used in both Python and Java for RQ\(_1\) and RQ\(_2\) and in Python for RQ\(_3\); BigCodeBench is used in Python only and contributes to all three RQs. Generated code is evaluated for functional correctness (pass@1 [90]) within a sandboxed Docker environment and for code quality via SonarCloud metrics. We set temperature to 0 for deterministic outputs and cap input tokens at 1024, consistent with related work [91], [92]. All experiments ran on Ubuntu 22.04.5 LTS with four NVIDIA L40S GPUs (48GB each).

4.2 Model Quantization↩︎

For AWQ, GPTQ, GGUF, and BitsAndBytes, we used pre-quantized checkpoints from Hugging Face. For AQLM and QuIP#, we performed quantization using official pipelines with WikiText-2 [93] as the calibration corpus, a standard choice for PTQ [44], [47], [54]. QuIP# quantization used the QuIP-for-all framework [94] to support Qwen-based architectures. All models were loaded via Hugging Face transformers to ensure a standardized inference framework.

4.3 Evaluation Metrics↩︎

Functional correctness. We use pass@1 [90], which evaluates whether the model’s top-ranked output passes all unit testsa strict single-attempt measure reflecting realistic deployment.

Code quality. We apply SonarCloud [73] to assess five quality indicators. Reliability quantifies code robustness by measuring bug density. Maintainability captures code smells, suboptimal patterns that increase technical debt. Lines of Code (LoC) counts non-whitespace lines as a proxy for implementation effort. Cyclomatic Complexity (CyC) measures structural complexity via the control flow graph (\(M = E + 2Q - N\), where \(E\), \(N\), and \(Q\) denote edges, nodes, and connected components), with higher values indicating more branching paths. Cognitive Complexity (CoC) captures human-perceived difficulty by accounting for nested control flow and conditional depth [95].

4.4 Input Complexity Analysis (RQ\(_3\))↩︎

To investigate whether prompt complexity modulates quantization-induced degradation, we analyze all Python tasks across McEval, CoderEval, and BigCodeBench. For each task, we compute token length (word count) and Shannon entropy [24] at the word level, capturing input size and lexical diversity respectively.

We pool all tasks and partition them into High and Low entropy buckets at the median, retaining benchmark identity within each bucket. Table 1 summarizes the distribution: the High bucket is dominated by McEval and BigCodeBench (mean entropy \(\approx\)​6.1), while the Low bucket is dominated by CoderEval (mean entropy \(\approx\)​3.9). This partitioning captures benchmark-level complexity differences while enabling controlled within-bucket comparisons.

We employ three complementary analyses: (1) McNemar’s test [96] on 2\(\times\)​2 contingency tables of discordant pairs to test whether quantization significantly alters per-task correctness; (2) stratified analysis of pass@1 and degradation rates within each entropy bucket across the three benchmarks [97]; and (3) point-biserial correlations [98] between prompt complexity (entropy, length) and degradation outcome, supplemented by Mann–Whitney U tests [99] comparing complexity distributions of degraded vs.non-degraded tasks.

4.5 Analysis and Comparison Framework↩︎

We collect generated solutions from both Qwen2.5-Coder-7B-Instruct and CodeLlama-7B-Instruct across all McEval and CoderEval tasks in Python and Java (RQ\(_1\), RQ\(_2\)), and across BigCodeBench tasks in Python (RQ\(_3\)). Each model is evaluated under seven configurations: one full-precision baseline and six 4-bit quantized variants. For every configuration, we capture the generated code and evaluate it along both dimensions: functional correctness via pass@1 and code quality via SonarCloud metrics.

To determine whether quantization introduces statistically significant changes, we conduct pairwise comparisons between each quantized variant and its full-precision counterpart. For pass@1 (binary), we apply McNemar’s test [96], which evaluates whether the pattern of per-task pass/fail outcomes shifts significantly based on the 2\(\times\)​2 contingency table of discordant pairs. For SonarCloud quality metrics (continuous), we apply the Wilcoxon signed-rank test [100], which assesses whether per-task quality score distributions differ significantly between full-precision and quantized variants. All p-values are adjusted via Holm–Bonferroni correction [101] for six pairwise comparisons. Effect sizes are quantified using Cliff’s delta [102], categorized as negligible (N), small (S), medium (M), or large (L). To account for output variability, we generated ten predictions per instance across all seven configurations of Qwen2.5-Coder-7B-Instruct on McEval-Python. Friedman’s test [103] found no significant run-level differences in pass@1 or any SonarCloud metric, indicating that the reported differences reflect technique effects rather than inference noise.

5 Results and Discussion↩︎

4pt

We present and discuss the results of our empirical study, organized by research question.

RQ\(_1\): How do different quantization techniques impact the functional correctness of code generated by large code models?

Table 2 presents the pass@1 scores for both model families on McEval-Java and CoderEval-Java. On McEval-Java, CodeLlama-7B (FP: 0.25) exhibits wide variation across techniques: GPTQ and AQLM both reach 0.34, while QuIP# drops to 0.08—a substantial reduction indicating that this rotation-based approach is particularly detrimental for CodeLlama on this benchmark. AWQ (0.23) and GGUF (0.23) show marginal degradation, while BitsAndBytes preserves the baseline exactly (0.25). For Qwen2.5-Coder-7B (FP: 0.45), the pattern differs considerably: BitsAndBytes achieves the highest pass@1 at 0.60—a gain of 15 percentage points—followed by AQLM (0.58) and GGUF (0.51). AWQ matches the baseline (0.45), while GPTQ (0.32) and QuIP# (0.32) exhibit meaningful losses.

On CoderEval-Java—which features pragmatic code generation tasks with real-world context dependencies—the quantized variants cluster more tightly around their baselines. For CodeLlama-7B (FP: 0.31), AQLM (0.33) slightly exceeds FP, while the remaining techniques produce pass@1 scores between 0.28 and 0.30. Even QuIP# (0.28), which suffered substantially on McEval, shows only modest decline—suggesting its degradation pattern is benchmark-sensitive rather than uniform. For Qwen2.5-Coder-7B (FP: 0.20), AQLM leads at 0.27, followed by BitsAndBytes (0.26) and GGUF (0.24).

Two cross-cutting observations emerge. First, the impact of quantization is highly technique-dependent: within the same model and benchmark, pass@1 can range from substantial degradation (QuIP# on CodeLlama-McEval) to notable improvement (BitsAndBytes on Qwen-McEval). Second, the effect is model-dependent: techniques that degrade one model may benefit another. For instance, GPTQ improves CodeLlama’s McEval pass@1 by 9 percentage points but reduces Qwen’s by 13.

McNemar’s test (detailed statistical tables are available in our online appendix [25]) confirms that only one Java comparison reaches significance: CodeLlama FP vs.QuIP# on McEval (\(p < 0.05\), OR = 19.00). All other pairwise comparisons are non-significant, indicating that despite considerable variation in aggregate scores, task-level correctness patterns are largely preserved under 4-bit quantization.

Table 3 presents the Python results across BCB-Python, McEval-Python, and CoderEval-Python. The patterns largely mirror Java. On McEval-Python, BitsAndBytes achieves the highest pass@1 for CodeLlama (0.19 vs.FP 0.17), while GGUF and AQLM both reach 0.43 for Qwen (vs.FP 0.31). On CoderEval-Python, AQLM leads for both models (CodeLlama: 0.26; Qwen: 0.31). The BCB-Python benchmark reveals the starkest contrasts: AQLM achieves the best CodeLlama pass@1 (0.27 vs.FP 0.23), while QuIP# drops to 0.12—roughly halving the baseline. Qwen’s quantized variants preserve correctness more consistently, with AWQ, GPTQ, and AQLM all matching FP (0.41); QuIP# again shows the largest drop (0.32).

The Python statistical analysis reveals more significant findings than Java, concentrated on BCB-Python: CodeLlama QuIP# (\(p < 0.05\), OR = 5.10) and BitsAndBytes (\(p < 0.05\), OR = 2.05) significantly degrade correctness, while AQLM (\(p < 0.05\), OR = 0.62) significantly improves it. For Qwen, only QuIP# reaches significance (\(p < 0.05\), OR = 2.71). McEval-Python and CoderEval-Python comparisons remain non-significant. Across both languages, AQLM is the most consistently competitive variant, frequently matching or exceeding FP, while QuIP# exhibits the widest performance variance. Statistical significance is concentrated on BCB-Python, suggesting that quantization-induced correctness shifts become more detectable as task complexity increases—a pattern we examine in RQ\(_3\).

RQ\(_2\): How do different quantization techniques impact the quality attributes of automatically generated code?

6pt

Tables 2 and 3 report SonarCloud metrics—LoC, Security, Reliability, Maintainability, Cyclomatic Complexity (CyC), and Cognitive Complexity (CoC)—for all configurations.

Beginning with McEval-Java, we observe a mixed pattern for CodeLlama-7B. AWQ and QuIP# reduce LoC relative to the baseline (1,261 and 1,217 vs.,301), while AQLM produces the longest code (1,418). Security remains unchanged across all techniques (0 hotspots), and Reliability is stable at 9 for all variants except QuIP#, which introduces one additional issue. Maintainability shows notable variation: AWQ increases code smells substantially (328 vs. in FP), while QuIP# reduces them (250). For Qwen2.5-Coder-7B, GPTQ stands out as the most quality-preserving technique, improving LoC (1,222 vs.,325), CyC (215 vs.), and CoC (164 vs.). In contrast, AWQ again inflates Maintainability (328 vs.)—a pattern consistent across both model families.

On CoderEval-Java, most CodeLlama variants produce fewer quality issues than FP—AWQ, GGUF, and BitsAndBytes all reduce LoC, Maintainability, and CyC. The exceptions are AQLM and QuIP#, which generate longer code (1,833 and 2,009 vs.,810) with higher complexity. For Qwen, QuIP# produces the most compact code across all metrics (LoC: 1,049, Maintainability: 194, CyC: 302, CoC: 225), while BitsAndBytes and AQLM tend to increase complexity. Directional improvements from quantized models likely reflect stochastic variation rather than systematic quality gains.

The Wilcoxon signed-rank test (see online appendix [25]) confirms that most quality differences are non-significant. The clearest finding is AWQ’s effect on Maintainability: both CodeLlama (\(d = -0.346\), medium) and Qwen (\(d = -0.394\), medium) show significant increases in code smells on McEval-Java, suggesting AWQ’s activation-aware scaling systematically alters patterns flagged by static analysis. On CoderEval-Java, AWQ shows significant LoC, CyC, and CoC differences for CodeLlama, though all with negligible effect sizes. Across all Java comparisons, Security and Reliability remain unchanged, confirming that quantization does not introduce security vulnerabilities or reliability regressions.

Turning to Python, the quality landscape is more differentiated. On McEval-Python, CodeLlama exhibits a pattern reminiscent of Java: most variants reduce LoC relative to FP, Security remains zero, and directional changes in Maintainability are mixed. For Qwen, BitsAndBytes substantially increases complexity (CyC: 223 vs.; CoC: 224 vs.), while GPTQ reduces both (CyC: 134; CoC: 80). On CoderEval-Python, a notable asymmetry emerges: CodeLlama’s quantized variants generally improve quality relative to FP, while all six Qwen variants substantially increase LoC (1,367–1,618 vs.), CyC, and CoC—a consistent quality degradation unique to this model–benchmark–language combination. On BCB-Python, QuIP# inflates CodeLlama’s Reliability (43 vs.) and Maintainability (886 vs.) despite reducing LoC; BitsAndBytes shows the largest divergence for Qwen across LoC (17,909 vs.,281), Reliability (30 vs.), and complexity.

The Python statistical analysis reveals substantially more significant findings. On CoderEval-Python, all six Qwen variants show significant LoC differences (small to medium effects), with most also showing significant CyC and CoC shifts—confirming that this quality degradation is statistically robust. On McEval-Python, BitsAndBytes produces the only large effect size in our entire study: CyC (\(d = -0.452\), medium) and CoC (\(d = -0.487\), large) for Qwen. On BCB-Python, the large sample size enables detection of many significant but overwhelmingly negligible effects. Security remains zero across all Python configurations. Synthesizing across languages, Security is universally unaffected, Maintainability is the most sensitive dimension (AWQ in Java, BitsAndBytes in Python), and Java quality differences are largely non-significant while Python—especially CoderEval and BCB for Qwen—reveals statistically robust quality shifts.

4pt

RQ\(_3\): Does input prompt complexity influence quantization-induced correctness degradation?

We pool all Python tasks from McEval, CoderEval, and BigCodeBench, split them at the median Shannon entropy into High and Low complexity buckets (Table 1), and measure the point-biserial correlation between complexity (entropy and token length) and the binary degradation outcome—i.e., whether a task solved by FP was broken by the quantized variant. Table 4 presents per-benchmark degradation rates within each bucket alongside the correlation results.

Across both models, CoderEval tasks—short docstring-style prompts dominating the Low bucket—exhibit near-zero degradation. BigCodeBench concentrates the most severe losses, with QuIP# reaching 12.3–15.3% degradation on CodeLlama. For CodeLlama, the High bucket consistently shows higher degradation on McEval (e.g., AWQ: 6.7% vs.%; AQLM: 16.7% vs.%), while for Qwen this pattern partially reverses—several techniques degrade more on Low-entropy prompts (e.g., AWQ: 16.7% Low vs.% High).

The correlation analysis quantifies this model-level divergence. For CodeLlama, five of six techniques show significant positive correlations: QuIP# has the strongest signal (\(r = +0.325\), \(p < 0.001\) for entropy; \(r = +0.321\), \(p < 0.001\) for length), followed by BitsAndBytes (\(r = +0.203\)) and AWQ (\(r = +0.192\)). GPTQ is the sole exception. For Qwen, only two of twelve correlations reach significance, and one runs in the opposite direction—GPTQ shows a negative correlation (\(r = -0.156\), \(p < 0.001\)), meaning simpler prompts degrade more. All other Qwen techniques show near-zero correlations. One possible explanation is that Qwen, trained on a larger and more diverse corpus, develops more redundant internal representations that are more resilient to precision loss—whereas CodeLlama may rely on less redundant weight configurations for complex reasoning. Since the High-entropy bucket is dominated by BigCodeBench (Table 1), we recompute the correlation within BigCodeBench alone: the model-level divergence persists—five of six CodeLlama techniques retain significance while Qwen remains largely insensitive—confirming the finding is not a benchmark artifact. Full results are in our replication package [25].

The BCB-Python quality metrics (Table 3) add a complementary perspective. For CodeLlama, the techniques that most aggressively quantize—QuIP# and AQLM—not only degrade correctness on complex tasks but also inflate Reliability (43 and 32 vs. in FP) and Maintainability (886 and 883 vs.), despite producing shorter code. Conversely, GGUF and BitsAndBytes reduce both LoC and Maintainability relative to FP, suggesting that their correctness losses are not accompanied by proportional quality degradation. For Qwen, the quantized variants are more tightly clustered around the baseline, with BitsAndBytes showing the largest divergence (LoC: 17,909 vs.,281; Reliability: 30 vs.). These patterns indicate that on complex benchmarks, correctness degradation and quality degradation do not always co-occur—a technique may preserve quality while losing correctness, or vice versa.

5.1 Qualitative Example↩︎

To complement the aggregate statistics, Figure 1 shows Qwen2.5-Coder-7B’s outputs for a single Python task—verifying that a candidate class implements an interface—under all seven configurations (FP + six quantized variants), annotated with the corresponding pass/fail outcome and SonarCloud/Lizard metrics. The example illustrates two patterns observed throughout our results. First, technique-level variation is visible at the source level: the FP baseline produces a structured 18-line routine that explicitly checks whether the candidate implements the interface’s required abstract methods, while the abbreviated comments indicate the same checking pattern for abstract properties and attributes. In contrast, QuIP# collapses the logic into a two-line expression (isinstance(candidate, iface) or (not tentative and not isinstance(candidate, iface))) that bypasses the abstract-method verification logic and is the only variant to fail the test suite—consistent with the QuIP# degradation observed in RQ\(_1\). Second, the remaining quantized variants (AWQ, GPTQ, GGUF, BitsAndBytes, AQLM) produce concise yet correct alternatives, reducing LoC from 18 to 5–6 and Cyclomatic Complexity from 11 to 2–5 while preserving functionality—consistent with the technique-dependent trade-offs reported in RQ\(_2\).

6 Implications of our Findings↩︎

6.1 Implications for Practitioners↩︎

Our findings translate into concrete deployment guidance. Across both model families, 4-bit quantization yields a 63–66% VRAM reduction (14.23 GB \(\rightarrow\) \(\sim\)​5 GB on Qwen2.5-Coder-7B), making local inference feasible on consumer GPUs without sacrificing aggregate correctness. Technique selection should follow the dominant deployment constraint:

• Correctness-first: AQLM is the only technique with a significant pass@1 improvement (BCB-Python, OR = 0.62), at the cost of higher latency.

• Throughput-first: GPTQ leads in throughput (1.65\(\times\) FP16) while preserving correctness.

• CPU-only: GGUF remains the practical choice via the llama.cpp ecosystem.

• Footprint-first: QuIP# achieves the smallest footprint but produces the only significant correctness degradations and the strongest complexity sensitivity (\(r\,\approx\,+0.32\), CodeLlama). Use only when memory is binding and prompts are short.

• Quality-sensitive: AWQ inflates Java code smells (medium effect) and BitsAndBytes produces the largest Python complexity effect (CoC, large)—weigh these against their otherwise strong correctness profiles.

Security attributes remain stable across all configurations.

6.2 Implications for Researchers↩︎

Three findings have broader implications. First, the technique-specific variation we report shows that single-technique evaluations cannot surface the trade-offs that emerge in comparative designs. Second, the model-dependence of complexity sensitivity (CodeLlama: 5/6 techniques significant; Qwen: largely insensitive) suggests representational redundancy from larger, more diverse pretraining corpora may act as implicit robustness to weight compression—a hypothesis warranting investigation. Third, RQ\(_3\) introduces a previously uninvestigated evaluation axis: pass@1 averaged over a benchmark conceals systematic, complexity-correlated degradation that emerges only under stratified analysis.

7 Threats to Validity↩︎

Construct validity. We rely on SonarCloud as the sole static analysis tool, justified by prior evidence of strong consistency with language-specific alternatives [20]. Different tools may capture additional quality dimensions. Our complexity characterization (token length, Shannon entropy) represents only two facets of prompt difficulty; other dimensions such as algorithmic complexity may also influence quantization sensitivity.

Internal validity. We chose two model families (Qwen2.5-Coder-7B, CodeLlama-7B) extensively used in prior quantization research [18], [20], though results may differ for other architectures or scales. For quantization, we used pre-quantized Hugging Face checkpoints where available and quantized locally with WikiText-2 calibration for AQLM and QuIP#—a standard choice [44], [47]; code-specific calibration corpora could yield different results. Temperature was set to 0 for deterministic outputs; results under stochastic sampling may differ. We partially mitigated output variability via a ten-run check on Qwen2.5-Coder-7B-Instruct/McEval-Python; Friedman’s test confirmed no significant run-level differences. For RQ\(_3\), the High-entropy bucket is dominated by BigCodeBench; we mitigate this confound through a within-BigCodeBench replication (Section 5) that preserves the model-level divergence.

Conclusion validity. We employ McNemar’s test (binary correctness) and Wilcoxon signed-rank test (continuous quality metrics) with Holm-Bonferroni correction for multiple comparisons, supplemented by Cliff’s delta effect sizes to distinguish statistical from practical significance.

External validity. Our findings are scoped to 7B models, two languages (Python, Java), three benchmarks, and 4-bit precision. Results may differ at other model scales, bit-widths, languages, or task types. RQ\(_3\) is restricted to Python due to BigCodeBench’s language coverage.

8 Conclusion and Future Work↩︎

We compared six weight-only quantization techniques—GPTQ, AWQ, QuIP#, AQLM, BitsAndBytes, and GGUF—at 4-bit precision on Qwen2.5-Coder-7B and CodeLlama-7B, evaluating functional correctness, code quality, and robustness to input complexity. Our results show that 4-bit quantization largely preserves pass@1, but the choice of technique matters: AQLM consistently matches or exceeds the full-precision baseline, while QuIP# produces the only significant correctness losses. Security is unaffected across all configurations, but AWQ significantly increases maintainability issues in Java (medium effect), and BitsAndBytes introduces the largest quality degradation on Python complexity metrics (large effect). Quantization robustness to prompt complexity is model-dependent: CodeLlama shows significant complexity–degradation correlations for five of six techniques (\(r\) up to \(+0.32\), \(p<0.001\)), while Qwen is largely insensitive. Unlike prior work [20] that assessed AWQ alone, our six-technique sweep reveals technique-specific trade-offs that single-technique studies cannot surface; the practical and research consequences are discussed in Section 6.

Future work. Several directions follow naturally from our results. First, extending the analysis to other precision levels (2–3 bit) and larger model scales would test whether the technique-specific trade-offs reported here persist or invert. Second, the use of code-specific calibration datasets [18] may further mitigate the qualitative degradations we observe—notably AWQ on Java maintainability. Third, the interaction between weight quantization and inference-time optimizations such as speculative decoding, KV-cache quantization, and structured pruning remains an open question. Finally, the model-dependence of complexity sensitivity invites a controlled study isolating the role of pretraining-corpus diversity in shaping quantization robustness. A replication package containing all generated code, evaluation scripts, and detailed efficiency measurements is available at [25].

Acknowledgments↩︎

The authors acknowledge the support of the National Science Foundation, which funded this research under grant NSF CCF-2451058.

References↩︎