1 Introduction↩︎

Recommendation systems have become indispensable across modern online platforms, mitigating information overload by surfacing items aligned with users’ interests[1]–[3]. Over the past decade, deep neural networks built on large embedding tables have dominated the modeling of user–item interactions [4]–[9]. More recently, Large Language Models (LLMs) have emerged as a transformative paradigm for recommendation [10]–[13], driven by their continual scaling gains, broad world knowledge, and contextual understanding. Representative efforts include P5 [14], which unifies diverse recommendation tasks within a single LLM model; OneRec-Think [15], which fuses retrieval and ranking stages via fine-tuned LLMs; and PLUM [16] that adapts pre-trained LLMs for scalable retrieval at YouTube.

Within the multi-stage industrial funnel, the final re-ranking step disproportionately shapes what users see and interact with—most notably in carousel and grid layouts where top positions dominate engagement [17]–[19]. Despite this, LLM-based re-ranking remains underexplored: existing work concentrates on retrieval and early ranking, while industrial re-rankers still rely on point-wise CTR scoring without explicit reasoning over user intent or item semantics. Three concrete gaps block adoption at scale. (G1) Reasoning is left on the table. LLMs are typically deployed zero-shot or supervised-fine-tuned on plain ranking labels, leaving the chain-of-thought capabilities most effectively elicited through reinforcement learning (RL) on verifiable rewards largely unused for re-ranking. (G2) Vocabulary mismatch. Industrial catalogs index billions of items via non-semantic identifiers that lie outside any base-LLM vocabulary, breaking the model’s ability to reason about candidates directly. (G3) Industrial-scale tax. Naive recipes incur prohibitive training cost, low serving throughput from long thinking traces, and—as we show—reward hacking that inflates offline metrics without reflecting genuine ranking quality.

To close these gaps we introduce GR2, the Generative Reasoning Re-Ranker. GR2 is built on a three-stage training pipeline (Figure 1) tailored to large-scale re-ranking. First, a pre-trained student LLM (e.g., Qwen3-8B [20]) is mid-trained on semantic item IDs produced by a tokenizer that achieves \(\geq\)​99% uniqueness, allowing the model to recognize and generalize over the catalog without vocabulary blowup. Second, a stronger teacher (e.g., Qwen3-32B [20]) is prompted with re-ranking-specific templates to generate hierarchical reasoning traces; targeted sampling or rejection sampling yields a curated SFT corpus that grounds the student’s reasoning in user behavior and item semantics. Third, we adapt DAPO [21] with a re-ranking-specific reward composed of a format term, an AUC/NDCG-based verifiable ranking term, and an optional LLM-as-a-judge reasoning term, providing scalable, reward-driven supervision. Bridging this recipe to industry exposes failure modes that is often overlooked. Our contribution is to propose targeted solutions for each: a context compressor that cuts input length by \({>}80\%\) while matching full-context quality, amortizing training cost; On-Policy Distillation (OPD) as a scalable replacement for SFT—naive SFT collapses at industrial scale, whereas a 1.7B OPD student recovers \(\approx\)​82% of a 32B teacher’s gain at 5% of its size; de-hacking conditional reward, without which RL learns to preserve the incoming order or fail to generate valid output format; and reasoning distillation that internalizes chain-of-thought into a non-thinking policy, yielding \({\sim}15\times\) serving ROI at iso-quality.

We summarize our contributions as follows:

• Re-ranking-first LLM design. Distinct from prior work that adapts LLMs for retrieval and early ranking, we establish design principles for LLM-based re-ranking and demonstrate that LLM world knowledge and reasoning translate into measurable ranking gains.

• Semantic-ID mid-training. A tokenization scheme that converts non-semantic item IDs into semantic IDs with \(\geq\)​99% uniqueness, combined with a mixed mid-training corpus of semantic IDs and world knowledge, enabling LLMs to reason over catalog items directly.

• Reasoning activation pipeline. Re-ranking-specific prompts paired with targeted and rejection sampling produce hierarchical reasoning traces; SFT (or OPD at industrial scale) on these curated traces installs a reasoning prior that pure RL cannot recover.

• Verifiable-reward RL for re-ranking. A re-ranking-tailored adaptation of DAPO with a multi-component reward; we identify and mitigate two reward-hacking modes (incoming-order preservation and position-bias exploitation) through conditional verifiable rewards and debiased training data.

• Industrial adaptations. A context compressor, OPD-based scalable distillation, and CoT distillation for non-thinking serving—each ablated to quantify its contribution to deployment feasibility.

• Comprehensive evaluation. GR2 improves over internal baselines on industrial traffic (+18.7% R@1, +9.6% N@3) and shows no decay over subsequent 9 days serving; ablations confirm that OPD supplies the reasoning prior and RL sharpens the ranking objective on top of it.

2 Tokenized Mid-Training↩︎

In this section, we introduce the key components for attaining the LLM-based generative recommender: tokenization and mid-training. To obtain the generative retriever, we stemmed from the mid-training workflow from the advanced LLM-based generative retriever that perform multi-task training on an LLM.

2.1 Tokenizer and Semantic ID (SID)↩︎

Learning the so-called semantic IDs (SIDs) has been widely adopted in recommendation systems to mitigate generalization issues pertaining to large embedding tables. The seminal work, TIGER [22], was the first to apply this technique to sequential recommendation. At a high level, given the textual feature \(x\) of an item, the tokenizer maps it to a sequence of discrete integers, i.e., a compact symbolic representation, defined as \(\text{Tokenizer}(x) = (z_1, z_2, \ldots, z_K)\) where the final representation is a memeber of the set \(\{1,\ldots,C_1\}\times \cdot\cdot\cdot \times \{1,\ldots,C_K\}\), where \(C_i\) is the cardinality of the \(i\)-th codebook. The core of the tokenizer is an (RQ-VAE) [23]. The SID tokens are added to the vocabulary as special tokens. We refer the reader to Sec 2 from our previous technical paper [24] for further details.

2.2 Mid-Training via Multi-Task Learning↩︎

TIGER [22] trains an autoregressive model purely over Semantic ID sequences. In the era of LLMs, leveraging world knowledge encoded in pretrained LMs has become an important direction. To this end, OneRec-Think [15] developed by Kuaishou introduces a mid-training stage. Referred to as the ‘item alignment’, this stage enables the LLM to align the recommendation knowledge and language (Semantic IDs) with the LLM’s linguistic space and world knowledge. More concretely, the idea here is to interleave Semantic IDs (SIDs) with natural language tokens within a single sequence and to optimize the SID embedding table through the next-token prediction objective. The mid-training tasks follows the same design from our previous technical paper [24].

3 Reasoning Enhancement↩︎

We now present the design of reasoning enhancement for re-ranking. In Sec.3.1, we introduce a chat-format training-sample structure that grounds item representation in semantic IDs and enables chain-of-thought reasoning with structured JSON output. In Sec.3.2, we present two complementary strategies for generating high-quality reasoning traces, i.e., targeted sampling, which leverages ground-truth guidance, and rejection sampling, which ensures reasoning authenticity through iterative verification, together with five prompt-design principles that enable grounded, interpretable, and behaviorally-aware reasoning from large language models.

3.1 Chat-formatted Template of Reasoning Data↩︎

We construct training samples in a chat-format structure with three message roles. The design follows six principles:

• Role-based System Prompt. The system message instantiates an analyst persona and specifies the re-ranking objective, activating domain reasoning without invoking purchase-intent or persuasion framing.

• Rich Item Metadata. Each item carries its title and category hierarchy, supporting both semantic and category-aware reasoning.

• Unified Item Format. History and candidates share the same structural format (SID + title + categories), ensuring consistent representation learning across input contexts.

• Chain-of-Thought Reasoning Trace. The assistant response contains step-by-step reasoning that explicitly cites items by their SIDs, producing grounded, verifiable traces.

• Structured JSON Output. The response is a JSON object containing the reasoning explanation and the ranked list, enabling deterministic parsing and evaluation.

3.2 sec:Reasoning32Trace32Generation↩︎

To generate high-quality reasoning trace, we consider two generation strategies for the reasoning trace: targeted sampling and rejection sampling.

3.2.0.1 Targeted Sampling.

In the targeted approach, we provide the target item \(\mathbf{s}_{v_{n+1}}\) and most recent \(k\) historical items, then query a larger LLM to generate a reasoning trace \(\tau\) explaining the target interaction: \[\tau \sim P_\theta(\cdot | \mathcal{P}_{\text{targeted}} ([\mathbf{s}_{v_1},...,\mathbf{s}_{v_k}], [\mathbf{s}_{y_1},...,\mathbf{s}_{y_c}], \mathbf{s}_{v_{n+1}})),\] where \(\mathcal{P}_{\text{targeted}}(x, y, z)\) constructs a prompt to query the rationale for why a user who interacts with item sequence \(x\) would be most interested in \(z\) from candidate list \(y\). As the ground truth is involved, the targeted approach always yields rationales for why the target might be favored by the user. Notice that this requires having ground truth in the prompt.

3.2.0.2 Rejection Sampling.

In sharp contrast to targeted sampling, in rejection sampling we do not provide the ground truth. Instead we repeatedly query the LLM to predict which item \(\mathbf{\hat{s}}_{y_c}\) among the pre-ranked candidates is most likely to be the user’s next interest, until the prediction matches the ground-truth target: \[\begin{align} (\tau, \hat{\mathbf{s}}_{y_c}) \;\sim\;& P_\theta\!\left(\cdot \,\middle|\, \mathcal{P}_{\text{rejection}}\!\bigl([\mathbf{s}_{v_1},\ldots,\mathbf{s}_{v_k}],\,[\mathbf{s}_{y_1},\ldots,\mathbf{s}_{y_c}]\bigr)\right) \\ & \text{s.t.}\quad \hat{\mathbf{s}}_{y_c} = \mathbf{s}_{v_{n+1}}. \end{align}\] where \(\mathcal{P}_{\text{rejection}}(x,y)\) constructs a prompt to query which item in candidate list \(y\) is most likely to be the user’s next interest, given history \(x\).

To generate high quality reasoning trace for re-ranking, we leverage following principles to design the prompt for instructing LLM for reasoning:

• Concrete System Role and Re-ranking Task definition: To incentivize re-ranking capabilities, we elucidate the recommendation domain, the dedicated roles for the LLM to display, and the concrete definition of the re-ranking task.

• Collaborative Context Presentation: To provide comprehensive decision context, we present both the user’s engagement history (with structured metadata including titles and categories) and the complete candidate set, enabling the model to perform holistic comparison rather than isolated item evaluation.

• Domain Knowledge Priming: To leverage sequential engagement patterns inherent in e-commerce (e.g., shampoo \(\rightarrow\) conditioner \(\rightarrow\) styling products), we explicitly prompt the model to consider such domain-specific heuristics, enabling it to apply common-sense reasoning about product complementarity and routine-based purchasing behavior.

• Critical Guidelines as Output Constraint: To ensure reasoning traces are grounded and verifiable, we impose explicit constraints such as requiring the model to cite items by their SIDs. This forces the model to anchor its reasoning in specific historical items rather than generating hallucinated justifications, enabling direct traceability between reasoning steps and context.

• Structured Multi-Step Reasoning Format: To elicit progressive and interpretable reasoning, we provide an explicit step-by-step output format with examples. This hierarchical structure guides the model through: (1) broad pattern recognition, (2) identifying complementary product types, and (3) matching to specific candidates, mirroring human decision-making processes.

3.3 Supervised Fine-tuning on Reasoning Traces↩︎

3.3.0.1 Problem Setup and Prompt Interface

Given a user engagement history and a pre-ranked list of candidate items produced by a previous stage, our desire is to re-rank the candidates so as to promote the ground-truth next item. Each training or inference instance is formatted as a chat prompt consisting of three roles: system, user, and assistant. During supervised fine-tuning (SFT), the model is trained to generate the full assistant message. During reinforcement learning (RL), the policy conditions only on the system and user messages, and the assistant output is treated as the action.

Recommender models often struggle to generate effective CoT reasoning from noisy and lengthy real-world user behavior sequences [15]. The generated reasoning traces are used to supervise a base LLM to acquire reasoning capability for re-ranking. We fine-tune the model to generate both the reasoning trace \(\tau = [r_1, \ldots, r_M]\) and the ranked output \(\mathbf{o} = [o_1, \ldots, o_T]\).

To preserve ranking performance while enabling reasoning, we decouple the losses for reasoning and ranking tokens. Specifically, we apply the language modeling loss only to the assistant message, with separate weights for reasoning and ranking segments: \[\mathcal{L}_{\text{SFT}} = - \lambda_r \sum_{i=1}^{M} \log P(r_i \mid \mathcal{P}, r_{<i}) - \lambda_o \sum_{j=1}^{T} \log P(o_j \mid \mathcal{P}, \tau, o_{<j}),\] where \(\lambda_r < \lambda_o\) balances reasoning fluency and ranking accuracy. This training procedure teaches the model to generate grounded, coherent reasoning traces that connect user history with candidate comparisons, while maintaining strong re-ranking behavior.

3.4 OPD for Reasoning Activation↩︎

We use on-policy distillation (OPD) [25] to transfer reasoning from a frozen teacher into the deployed student. A GRPO-style loop treats the student as the trainable actor and the teacher as the reference policy: at every step the student samples reasoning chains and ranked lists from its own distribution \(\pi_\theta\), each rollout receives the reward of Section 4, and the actor is updated with a clipped surrogate plus a per-token reverse-KL anchor to the teacher, \[\begin{align} \mathcal{L}_{\mathrm{OPD}}(\theta) ={}& -\mathbb{E}\!\left[\min\!\bigl(\rho_t \hat{A}_t,\;\mathrm{clip}(\rho_t,1{-}\epsilon_{\mathrm{lo}},1{+}\epsilon_{\mathrm{hi}})\hat{A}_t\bigr)\right] \\ & {}+ \beta\,\mathbb{KL}\!\bigl[\pi_{\theta}(\cdot\!\mid\! s_t)\,\|\,\pi_{\mathrm{T}}(\cdot\!\mid\! s_t)\bigr], \end{align} \label{eq:opd}\tag{1}\] where \(\rho_t\) is the per-token importance ratio, \(\hat{A}_t\) the group-relative advantage, and \(\beta\) the distillation strength. The teacher contributes only token log-probabilities, never labels, acting as a distributional anchor while gradients flow through the student.

3.4.0.1 Why on-policy beats off-policy CoT supervision.

The two CoT-SFT baselines introduced earlier in Section 3.3 both reduce reasoning transfer to behavior cloning on a static, teacher-defined corpus, and inherit the standard pathologies of teacher-forced sequence learning: the student is supervised on states it will never visit at deployment, producing a train/inference distribution mismatch that compounds along the generated chain. Each method also adds its own bias on top of this. Targeted sampling conditions the teacher on the answer, so the resulting traces tend to encode post-hoc, label-aware shortcuts that the student memorizes as fluent but non-causal reasoning. Rejection sampling, by contrast, silently discards every prompt on which the teacher cannot meet the AUC bar within the sampling budget, which both wastes teacher compute and removes precisely the hardest examples—where supervision is most needed—from the training set. OPD avoids both failure modes by construction: the gradient is computed on trajectories the student itself samples, every prompt contributes a continuous reward signal regardless of how easy or hard it is for the teacher, and the teacher’s role is reduced from generating the target trace to regularizing the student’s own distribution. This combination of on-policy exposure, dense per-rollout reward, and per-token KL anchoring is what allows the small student to absorb the teacher’s reasoning competence without inheriting its trace-style mismatch.

4 RL Post-Training↩︎

While SFT enables coherent reasoning, it does not directly optimize the re-ranking objective. Building on the distilled reasoning capability of Sec. 3.3, we apply RL to refine both the reasoning process and the final ranking quality. Given a prompt \(\mathcal{P}(\mathcal{H},\mathcal{D})\), the policy \(\pi_\theta\) generates an output \(o=(\tau,\mathbf{o})\); during RL the policy conditions only on the system and user messages and \(o\) is treated as the action. Unlike public benchmarks, where each impression has a single positive next item, industrial slates are multi-positive: a candidate list may carry several engagement labels at once. We therefore replace the single-target rank-delta of prior work with reward functions that score the whole permutation against a multi-label ground truth.

4.1 Ranking Reward↩︎

A candidate slate \(\mathcal{D}=\{c_1,\ldots,c_K\}\) is associated with a binary engagement label vector \(\mathbf{y}\in\{0,1\}^K\) that may contain multiple ones. We adopt the per-impression AUC of the predicted permutation \(\pi\) against \(\mathbf{y}\) as the primary ranking signal: \[R_{\text{AUC}}(\pi,\mathbf{y}) \;=\; \frac{1}{|\mathcal{M}||\mathcal{N}|}\sum_{i\in\mathcal{M}}\sum_{j\in\mathcal{N}}\mathbf{1}\!\left[\mathrm{rank}_\pi(i)<\mathrm{rank}_\pi(j)\right],\] where \(\mathcal{M}=\{i|y_i{=}1\}\) and \(\mathcal{N}=\{j|y_j{=}0\}\). AUC handles multiple positives natively, is bounded in \([0,1]\), and is permutation-invariant within each label class, providing a stable RL signal.

When richer post-engagement signals are available, we further incorporate a graded-relevance NDCG term using a three-level label \(g_i\in\{0,1,2\}\) for \(\{\text{no engagement},\text{ click},\text{ click}+\text{conversion}\}\): \[R_{\text{NDCG}}(\pi,\mathbf{g}) \;=\; \frac{1}{Z}\sum_{i=1}^{K}\frac{2^{g_{\pi^{-1}(i)}}-1}{\log_2(i+1)},\] with \(Z\) the ideal DCG. The combined ranking reward is \(R_{\text{rank}}=R_{\text{AUC}}~\text{or}~R_{\text{NDCG}}\) in our internal-data setup. Slates with no positives or no negatives (\(|\mathcal{P}|=0\) or \(|\mathcal{N}|=0\)) are filtered at data-loading.

4.2 Conditional Format Reward↩︎

To ensure parseable outputs, we add a format reward \(R_{\text{fmt}}=\Omega(o)\) that checks whether (i) the reasoning trace \(\tau\) and ranking output \(\mathbf{o}\) can be reliably parsed, and (ii) the parsed ranking is a valid permutation of \(\{1,\ldots,K\}\) matching the slate size. Naively combining \(R_{\text{rank}}\) and \(R_{\text{fmt}}\) exposes the policy to two reward-hacking pathways. First, an invalid permutation could still earn a non-trivial \(R_{\text{rank}}\) from a partial parse; we therefore gate \(R_{\text{rank}}\) on \(\Omega(o)=1\). Second, when the upstream slate is itself non-trivially ordered—i.e., the identity permutation already attains \(R_{\text{rank}}<1\)—the policy can harvest format reward by emitting \([1,2,\ldots,K]\) without performing any re-ranking, riding on whatever AUC the input order already supplies. We detect this identity-permutation cheating and zero out \(R_{\text{rank}}\) when the model output equals the input order and the input order is suboptimal: \[R = \begin{cases} R_{\text{rank}} + \alpha R_{\text{fmt}}, & \pi\neq[1,\ldots,K]\;\text{or}\;R_{\text{AUC}}([1,\ldots,K],\mathbf{y})=1,\\ \alpha R_{\text{fmt}}, & \pi=[1,\ldots,K]\;\text{and}\;R_{\text{AUC}}([1,\ldots,K],\mathbf{y})<1, \end{cases}\] where \(\alpha\) is the format-reward weight. When the upstream slate is already optimal, the identity permutation is the genuinely correct answer and reward flows normally.

4.3 Training via DAPO↩︎

We optimize the policy using Decoupled Clip and Dynamic sAmpling Policy Optimization (DAPO) algorithm [21], a recently state-of-the-art RL algorithm developed upon Grouped Policy Optimization (GRPO) [26]. It can effectively resolve the entropy collapse phenomenon and rollout length bias identified in the GRPO training process. For each prompt, we sample a group of \(G\) outputs \(\{o_i\}_{i=1}^G\) and optimize: \[\begin{align} \mathcal{J}_{\text{DAPO}}(\theta) = \;&\mathbb{E}_{(q,a)\sim\mathcal{D},\,\{o_i\}_{i=1}^G \sim \pi_{\theta_{\text{old}}}(\cdot|q)} \\ &\Bigg[ \frac{1}{\sum_{i=1}^G |o_i|} \sum_{i=1}^G \sum_{t=1}^{|o_i|} \min\Big( r_{i,t}(\theta)\,\hat{A}_{i,t}, \\ &\qquad \text{clip}\big(r_{i,t}(\theta),\, 1-\varepsilon_{\text{low}},\, 1+\varepsilon_{\text{high}}\big)\,\hat{A}_{i,t} \Big) \Bigg], \end{align} \label{eq:dapo}\tag{2}\] where \[r_{i,t}(\theta) = \frac{\pi_\theta(o_{i,t} | q, o_{i,<t})}{\pi_{\theta_{\text{old}}}(o_{i,t} | q, o_{i,<t})}, \quad \hat{A}_{i,t} = \frac{R_i - \text{mean}(\{R_i\}_{i=1}^G)}{\text{std}(\{R_i\}_{i=1}^G)}. \label{eq:9}\tag{3}\] \(\pi_\theta\) and \(\pi_{\theta_{\text{old}}}\) denote the current and previous policies, respectively. The pair \((q,D)\sim\mathcal{D}\) is sampled from the training distribution. The advantage estimate \(\hat{A}_{i,t}\) is computed for each output sequence \(o_i\) and normalized within the group \(G\). Adhering to the Clip-Higher strategy, DAPO decouples the lower and higher clipping range as \(\varepsilon_{\text{low}}\) and \(\varepsilon_{\text{high}})\). Additionally, to avoid zero policy gradients (advantages) and improve sample efficiency, it oversamples and filters out prompts with the accuracy equal to 1 and 0.

5 Serving ROI Optimization↩︎

The first three stages yield a capable but expensive reasoning ranker: mid-training adapts the model to RecSys inputs and compresses prompts via semantic IDs (§2), reasoning enablement instills a reasoning prior (§3), and RL post-training sharpens ranking behavior under that prior (§4). The final stage preserves these ROI gains while slashing serving cost along its two dominant axes: the input cost of prompting over an \(O(B)\) item catalog, and the decode cost of generating a chain-of-thought (CoT) per request. We address them with two complementary levers—input-side context compression and decode-side reasoning internalization—delivering higher throughput and lower latency without sacrificing ranking quality.

5.1 Input-Side: Context Compression↩︎

Mid-training over an \(O(B)\) item catalog requires generating large volumes of SID-aligned data and incurs heavy GPU cost per iteration, which slows online experimentation. As a faster alternative for cutting input cost, we train a context compressor that shrinks the input before it reaches the ranker.

The compressor is trained with GRPO using an LLM-as-a-judge that scores each compressed output along three axes: solvability \(s\!\in\!\{0,1\}\), information preservation \(p\!\in\!\{1,\ldots,10\}\), and ranking quality \(q\!\in\!\{1,\ldots,10\}\). The judge prompt is mode-aware (item-level vs.free-form summary). The reward combines a continuous compression term with a solvability-conditioned judge term: \[r_{\text{judge}} = \begin{cases} 0.2\,\bar{p} + 0.8\,\bar{q} & s = 1 \\ 0.8\,\bar{p} + 0.2\,\bar{q} & \text{otherwise} \end{cases}\] \[r = \bigl(\alpha\, r_{\text{comp}} + (1{-}\alpha)\, r_{\text{judge}}\bigr) \cdot \lambda_{\text{ellipsis}},\] where \(\bar{p}=p/10\), \(\bar{q}=q/10\), \(r_{\text{comp}}=\max(0,\,1-|\text{compressed}|/|\text{original}|)\), \(\alpha\) trades off compression against quality, and \(\lambda_{\text{ellipsis}}\!\in\![0,1]\) penalizes truncation shortcuts. When the task is solvable we weight ranking signal over raw information completeness, since we find preservation correlates weakly with downstream ranking while higher compression can suppress noise.

5.2 Decode-Side: Internalizing Reasoning for Efficient Serving↩︎

Generating a full CoT per request is the dominant decode cost of GR2. To preserve the reasoning gains under large-scale serving latency, we run a second RL pass on the RL post-training checkpoint (§4) with explicit reasoning bypassed: the policy is trained to emit only the structured ranked output, conditioned directly on the system and user prompts. Because the reasoning prior—instilled via OPD during reasoning enablement (§3) and sharpened by RL (§4)—is already encoded in the shared backbone, it transfers to the direct-output policy without incurring CoT generation cost. This reasoning-free variant is the deployed artifact; the CoT-enabled model is retained offline for evaluation and reward design.

5.3 Systems-Level Optimizations: Pruning and KV Caching↩︎

The above levers reduce token counts; two systems-level techniques further cut the cost per token. Model pruning shrinks the served network via depth-wise (layer) pruning followed by redistillation to recover quality, lowering compute across both prefill and decode. KV caching exploits the structure of RecSys prompts: the candidate set for a request is fixed and largely shared across users, so we precompute its key–value representations once and reuse them across requests, ordering the prompt as system prompt, candidates, then user context so the candidate KV remains reusable. Together these amortize compute over the stable parts of the input and the reduced model, compounding the savings from compression and reasoning internalization.

6 Experiments↩︎

We focus this section on the industrial setting and answer the following research questions:

• Q1: Does GR2 outperform the point-wise lagacy baseline, and is the gain robust to test-set scale, traffic freshness, and student model size?

• Q2: How do different reasoning-activation strategies (SFT, OPD) and RL post-training affect re-ranking quality at industrial scale?

• Q3: Can the enterprise-scale adaptations—context compression, OPD-based distillation, and implicit-CoT serving—preserve ranking quality under industrial latency and GPU budgets?

Public-benchmark results, the semantic-ID tokenizer design, and the codebook/uniqueness study are reported in a prior technical paper [24]; here we concentrate on the industrial setting, where adaptation challenges dominate.

6.1 Experiment Setup↩︎

6.1.1 Datasets↩︎

GR2 is trained on a single day (01-25) of internal logs containing roughly 70k user sessions and evaluated on held-out sessions from 02-01 through 02-09. Each training example corresponds to one impression list paired with the user’s historical feedback (e.g., clicked products). We apply standard industrial filtering to drop duplicated logging events and users with fewer than three interactions in the observation window. To stress-test generalization, the test split is constructed so that user identifiers and interacted products rarely overlap with training: \(>\)​99% of candidate products, 100% of user IDs, and 93% of user-history items are unseen during training. User histories are aggregated up to day \(-2\) relative to the impression to prevent label leakage. Unless noted otherwise, we report relative gains over the point-wise legacy baseline, which is online-trained with snapshot refresh every 60–90 minutes.

6.1.2 Metrics↩︎

We report standard sequential-recommendation metrics, Recall@K and Normalized Discounted Cumulative Gain at K (NDCG@K), computed over held-out impression lists.

6.2 Main Results↩︎

6.2.0.1 GR2 substantially outperforms the legacy baseline (Fig. 2).

On the internal test set, GR2 delivers +18.7% R@1 and +7.1% R@3 over a heavily-tuned legacy baseline, despite being trained on a single day of logs. Since the GR2 checkpoint is roughly two weeks stale relative to the test window while the baseline is refreshed hourly, the lift cannot be attributed to data freshness; it reflects a genuine improvement in ranking quality driven by reasoning over product semantics.

6.2.0.2 The lift is preserved at industrial scale (Fig. 3).

Holding training data fixed at 70k sessions, we sweep test-set size from \(0.14\times\) through \(1.4\times\) up to \(100\times\) training data size. Relative R@K and N@K gains remain essentially constant across all three scales, indicating that GR2’s improvement is not an artifact of small-sample evaluation and transfers to traffic volumes seen in industrial serving.

6.2.0.3 Gains do not decay under model staleness (Fig. 4).

Evaluated daily for nine consecutive days starting 02-01, the relative lift over the legacy baseline shows no measurable decay even though the GR2 checkpoint is more than two weeks stale by the end of the window. This is striking given the cold-start setting described above (Section 6.1.1): nearly every user and product at test time is novel, yet GR2 generalizes without retraining. We attribute this robustness to GR2’s reliance on LLM world knowledge and reasoning rather than memorized sparse IDs, which is precisely what forces the legacy baseline to refresh every 60–90 minutes to avoid distribution drift. We expect this gap to widen on lower-funnel signals (delayed conversions), where sparse-ID models are further disadvantaged by feedback delay and label sparsity.

6.2.0.4 Performance scales with model size (Fig. 5).

Sweeping over Qwen-3 from 1.7B to 32B parameters, we observe R@3 lift grows monotonically with model size, demonstrating that GR2 inherits the LLM scaling law in the ads re-ranking domain. This stands in contrast to sparse-ID baseline models, whose scaling is bounded by ID-vocabulary capacity rather than parameter count. The current curve has not yet flattened despite our \(\sim\)​10-GPU budget, suggesting potential headroom; we discuss how on-policy distillation (Section 3.4) closes the resulting serving-cost gap.

6.3 Serving Latency and Distillation↩︎

6.3.0.1 Serving ROI via On-Policy Distillation (Fig. 6)

Our default GR2 recipe, denoted RL-OPD, first applies OPD to bootstrap reasoning without the catastrophic forgetting we observe with token-level SFT, then runs RL post-training on the OPD checkpoint to sharpen ranking quality while preserving the reasoning prior. A 1.7B student distilled from a 32B teacher (5% of teacher size) recovers 82% of the 32B’s gain over the baseline baseline and delivers 2.6\(\times\) the gain of an 8B model trained without distillation, yielding a \(\sim\)15\(\times\) serving-ROI boost (\(32/1.7 \times 0.82\)) at iso-quality.

6.3.0.2 Context Compression (Fig. 7)

Both RL-OPD variants improve substantially over the zero-shot 0.6B baseline (+10–12% R@1, +4–6% R@5/N@5). The compressed-context variant matches or slightly surpasses the full-context counterpart at R@3/N@3 (+7.71% vs.+7.14%; +8.16% vs.+7.98%) while consuming \(<\)​20% of the tokens—an additional \(>\)​5\(\times\) context reduction at iso-quality. We attribute this to the solvability-conditioned reward: when \(s{=}1\), the compressor is pushed toward ranking quality \(\bar{q}\) rather than literal preservation \(\bar{p}\), learning to discard ranking-irrelevant signal. This confirms our hypothesis that information preservation correlates only weakly with ranking utility, and that aggressive ranking-aware compression acts as a denoiser.

6.3.0.3 Internalizing Reasoning (Fig. 8)

To stress-test whether implicit reasoning can replace explicit CoT, we evaluate on a hard subset—sessions flagged by a large foundation LLM as requiring above-average reasoning, where bypassing CoT should hurt most. The first two points trace the same RL-OPD checkpoint with and without CoT: removing the trace at inference drops Recall and NDCG sharply on both 0.6B and 1.7B students (\(-\)​6–7% R@1/N@1, \(-\)​3–5% R@3/N@3), confirming the CoT carries real ranking signal on this slice. A second RL pass closes the gap entirely and slightly surpasses the CoT reference at the head of the ranking (R@1/N@1: 0.3012 vs.0.2968 for 0.6B; 0.3070 vs.0.3059 for 1.7B), and matches it at deeper cutoffs (0.6B R@5: 0.9256 vs.0.9278; N@5: 0.6174 vs.0.6190, both within \(\sim\)​0.3%). The OPD-pretrained backbone already encodes a robust ranking prior over reasoning traces, so the second RL phase only needs to re-route this prior into the direct-output policy, internalizing CoT through the shared backbone. The deployed artifact is thus reasoning-free at iso- or super-CoT quality even on reasoning-heavy traffic.

6.4 Reasoning Quality and Ablation↩︎

6.4.0.1 Reasoning quality across training recipes (Fig. 9).

Applying RL directly on top of the zero-shot checkpoint slightly degrades reasoning quality across all five rubrics relative to OPD-initialized variants; the degradation is sharpest on Depth (\(\approx\) \(1.02\), near the rubric floor), confirming that RL alone learns to rank well without thinking well. The RL-OPD recipe recovers reasoning quality while preserving the ranking gains, producing traces that are simultaneously shorter and more focused.

6pt

6.4.0.2 Ablation: training recipe trade-offs (Fig. 10).

We compare three recipes side-by-side on both ranking metrics and reasoning quality: (i) RL-only on the zero-shot checkpoint, and (ii) RL-OPD. RL-only matches RL-OPD on ranking metrics but underperforms on every reasoning rubric; RL-OPD Pareto-dominates both, supporting our headline claim that OPD supplies the reasoning prior while RL sharpens the ranking objective on top of it.

6.4.0.3 Case study (Table 1).

GR2 grounds its ranking in product attributes (category, material, function) and the user’s revealed click patterns—semantics the sparse-ID baseline cannot represent. From a history dominated by leather holsters, pistol cases, and leather outerwear, GR2 infers a coherent “leather goods + firearms accessories” profile and promotes the vintage leather police coat to the top, recovering the click that the baseline ranks fourth. Non-leather, off-theme candidates (jewelry-making hammer, safety razor, Oakley sunglasses) are correctly pushed to the bottom, while the leather boots and leather pants are kept in the middle as plausible but weaker matches.

7 Related Works↩︎

7.0.0.1 Generative LLM RecSys

Recent work increasingly reformulates recommendation as sequence generation. TIGER [22] pioneered the use of Semantic IDs (SIDs), discretizing item representations via RQ-VAE to enable autoregressive recommendation without large embedding tables. OneRec [27] cast multiple recommendation tasks into a unified text-to-text LLM framework, and OneRec-Think [15] extended it with explicit reasoning traces, showing that chain-of-thought intermediate steps improve accuracy and robustness. OpenOneRec [28] contributes a reproducible research framework for this family. PLUM [16] similarly uses continued pre-training to align LLMs with the recommendation domain and adds a task-specific fine-tuning objective for generative retrieval. GR2 inherits the SID and item-alignment ideas but differs in three respects: (1) a tokenizer that achieves substantially higher SID uniqueness, (2) high-quality reasoning traces produced via targeted prompting and rejection sampling, and (3) a DAPO-based reward optimization.

7.0.0.2 Reasoning LLM

Chain-of-thought (CoT) prompting elicits multi-step reasoning by generating intermediate steps before final answers [29], and subsequent work has formalized and extended reasoning structures beyond CoT [30], [31]. RL methods such as GRPO and its variants further improve reasoning faithfulness and coherence via verifiable rewards [21], [26], [32], [33], providing the broader context for our optimization strategy. A close work is LLM-based document re-ranking. ReaRank reasons before re-ranking passage lists via RL [34]; Rank-R1 applies RL-based reasoning optimization to document ranking [35]; ReasonRank and MM-R5 extend reasoning-augmented reranking to multi-view and multi-modal settings [36]; \(R^4ec\) [37] adds iterative reflection and refinement; and [38] use a candidate-randomization bootstrap to improve fairness in LLM-based reranking. GR2 differs in its focus on semantic IDs and reward functions tailored to recommendation re-ranking, combined with structured reasoning traces and rejection sampling for higher-quality supervision.

8 Conclusion↩︎

We presented GR2, a Generative Reasoning Re-Ranker that brings LLM world knowledge and chain-of-thought reasoning to the final re-ranking stage of large-scale recommendation systems. GR2 combines mid-training on SIDs, reasoning-trace distilled from a stronger teacher, and RL with verifiable rewards. To make the recipe industry-viable, we further introduced a context compressor that cuts input length by \(>\)​80% at iso-quality, OPD as a scalable alternative to SFT, and reasoning-internalizing CoT distillation that delivers a \(\sim\)​15\(\times\) serving-ROI gain. Across industrial-scale traffic, GR2 improves over internal baselines (+18.7% R@1, +7.1% R@3; +9.6% N@3), with gains robust to test-set scale, two-week traffic staleness, and increasing model size. Ablations and LLM-as-judge analysis confirm that OPD supplies the reasoning prior while RL sharpens the ranking objective on top of it—neither is sufficient alone.

References↩︎