1 Introduction↩︎

Reliable reward signals are central to aligning large language models (LLMs) with human preferences, motivating reward models that handle implicit or noisy feedback and expose predictive uncertainty [1]–[3]. Yet beyond learning reliably from available feedback, open-ended tasks require specifying which dimensions of response quality should be rewarded. Rubrics provide a structured interface for this purpose: a rubric set breaks response quality into explicit checks, making judgments easier to inspect and reward signals more fine-grained. Recent work on long-horizon agents underscores the growing need for transparent evaluation of complex, multi-step outputs [4], [5]. In evaluation, generated rubric checklists can improve agreement between LLM judges and human preferences [6]. In post-training, rubric-based rewards have been used for on-policy reinforcement learning in domains without automatically verifiable outcomes [7]. As rubrics become useful for both evaluation and learning, a key bottleneck moves upstream: where do reliable, query-specific rubrics come from?

Many existing approaches rely on annotated or external data, either to train rubric generators or to infer and validate rubrics from preference pairs [8]–[11]. Such supervision can improve discrimination, but collecting preference labels, response pairs, or human-written rubrics is costly and ties generation to a dataset. To reduce this cost, annotation-free methods generate rubrics directly from a query through one-pass prompting, recursive expansion, or multiple evaluator roles [6], [12], [13]. We study this strict query-only setting: given only \(q\), evolve \(\mathcal{R}(q)\) from an empty set. The generator never sees benchmark responses, preference labels, reference answers, or reference rubrics and receives no model training.

Removing external supervision creates a rubric-quality gap: a rubric may sound sensible without providing a useful quality signal. A useful rubric should distinguish materially better responses from worse ones without rewarding optional surface choices or excluding valid alternative strategies. However, direct generation does not explicitly test whether each proposed rubric has these properties, and may therefore produce non-discriminative, style-only, or over-specific rubrics [13], [14], which may cause reward hacking in rubric-based reinforcement learning [15].

Closing this gap requires validating each candidate before it enters the rubric set. At evolution step \(t\), the underlying decision is whether a proposed update \(\mathcal{R}'_t\) improves on the current set \(\mathcal{R}_t\). Yet we find empirically that LLMs cannot reliably make this judgment by reading the two abstract rubric sets. We instead move the comparison into response space. For each candidate rubric \(r\), we construct pairs that preserve all unchanged rubrics but differ in whether \(r\) is satisfied, and ask which response is better. This turns abstract rubric-set comparison into pairwise response comparison, a task that LLM judges handle more reliably [16], [17].

Rubrics on Trial uses two complementary pairs. The local pair begins with a strong response \(a^+\) satisfying the background rubrics together with \(r\), then makes the smallest coherent edit \(a^-\) that violates \(r\). This tests whether violating \(r\) makes an otherwise unchanged response worse. But a local edit can overstate the value of \(r\). If \(a^+\) is organized around one method, heading, or style, removing it may make \(a^-\) awkward even when \(q\) does not require that choice. The alternative pair is instead written independently from scratch. It constructs the strongest response \(b^-\) that satisfies the background rubrics while violating \(r\), then minimally repairs it into \(b^+\). If \(b^-\) remains good under another solution strategy, \(r\) is over-specific. If it remains good without a prescribed surface form, \(r\) is optional or style-only. A rubric-blind judge compares the two responses within each pair, with both pairs presented in both orders. A fixed rule adds \(r\) only when both comparisons favor the responses satisfying it.

Starting from an empty rubric set, Rubrics on Trial applies this test one patch at a time in a tree-structured evolution process. A proposer either Adds one atomic rubric or Splits a bundled rubric. Accepted rubrics update the current active leaves; rejected proposals leave them unchanged. Every decision and its reason enter evolution memory to guide the next proposal. The final rubric set is derived only from \(q\) and synthetic response comparisons.

The main contributions of this paper can be summarized as follows: (1) Query-only rubric evolution. We evolve a rubric set from scratch, using comparisons of rubric-conditioned response pairs as supervision. (2) A complementary two-pair gate. The local pair tests quality discrimination, while the from-scratch pair rejects over-specific and optional/style-only rubrics. (3) Empirical effectiveness. Across five preference benchmark suites, we demonstrate that Rubrics on Trial achieves the strongest overall preference-judgment performance among query-only and trained rubric generators.

2 Preliminaries and Related Work↩︎

2.1 Rubric-based evaluation↩︎

Let \(q\) be a query, \(y\) a response, and \(r\) a binary rubric that describes one checkable property of a good response. A verifier returns \(v(q,y,r)\in\{0,1\}\), indicating whether \(y\) passes \(r\). A query-specific rubric set \(\mathcal{R}(q)=\{r_1,\ldots,r_K\}\) therefore maps a response to a vector of interpretable judgments. An evaluator may aggregate this vector to compare responses, while a learning algorithm may reuse the individual judgments as fine-grained rewards. Our method generates the rubric set itself and does not require a particular weighting rule. In our experiments, unless stated otherwise, we use the uniform score \[s_{\mathcal{R}}(q,y)=\sum_{r\in\mathcal{R}(q)}v(q,y,r), \label{eq:rubric95score}\tag{1}\] so that a pair is predicted by comparing its two scores.

2.2 Existing rubric generation↩︎

Existing methods obtain rubric-quality signals in two broad ways. Externally supervised methods use annotated rubrics or externally provided response preferences. OpenRubrics constructs rubrics from contrasts between preferred and rejected responses and trains a prompt-conditioned rubric generator [8]. CDRRM likewise profiles preference pairs before synthesizing discriminative rubrics [9]. Auto-Rubric uses preference pairs in a propose–evaluate–revise pipeline, while query-specific rubric learning for DeepResearch trains on human-labeled report pairs [10], [11]. These methods show that behavioral supervision can improve rubric quality, but require external response data, and several additionally require training a dedicated generator.

Query-only methods instead construct a rubric for each query without training a task-specific generator or receiving external preference data. TICK produces instruction-specific yes/no checklists [6], while the checklist-generation stage of RocketEval creates an instance-specific grading checklist [18]. RRD samples responses conditioned on the query, then recursively decomposes broad rubrics and filters misaligned or redundant criteria [14]. Qworld recursively expands a question into scenarios, perspectives, and binary criteria [12], and Multi-Role Rubric Generation (MRRG) aggregates criteria proposed by multiple evaluator roles [13]. These methods reduce annotation cost and improve rubric coverage or structure. However, the selection of an individual rubric is still mainly determined by generation, decomposition, or refinement prompts. A plausible rubric can therefore enter the final set even when it adds little discrimination, rewards an optional surface form, or rules out a valid alternative strategy. We target this candidate-level validation gap.

2.3 Pairwise comparison as rubric supervision↩︎

Comparing two responses gives an LLM judge a direct behavioral contrast and has been shown effective for zero-shot generation evaluation and text ranking [16], [17]. Rubric-generation methods also exploit this signal. OpenRubrics, CDRRM, and Auto-Rubric derive criteria from preference-labeled response pairs [8]–[10]. OnlineRubrics elicits rubrics from comparisons between current-policy and reference-policy responses [19], while EvoLM co-evolves rubrics and a policy using contrasts between responses from different policy checkpoints [20]. Together, these works establish pairwise response comparison as a useful source of rubric supervision, but obtain the compared responses from a dataset or a policy trajectory.

Our framework creates that supervision from the query itself. For each proposed rubric, it synthesizes two rubric-conditioned response pairs and asks a rubric-blind judge which response is better within each pair. This replaces difficult direct comparison between rubric sets with targeted response comparisons. The resulting pairwise acceptance gate screens every proposed Add or Split before the current rubric set is updated, without requiring external response pairs or preference labels.

2.4 Problem definition: query-only rubric evolution↩︎

We study query-only rubric evolution: given only a query \(q\), the goal is to construct a query-specific rubric set \(\mathcal{R}(q)\) from an empty set, without external supervision or task-specific model training. The framework has no access to benchmark responses, preference labels, reference answers, or externally written rubrics. We formulate this task as a tree-structured evolution problem, rather than generating the complete rubric set in one pass. At evolution step \(t\), the tree \(\mathcal{T}_t\) records the lineage of accepted rubric additions and refinements, \(\mathcal{H}_t\) stores feedback from earlier proposals, and its active-leaf frontier defines the current rubric set \[\mathcal{R}_t=\operatorname{Leaves}(\mathcal{T}_t). \label{eq:active95frontier}\tag{2}\] Here \(\operatorname{Leaves}(\cdot)\) omits the virtual root. Given \((q,\mathcal{R}_t,\mathcal{H}_t)\), a proposer emits one patch \(p_t\). An Add\((r)\) patch proposes attaching one new atomic rubric, while a Split\((r_p\!\rightarrow\!\{r_1,\ldots,r_m\})\) patch proposes retiring a bundled parent and replacing it with atomic children. An accepted patch produces \(\mathcal{T}_{t+1}\) and its new frontier \(\mathcal{R}_{t+1}\); a rejected patch leaves the tree unchanged. Thus, the tree preserves how broad rubrics are progressively introduced and refined, while its leaves always represent the rubric set used for evaluation. Every outcome is appended to memory to produce \(\mathcal{H}_{t+1}\). Initially, \(\mathcal{T}_0\) contains only the virtual root and \(\mathcal{R}_0=\mathcal{H}_0=\emptyset\). The output is the final active-leaf set \(\mathcal{R}_T\).

3 Methodology↩︎

3.1 Framework overview↩︎

Rubrics on Trial is a role-specialized multi-agent framework with three LLM roles: a rubric proposer, response generators, and a rubric-blind pairwise judge. A deterministic controller maintains \(\mathcal{T}_t\), \(\mathcal{R}_t\), and \(\mathcal{H}_t\). At step \(t\), the proposer reads \((q,\mathcal{R}_t,\mathcal{H}_t)\) and emits a patch \(p_t\). An Add patch contains one candidate rubric \(r\); a Split patch contains multiple child rubrics, each of which is evaluated separately as a candidate rubric.

The central decision is whether a candidate rubric should enter the active rubric set. Directly comparing \(\mathcal{R}_t\) with its hypothetical update asks an LLM to judge two abstract rubric sets. We instead evaluate the candidate rubric through pairwise response comparisons. For each candidate rubric \(r\), the controller first defines the trial background \[\mathcal{B}_t= \begin{cases} \mathcal{R}_t, & \text{for }\mathrm{\small Add}(r),\\ \mathcal{R}_t\setminus\{r_p\}, & \text{for each child of }\mathrm{\small Split}(r_p). \end{cases} \label{eq:trial95background}\tag{3}\] The trial background contains the existing rubrics that must remain satisfied while \(r\) is evaluated, and it remains unchanged across all four constructed responses. To construct the two pairs, response generators are given \(q\), \(\mathcal{B}_t\), and the candidate rubric \(r\). Each generator is instructed to produce a response that either satisfies or violates \(r\), while satisfying every rubric in \(\mathcal{B}_t\). Together, the four responses form the local and alternative-answer pairs. The rubric-blind judge then independently compares the two responses within each pair, with each pair presented in both orders. It sees only \(q\) and the responses—not \(r\), \(\mathcal{B}_t\), or the plus/minus labels. A fixed lookup maps the two pairwise outcomes to an accept/reject decision. The controller updates the tree after an acceptance and records every outcome in \(\mathcal{H}_{t+1}\) before the next proposal. Figure 1 summarizes the framework.

3.2 Rubric-conditioned response pair construction↩︎

At step \(t\), consider a candidate rubric \(r\) and its trial background \(\mathcal{B}_t\). We call the response pairs rubric-conditioned because each response generator sees \(q\), \(\mathcal{B}_t\), \(r\), and whether its assigned response must pass or fail \(r\). Every response must continue to pass all rubrics in \(\mathcal{B}_t\); within each pair, the intended difference is whether the response passes \(r\). Table 1 summarizes the four constructions.

3pt

3.2.0.1 Local edit pair.

We first generate a strong \(a^+\) from scratch that passes \(\mathcal{B}_t\cup\{r\}\). We then minimally edit it into \(a^-\), which fails only the candidate rubric \(r\) while continuing to pass every rubric in \(\mathcal{B}_t\). Comparing \(a^+\) with \(a^-\) tests whether violating \(r\) makes an otherwise unchanged response materially worse. Because \(a^-\) is derived from \(a^+\), however, this comparison may depend on the particular strategy or surface form around which \(a^+\) was written.

3.2.0.2 Alternative-answer pair.

We therefore generate \(b^-\) independently from scratch as the strongest response that passes \(\mathcal{B}_t\) while failing \(r\), and then minimally repair it into \(b^+\). This pair tests whether a response can remain fully correct, compliant, and useful without satisfying the candidate rubric. If \(b^-\) solves the task through another valid strategy and remains as good as \(b^+\), then \(r\) is over-specific. If \(b^-\) is equally good while omitting an unrequested heading, list format, length, or stylistic choice, then \(r\) is optional or style-only. Generating \(b^-\) from scratch allows the model to find a strong alternative that does not rely on \(r\). This prevents a poor edit of \(a^+\) from making \(r\) appear more useful than it is.

3.2.0.3 Construction verification.

Before either pair is judged, a rubric-aware verifier checks that every response satisfies all rubrics in \(\mathcal{B}_t\) and that its assigned pass/fail condition for \(r\) holds. A response that fails any check is regenerated. If no valid construction is obtained within a fixed retry budget, the trial is treated as inconclusive and \(r\) is not admitted at that step. The subsequent pairwise judge remains rubric-blind.

3.3 Rubric-blind pairwise gate↩︎

The judge evaluates the local and alternative pairs independently. For each pair, it receives only \(q\) and the two responses; it does not see the candidate rubric \(r\), the trial background \(\mathcal{B}_t\), the construction instructions, or the plus/minus labels. Each pair is presented in both orders. An order-consistent result is normalized as the plus response being better, the responses being equivalent, or the minus response being better. Let \(J_a\) and \(J_b\) denote the outcomes for the local and alternative pairs. Table 2 gives the terminal lookup rule.

3pt

The gate accepts the candidate rubric only in the first row. The asymmetric case \(a^+\simeq a^-\) but \(b^+\succ b^-\), and any order conflict, do not support a clear decision; we reconstruct and rejudge only the ambiguous pair. An unstable comparison provides no reliable evidence, so \(r\) is not added at this step. We do not conclude that \(r\) is inherently invalid or harmful.

3.4 Tree-structured rubric evolution↩︎

The evolution tree records how accepted Add and Split operations update the active rubric set. Algorithm 2 gives the complete procedure, including the pairwise trial for every candidate rubric and the two update rules. PairwiseTrial returns the gate decision and feedback, which contains the gate interpretation and, when applicable, a compact counterexample for evolution memory.

An accepted Add attaches a new rubric \(r\) as an active leaf. For Split\((r_p\!\rightarrow\!\{r_1,\ldots,r_m\})\), every child is evaluated against the same \(\mathcal{B}_t=\mathcal{R}_t\setminus\{r_p\}\); neither the parent nor any sibling is included in a child’s trial background. A committed split retires \(r_p\) and attaches its accepted children. Rejected patches leave \(\mathcal{T}_{t+1}=\mathcal{T}_t\), and at every step the active leaves form \(\mathcal{R}_{t+1}\).

Ordinarily, a split is committed only when all children pass. We allow a partial split when at least one child passes and every omitted child receives a stable over-specific or optional/style rejection. A harmful or inconclusive child result leaves the entire patch unchanged. Evolution memory \(\mathcal{H}_{t+1}\) records the patch, each decision, its plain-language reason, and a compact counterexample. For example, an over-specific rejection stores the valid \(b^-\) strategy so that the next proposer does not restate the same strategy as another universal requirement. Rejected operations guide subsequent proposals without entering the active rubric set.

4 Experiments↩︎

4.0.0.1 Benchmarks and metric.

We first evaluate whether query-only rubric evolution produces rubric sets that support accurate pairwise preference judgments. Our evaluation covers five benchmark suites and seven evaluation sets: JudgeBench [21], RM-Bench Chat [22], RewardBench Chat and Chat-Hard [23], RewardBench 2 Precise-IF and Focus [24], and RubricBench [25]. We report pairwise accuracy on every set, assigning 0.5 credit when the two responses receive equal rubric scores.

4.0.0.2 Baselines.

We compare against two groups of rubric generators. The query-only baselines—Direct-Generate, TICK [6], and RocketEval [18]—construct a rubric from the query without training a task-specific generator. The trained open-weight baselines—Rubric-RM-8B [8], Rubric-ARM-8B [26], and Rubric-ARROW-8B [27]—learn dedicated rubric generators from external data.

4.0.0.3 Implementation details.

For a controlled comparison, all API-based query-only methods use Gemini 3.5 Flash for rubric generation, and all methods use Gemini 3.1 Pro as the shared verifier. Our rubric proposer and alternative-pair generator use temperatures of 0.7 and 0.2, respectively; all other decoding temperatures, including those of every baseline generator and the shared verifier, are set to 0.

4.0.0.4 Results & Analysis.

Table 3 reports the pairwise preference accuracy of all rubric generators.

3.2pt

We have the following observations: Direct rubric generation is unreliable without a quality signal. Although a one-pass generator can produce plausible rubrics from the query, it receives no supervision that distinguishes a useful rubric from a merely reasonable-sounding one. It therefore has no mechanism to identify or correct poor rubrics before they enter the final set. Existing generators can propose useful rubrics but cannot ensure their quality. Query-only and trained generators both perform well on individual benchmarks, showing that they can recover important aspects of response quality. However, their rankings vary substantially across evaluation sets because they lack candidate-level validation. Over-specific requirements and shallow criteria based on format or style can therefore remain in the rubric set and degrade preference judgments. Pairwise validation enables Rubrics on Trial to produce consistently high-quality rubrics. Rubrics on Trial obtains the highest accuracy on six of the seven evaluation sets, improving over the strongest baseline in each of these columns by 1.31–4.19 percentage points. It also outperforms every trained open-weight generator on all seven sets. The only exception is JudgeBench, where Rubrics on Trial ranks second to TICK (88.55 versus 91.21). By admitting a candidate rubric only after both complementary response-pair checks support it, Rubrics on Trial filters out low-value, over-specific, and style-level rubrics before they affect the final rubric set.

5 Conclusion↩︎

In this work, we introduced Rubrics on Trial, a query-only framework that evolves a rubric set from an empty set using synthetic rubric-conditioned response pairs. Rather than directly comparing abstract rubric sets, the framework tests each candidate rubric through two complementary response comparisons. The local pair evaluates whether violating the candidate makes an otherwise matched response worse, while the alternative-answer pair checks whether another correct strategy can remain equally strong without it. Accepted Add and Split operations update a tree-structured rubric set, and rejected proposals provide feedback for subsequent evolution. Across five preference benchmark suites and seven evaluation sets, Rubrics on Trial achieves the best average accuracy and the highest accuracy on six sets, demonstrating the effectiveness of pairwise validation for query-specific rubric generation.

5.0.0.1 Limitations & Future Work.

Our current experiments focus primarily on preference benchmarks, which measure whether generated rubrics can discriminate between responses in agreement with human preferences. Although this provides broad evidence for rubric quality in evaluation, it does not yet establish whether the resulting rubric signals improve downstream reinforcement learning or policy performance. Future work will therefore evaluate Rubrics on Trial in rubric-based post-training. We also have not yet conducted systematic component ablations or sensitivity analyses. These studies should isolate the contributions of the local pair, alternative-answer pair, tree-structured evolution, and evolution memory, and examine sensitivity to the evolution budget, generator and judge models, and decoding settings. Finally, because the framework relies on synthetic responses and LLM pairwise judgments, broader model families, task domains, and human evaluation are needed to characterize its robustness.

References↩︎

---

1. Haoxuan Li and Hao Wang are the corresponding authors.↩︎