1 Introduction↩︎

Language models (LMs) are increasingly deployed as agents that act, observe, and revise rather than as systems that produce a single isolated answer. Coding agents react to tests and error traces, computer-use agents observe changing interface states, and assistants revise plans in response to user corrections or clarifications. Across these settings, feedback can do more than mark success or failure: it can explain what went wrong, identify missing constraints, and suggest a path to recovery. Improving from feedback is therefore becoming an important capability in its own right: an agent must recognize which parts of the feedback are actionable, preserve the parts of its attempt that are already correct, and revise the parts that caused failure.

Studying improvement from feedback requires separating several mechanisms that are usually entangled. A model may improve over turns because it receives useful corrective information, but it may also improve simply because it samples another attempt with more computation. When feedback does help, the limiting factor may lie on either side of the interaction: the teacher may fail to identify the relevant error, or the student may fail to translate a correct diagnosis into a better solution. We therefore distinguish feedback generation from improving from feedback by evaluating separate student and teacher models under controlled interaction histories, teacher-information conditions, and repeated-attempt baselines. This lets us ask not only whether feedback improves performance, but when it improves performance beyond self-refinement, and which part of the feedback system constrains that improvement.

We answer these questions in a student-teacher protocol on verifiable tasks. In each episode, a student model attempts a problem. If the attempt is incorrect, a teacher model provides natural-language feedback, and the student revises its answer. The verifier makes every attempt measurable, while the student-teacher split lets us vary the student, the teacher, the interaction budget, the visible history, and the teacher’s access to privileged task information. We instantiate this evaluation across four verifiable reasoning environments: Omni-MATH, Codeforces, BBEH Linguini, and ARC-AGI1, and run student-teacher interaction grids in which each model acts in both roles.

Across these settings, we find that multi-turn improvement is often not evidence of feedback use: repeated attempts explain a large share of the gain, self-feedback improves over generic retry inconsistently, the strongest feedback-specific improvements come from high-quality external teachers, and the student’s ability to use feedback explains more variation than the teacher’s identity.

In summary, our contributions are as follows:

• We introduce a controlled zero-shot student-teacher evaluation for natural-language feedback across four verifiable reasoning environments.

• We show that multi-turn improvement is not necessarily feedback use: self-feedback adds little beyond self-refinement, while the best external teachers add substantial feedback-specific gains.

• We find that interactive performance is primarily student-dependent: student identity explains substantially more gain variation than teacher identity, although teacher choice can matter for a fixed student.

• We show that longer histories and privileged teacher information are selective interventions rather than default improvements, with benefits depending on task and model capability.

2 Related Work↩︎

2.0.0.1 Self-refinement and repeated sampling.

Large language models are increasingly used as interactive agents that solve tasks over multiple turns rather than through a single generation [1]. In such settings, models must follow sequential instructions and incorporate new feedback across turns [2]. However, multi-turn improvement does not necessarily imply that the model has used feedback: later attempts also receive more test-time computation and additional samples. This distinction is especially important because additional computation is known to improve performance through mechanisms such as chain-of-thought prompting [3], and Best-of-\(N\) sampling [4]. Related agent systems such as Reflexion use verbal feedback or memories of past failures to improve future attempts [5]. Our work builds on this line of work by explicitly comparing feedback-conditioned improvement against unguided self-refinement, allowing us to estimate how much of the observed gain is feedback-specific rather than a consequence of repeated attempts.

2.0.0.2 Post-training LMs with teacher feedback.

Recent post-training methods increasingly frame model improvement as a student-teacher problem, where a student model learns from signals produced by a stronger teacher, verifier, or environment. Standard approaches include supervised fine-tuning, preference optimization, reinforcement learning from AI feedback, and reinforcement learning with verifiable rewards [6]–[10]. These methods are effective, but their feedback signals are often demonstrations, pairwise preferences, scalar rewards, or outcome-level verification. Such signals can improve behavior, but they provide limited information about which part of a student’s attempted solution

caused failure. This limitation has motivated recent work on denser teacher supervision, including on-policy distillation, where trajectories are sampled from the student and a teacher provides supervision on the student-generated states [11]. It has also motivated work on synthetic and interactive data generation, especially as high-quality human-generated training data becomes increasingly limited [12], [13]. In verifiable domains, natural-language feedback can be used to construct multi-turn training trajectories or improve interactive learning from feedback [14], [15]. Related work has also used natural-language feedback as a fine-tuning signal or as part of reinforcement-learning-style updates [16], [17]. Our work studies the test-time analogue of this post-training shift: a teacher observes a student’s failed attempt and provides diagnostic feedback, and we ask when this feedback improves performance beyond repeated attempts, whether stronger teachers provide more useful corrections, and whether the main bottleneck lies in feedback generation or student uptake.

2.0.0.3 Feedback generation and tutoring.

Several works study the quality of feedback itself. [18] identify conditions under which language feedback can support no-regret learning, including unbiased feedback, the agent’s ability to translate feedback into evaluations of candidate hypotheses, and sufficient representational coverage of the true environment. In mathematical reasoning, [19] evaluate whether models can detect and correct errors in flawed reasoning traces, while [20] analyze the tutoring behavior of GPT-family models beyond final-answer correctness. These works focus on whether feedback is valid, helpful, or pedagogically appropriate. Our work complements them by evaluating the full student-teacher interaction: a teacher may generate useful feedback, but improvement also depends on whether the student can act on it.

3 Experimental Setup↩︎

In this section we describe our experimental setup.

3.0.0.1 Environments.

We restrict our evaluation to environments, where for each problem and solution pair, it is possible to verify whether the given solution is correct. This assumption is often made in the literature . To make our results robust, we consider four challenging environments, that focus on different domains and test for different abilities: Omni-MATH [21] Olympiad-level mathematical reasoning benchmark. Codeforces [22], a competitive programming benchmark, the Linguini task from BIG-Bench Extra Hard (BBEH) [23], which specifically challenges the model’s ability to learn new rules and ARC-AGI [24], which presents grid transformation puzzles solvable only by inferring abstract rules from a few examples.

3.0.0.2 Interaction protocol.

For investigating the influence of feedback, we consider a realistic scenario of a back-and-forth conversation between a model and an entity capable of giving it feedback (for simplicity, we call them student and teacher). Each episode is a trajectory \(\tau=(\text{problem formulation}, s_1, a_1, t_1, \dots, s_n, a_n)\), where the initial model input contains the problem formulation. In turn \(i\), \(s_i\) is the student’s output, \(a_i\) is the student’s answer. In the case where the task-specific verifier recognized \(a_i\) as the wrong answer, \(t_i\) is the teacher’s feedback on the student’s attempt. Then, the next turn \(i+1\) starts, where the student model attempts to solve the problem again, with the added information of the teacher’s feedback. The conversation continues until a maximal \(n\) is reached or the student’s answer is correct. Our interaction protocol is visualized in  2.

3.0.0.3 Student’s input.

Lead by the intuition that the recent feedback is more important than old feedback, we restrict the student’s input to only \(h\) historical turns, where \(h\) is a hyperparameter. Therefore, the student’s input is \((\text{problem formulation}, s_k, a_k, t_k, \dots, s_n, a_n)\), where \(k=\text{max}(n-h, 0)\).

3.0.0.4 Teacher’s input.

In previous works [14], [15], the teacher model was granted access to specific task knowledge, denoted as \(k_t\). Due to the varying nature of the evaluation environments we have chosen, this additional information in this work have taken different forms depending on dataset used: an in-depth step-by-step solution, a plain final answer, or an error stack trace derived from the student’s generated code. Because smaller models are prone to inadvertently leaking the direct solution when given such context, we establish a strict baseline where no additional information is provided to the teacher (\(k_t = \text{none}\)). However, to better understand the impact of privileged context, we also include ablations that selectively provide this knowledge based on the specific environment. The configurations are as follows: for the Math environment, the teacher receives either no information, the final answer, or the full solution; for the BBEH environment, the knowledge is restricted to either no information or the final answer; and for the Code environment, the teacher receives either no additional information or the complete solution.

3.0.0.5 Models considered.

To evaluate the performance of both teacher and student, we choose from thirteen models from four open-weight families, spanning a parameter range of 1B to 120B, both dense and Mixture-of-Experts (MoE) architectures, and models with and without dedicated reasoning modes. These are series of Gemma4-{E2B, E4B, 26B-A4B, 31B} [25]; the collection of Llama-3.x-{1B, 3B, 8B} [26]; the family of Qwen3.5-{2B, 4B, 9B, 27B} [27]; and the GPT-OSS models: GPTOSS-{20B, 120B} [28].

3.0.0.6 Ablation tier.

To further deepen our research, we conducted ablation studies to isolate the effect of different parameters on multi-turn performance. Due to the large search space and computational limitations, we focused solely on the Gemma-4 family, resulting in a \(4 \times 4\) student-teacher matrix for each ablation. This provides us with a relatively exhaustive set of interactions between smaller and larger models. Because of the conversation setup, we test whether access to information from previous turns yields any benefits. For this purpose, we vary the maximum visible history length in \(\{1,2,3,4,5\}\). Additionally, we test how the number of turns, also called the interaction budget, affects the final accuracy. We evaluate it for \(K \in \{1,5,10\}\). Since the conversation progresses sequentially, we can generate the full sequence for \(K=10\) and subsequently extract the states for \(K=5\) and \(K=1\) during post-processing. Finally, to decompose the gains from additional test-time compute, we added a self-refinement baseline for each model. This approach maintains the student’s history but does not receive feedback from the teacher.

3.0.0.7 Metrics.

To evaluate the different configurations and teacher-student pairs, we measure the accuracy at various steps of the interaction trajectory, denoted as \(\text{acc}@K\). For a more comprehensive analysis, we introduce additional metrics: raw gain (\(\text{acc}@K - \text{acc}@1\)), normalized gain (\(\frac{\text{acc}@K - \text{acc}@1}{1 - \text{acc}@1}\)), and the Area Under the Curve computed as the mean cumulative accuracy over all turns (\(\frac{1}{T} \sum_{k=1}^{T} \text{acc}@k\)). Both \(\text{acc}@K\) and AUC serve as primary measures of the system’s overall problem-solving capability, additionally reflecting how quickly the student improves across turns. Raw and normalized gain instead isolate how much of that improvement is attributable to the teacher’s feedback and iterative reasoning, independent of the student’s starting level. Notably, normalized gain provides a saturation-adjusted comparison across different model scales by accounting for benchmark saturation, ensuring that larger models are not penalized for possessing a higher initial baseline accuracy (\(\text{acc}@1\)).

4pt

4 What matters for improvement from feedback?↩︎

In this section we present our experimental results. We organize them around the following six empirical questions, which we answer through research findings (RFs):

4.1 Feedback-specific gains and interaction budget↩︎

4.1.0.1 RF1: Feedback-specific gains are limited unless the teacher provides substantially better guidance than generic retry or self-feedback.

The feedback-control comparison in 1 shows that multi-turn improvement is not, by itself, evidence that a student benefits from feedback. Averaged over matched students, self-feedback is inconsistent relative to the self-refinement baseline: it changes \(\mathrm{acc}@10\) by \(-1.2\) to \(+8.7\) points and \(\mathrm{gain}@10\) by only \(+0.1\) to \(+6.0\) points across environments. The largest self-feedback lift appears on ARC-AGI1, while BBEH Linguini slightly favors unguided self-refinement in final accuracy. Thus, simply adding a feedback message, even one generated by the same model, does not guarantee feedback-specific improvement. In contrast, selecting the best available teacher for each student adds \(9.2\) to \(16.6\) points in \(\mathrm{acc}@10\) and \(9.9\) to \(17.3\) points in \(\mathrm{gain}@10\) over self-refinement. This suggests that gains depend not just on the presence of a feedback message, but on whether the feedback provides corrective information that the student can use.

4.1.0.2 Interaction budget also matters.

The marginal-gain curves in 3 show that the benefits of additional turns are front-loaded. Attempt 2, the first with feedback, accounts for \(30.5\%\)–\(58.1\%\) of total \(\mathrm{gain}@10\), and attempts through \(K=5\) recover \(70.8\%\)–\(86.8\%\). Later turns can still matter for hard long-horizon cases, especially ARC-AGI1, but their average marginal return is small. For data collection, this favors shorter trajectories with informative feedback over long rollouts whose gains mostly come from repeated attempts.

4.2 Dense model matrix: separating student and teacher effects↩︎

We next analyze the dense model matrix described in 3: each model is evaluated both as a student and as a teacher across the four environments, with episodes run for up to \(K=10\) student attempts. This cross-product lets us ask whether interactive outcomes are explained more by the model receiving feedback or by the model generating it. [fig:omni-math-accuracy-matrices-main] shows a concrete slice of this matrix for Omni-MATH, while the full dense matrices for all environments and metrics are reported in 7.3.

4.2.0.1 RF2: First-turn accuracy and feedback adherence are different capabilities.

A high initial solve rate does not imply that a model makes the best use of feedback. 6 shows that this distinction is not a one-off comparison: students with similar \(\mathrm{acc}@1\) can have very different \(\mathrm{ngain}\). A qualitative example can be seen in 6.9. It showcases how teacher and student cooperate to get to correct solution. A concrete comparison is given in [tab:single-turn-vs-plasticity] of 7.2: GPT-OSS-120B starts higher on Omni-MATH, Codeforces, and BBEH Linguini, but Gemma4-31B usually has much larger \(\mathrm{gain}\) and reaches the better final outcome. ARC-AGI1 is the complementary case: GPT-OSS-120B has slightly larger \(\mathrm{gain}\), but Gemma4-31B starts much higher and remains the best final student. Thus \(\mathrm{acc}@1\) measures what the student can solve immediately, while \(\mathrm{ngain}\) measures how efficiently it converts feedback rounds into recovery opportunities.

4.2.0.2 RF3: Interactive performance is mostly student-dependent.

We use the dense matrix to marginalize over roles: for each student, we average performance over teachers; for each teacher, we average performance over students. 5 shows the resulting role means for \(\mathrm{gain}\) at \(K=10\). Across all environments, the spread across students is much larger than the spread across teachers, indicating that the model receiving feedback explains more of the interactive gain than the model generating it. The appendix fixed-effect analysis in 3 supports the same conclusion. Student fixed effects explain \(77.1\%\)–\(96.5\%\) of pair-level \(\mathrm{gain}\) variation, while teacher fixed effects explain only \(1.3\%\)–\(12.4\%\). Adding teacher identity after student identity gives a small incremental improvement, at most \(0.124\) \(R^2\). This does not mean that teacher identity is irrelevant. For a fixed student, the gap between its best and worst teacher can be substantial, as shown by the appendix teacher-range analysis in 4. The dense-matrix results suggest that teacher effects are smaller, but can become meaningful once we condition on a particular student.

4.2.0.3 RF4: Teacher task competence only partially predicts interactive teacher competence.

We use each model’s \(\mathrm{acc}@1\) as a student as a noisy proxy for single-turn task competence, and ask whether this ranking predicts its performance as a teacher for a fixed student. If models that solve more problems on the first turn were always stronger teachers, teacher rank and interactive \(\mathrm{auc}\) would be monotonic within each student. [tab:teacher-monotonicity] shows that this monotonicity only partially holds. Codeforces has the strongest relation, with mean Spearman \(\rho=0.56\pm0.02\), but the other environments are weaker (\(0.21\pm0.06\) to \(0.39\pm0.06\)). Inversions are also common: in \(28.9\%\pm1.1\%\)–\(41.4\%\pm1.9\%\) of comparable teacher pairs, the lower-\(\mathrm{acc}@1\) teacher produces higher interactive \(\mathrm{auc}\). This suggests that single-turn solving ability and teaching ability are related but not equivalent. A stronger solver may be more likely to know the correction, but interactive performance also depends on diagnosing the student’s specific error, phrasing actionable feedback, and matching the student’s ability to use it.

4.3 Gemma4 ablations: interaction history and privileged teacher information↩︎

The dense matrix uses a relatively cheap and scalable protocol: short visible history and no ground-truth answer or solution access for the teacher. We use this as the main setting because both alternatives create tradeoffs. Longer histories may help models diagnose repeated failures, but they increase cost and can run into context limits. Privileged information may make feedback more accurate, but answers or solutions are not always available, may be costly to obtain, and can leak into the teacher’s feedback. The Gemma4 ablations measure how much performance these more expensive sources of information buy.

4.3.0.1 RF5: Longer visible history is a capability-dependent tradeoff.

Longer histories expose repeated failure modes to the teacher and give the student access to earlier corrections. This information is not free: every additional visible turn increases cost and makes context overflow more likely. It is also not automatically useful, the models must be able to identify which parts of the history matter and use them to revise the next attempt. The max-history ablation shows little change from history 1 to 2, followed by a larger jump at history 3. Averaged over teacher-access settings and Gemma4 pair cells, moving from history 1 to 5 increases \(\mathrm{auc}\) by \(0.094\) on Omni-MATH. In the full grid, Codeforces shows the largest average history benefit, with \(\mathrm{auc}\) increasing by \(0.123\). BBEH Linguini shows a smaller but positive gain, rising from \(0.210\) at history 1 to \(0.248\) at history 3. These averages suggest that history can help, but the benefit should be interpreted as conditional on model capacity rather than as a free improvement from longer context.

This capability dependence is visible when we split by teacher-student strength.

In the matched history-5 minus history-1 diagnostic (19), Codeforces gains about \(0.21\) \(\mathrm{auc}\) for pairs where the teacher’s single-turn \(\mathrm{acc}@1\) exceeds the student’s, compared with about \(0.05\) for the remaining pairs. This is consistent with stronger teachers using history to diagnose repeated failures, although the ablation still does not isolate teacher adaptation from student-side conditioning.

4.3.0.2 RF6: Privileged information helps selectively, not uniformly.

We ask whether giving the teacher additional task-side information reliably improves the interaction. The answer is task-dependent. At history 5, BBEH Linguini shows the largest effect: answer access increases \(\mathrm{auc}\) from \(0.151\) to \(0.331\) and \(\mathrm{acc}@10\) from \(0.195\) to \(0.463\). This suggests that privileged information is most useful when the teacher would otherwise struggle to infer the target correction. This large BBEH separation is visible in 7 The effect is much smaller elsewhere. On Omni-MATH, answer access improves \(\mathrm{auc}\) from \(0.428\) to \(0.475\), while full solution access reaches only \(0.471\). On Codeforces, where the teacher already receives structured execution feedback, adding the reference solution changes \(\mathrm{auc}\) only from \(0.547\) to \(0.556\). ARC-AGI1 shows a smaller but measurable answer-access effect: \(\mathrm{auc}\) increases from \(0.303\) to \(0.328\), while \(\mathrm{acc}@10\) increases from \(0.362\) to \(0.408\). Thus, additional teacher-side information is not automatically useful; it helps when it resolves a real teacher uncertainty and can be converted into actionable feedback. The teacher-level lift diagnostic in 20 shows the same non-uniformity across teachers and environments, and the full access summary is shown in 18. Because answers and solutions are not always available and may leak into the feedback, we treat these conditions as upper-bound ablations rather than the default scalable protocol. To quantify how privileged information affects solution leakage from teacher to student, we conduct a leakage audit in 8 of 6.6.

5 Discussion↩︎

5.0.0.1 Summary

Our findings have practical consequences for evaluating and building feedback-based agents. First, repeated-attempt and self-feedback controls should be included whenever multi-turn gains are reported; otherwise, extra inference compute can be mistaken for feedback use. Second, teacher selection should not be based only on single-turn task accuracy. The teacher monotonicity analysis shows that stronger solvers are not always stronger teachers for a particular student. Third, longer histories and privileged teacher information should be treated as targeted interventions rather than default improvements: they help when they resolve a real diagnostic bottleneck, but they also increase cost, context pressure and leakage risk.

5.0.0.2 Limitations

Our study has several limitations. First, the current ARC-AGI1 analysis does not yet include the same ablations as the other environments. Although we provided preliminary qualitative analysis of the feedback trajectories, conducting a broader analysis in the future will be necessary to achieve more conclusive results. These analyses are important for understanding whether improvements come from actionable feedback rather than formatting artifacts, repeated sampling, or direct leakage from the teacher context. Second, our experiments focus on a zero-shot setting with short-horizon verifiable tasks and open-weight models. This makes the protocol controlled and reproducible, but it leaves open how the conclusions transfer to models trained to use feedback, longer-horizon interactive domains, and closed-source systems.

5.0.0.3 Future Work

A natural direction for future work is to connect this diagnostic protocol to post-training. In supervised fine-tuning or reinforcement-learning settings, feedback trajectories could be used not only to measure whether a model improves within an episode, but also to train models that become better feedback users over time. Extending the study to longer-horizon domains such as computer usage would test whether the same student-side bottleneck appears when feedback must be integrated across many actions, persistent state, and delayed consequences. Finally, comparing open-weight and closed-source models would clarify which findings reflect general properties of current language models and which depend on the model families studied here.

6 Experimental Setup Details↩︎

6.1 Protocol Definition↩︎

Our evaluation protocol follows the didactic feedback loop of [14], but uses it only as a zero-shot evaluation environment. A single episode contains a task instance, a student model, a teacher model, and a maximum number of student attempts. The first student message contains the task but no teacher feedback. After each student attempt, the environment evaluates correctness. Correct attempts terminate the episode immediately. Incorrect attempts trigger a teacher turn unless the attempt budget has already been exhausted.

Let \(x\) be the public task prompt, \(z\) be optional privileged task information, and \(h_t\) be the prompt-visible interaction history before the \(t\)-th student attempt. The student generates \[a_t \sim \pi_S(\cdot \mid x, h_t, f_{t-1}),\] where \(f_{t-1}\) is empty on the first turn. The verifier returns \(c_t=1\) if \(a_t\) is correct. If \(c_t=0\) and \(t<K_{\max}\), the teacher generates \[f_t \sim \pi_T(\cdot \mid x, a_t, h_t, z_T),\] where \(z_T\) is determined by the teacher-information condition. In the dense model matrix, \(z_T\) excludes the final answer and full solution for math and BBEH, and excludes the trusted reference solution for code. In code, the teacher can still see execution feedback unless explicitly ablated.

The teacher is always instructed to provide diagnostic feedback rather than the final answer. In the main prompt style, the teacher may reason in <think> tags, but the feedback passed to the student is parsed from the final <feedback> block. If parsing fails, the raw teacher text is passed through.

6.1.0.1 History truncation.

The implementation maintains separate student-side and teacher-side chat histories. The parameter max_history controls how many completed interaction turns are kept in the prompt. When the history window is full, older refinement turns are removed before adding the next user message. The original user prompt containing the task description is preserved, so every later student or teacher call still has the problem context and the chat remains a natural continuation of the same task.

6.2 Run Configuration↩︎

The current dense-matrix runs use the following common settings:

• Agent: hierarchical_separate, with separate vLLM actor instances for student and teacher.

• Interaction budget: \(K_{\max}=10\) student attempts.

• History: max_history=1.

• Student sampling: temperature \(0.7\), top-\(p=0.95\), max tokens \(8192\).

• Teacher sampling: temperature \(1.0\), top-\(p=0.95\), max tokens \(8192\).

• Context length: \(32768\) tokens.

• Max output tokens: \(8192\) tokens.

• Number of workers: 16 evaluation workers.

ARC-AGI uses max output tokens \(16000\) for both student and teacher because the task JSON and output grids can be long.

6.3 Model Set↩︎

The dense matrix uses four model families totaling to thirteen model identifiers:

• Gemma4:

  • gemma-4-E2B-it,

  • gemma-4-E4B-it,

  • gemma-4-26B-A4B-it,

  • gemma-4-31B-it,

• Qwen3.5:

  • Qwen3.5-2B,

  • Qwen3.5-4B,

  • Qwen3.5-9B,

  • Qwen3.5-27B,

• Llama:

  • Llama-3.2-1B-Instruct,

  • Llama-3.2-3B-Instruct,

  • Llama-3.1-8B-Instruct,

• GPT-OSS:

  • gpt-oss-20b,

  • gpt-oss-120b.

Each model appears once as the student and once as the teacher, for 169 pairs per dense-matrix environment.

6.4 Datasets and Verifiers↩︎

6.4.0.1 Omni-MATH.

Each example contains a problem, final answer, ground-truth solution, and source. The student is asked to end with one boxed final answer. Correctness is evaluated by two math answer checkers: a PRIME-style normalization and symbolic equivalence checker, followed by a math_verify-based checker. The environment returns reward 1 if one checker accepts the answer.

6.4.0.2 Codeforces.

Each example contains a prompt, optional starter code, trusted reference solution, test specification, source, and difficulty metadata. The student returns a complete Python solution in a fenced code block. Candidate code is run in a subprocess sandbox on at most 16 test cases. A task is solved only when all tests pass. Current timeouts use dataset-aware scaling: per-case timeout defaults to 10 seconds, is multiplied by 10 when a dataset time limit is available, and is clamped between 5 and 20 seconds; the full-task timeout is derived from the case timeout and number of cases.

6.4.0.3 BBEH (Linguini).

Each example contains a task name, problem text, reference answer, source, and deterministic problem id derived from the task, problem, and answer. We filter the BBEH dataset to the Linguini task. Correctness is evaluated by lowercasing and normalizing the prediction and reference, stripping simple LaTeX wrappers, handling common answer prefixes, and applying a small set of fuzzy matches for parenthesized choices, numeric strings, quotes, brackets, and trailing question marks. The environment returns reward 1 if the normalized prediction matches the reference under this evaluator.

6.4.0.4 ARC-AGI1.

Each example contains a task id, training input/output grid pairs, test input grids, and hidden target output grids used only by the verifier. The student must return a JSON object, using the schema {"outputs": [[[...]], ...]}. Correctness is evaluated by parsing the JSON, validating that each predicted output is a well-formed ARC grid with the expected number of test outputs, and comparing the predicted grids to the hidden target grids. The environment returns reward 1 only when all test outputs match exactly. Structured feedback reports parsing failures, shape errors, or grid mismatch diagnostics, truncated to 2000 characters.

6.5 Teacher Information Conditions↩︎

We use the following teacher reference modes:

• none: the teacher receives no ground-truth answer or solution.

• answer: the teacher receives only the final answer or reference target.

• solution: the teacher receives the full ground-truth solution or trusted reference solution.

For code, reference mode is crossed with execution-context mode:

• structured: number of tests passed, pass rate, timeout status, and first-failure details when available.

• feedback: the environment’s compact textual execution feedback.

• traceback: only stderr or traceback text when available.

• none: no execution-side signal beyond the student code and problem prompt.

6.6 Leakage audit↩︎

For the purpose of evaluating we have chosen two out of four environments, specifically Omni-MATH and BBEH Linguini. To assess whether the models leak information, we prompted an external large language model judge with a triplet containing the (solution, student answer, teacher response). The judge model was instructed to return True if the teacher leaked confidential information that trivially helped the student solve the problem. Fixing the student model as Gemma 4 26B, we iterated across all teacher models to quantify the extent of information leakage. We calculated the leak rate as the ratio of identified leaks to the total number of judge responses.

We can see in 8 that no more than 5% of teacher answers contained a leak information. Consistently no additional teacher access scores lower leak rates compared to both answer access and solution access. The differences mostly vary in smaller models.

6.7 Metrics from Episode Logs↩︎

Each completed episode log contains at least solved, attempts_used, max_turns, and, when available, problem_id. For each cutoff \(k\), the postprocessing code marks an episode as solved by \(k\) if solved=1 and attempts_used <= k. If multiple episodes share a problem id, the code averages within problem first and then averages across problems. Otherwise each episode is treated as its own problem.

The main reported metrics are: \[\mathrm{acc}@k = \frac{1}{N}\sum_i \mathbf{1}[\text{problem } i \text{ solved by } k]\]

\[\mathrm{gain}@K = \mathrm{acc}@K-\mathrm{acc}@1,\]

\[\mathrm{ngain}@K = \begin{cases} (\mathrm{acc}@K-\mathrm{acc}@1)/(1-\mathrm{acc}@1), & \mathrm{acc}@1 < 1\\ 0, & \mathrm{acc}@1 = 1 \end{cases}\]

\[\mathrm{auc}= \frac{1}{K_{\max}}\sum_{k=1}^{K_{\max}}\mathrm{acc}@k\]

We also report \(\mathrm{acc}@2-\mathrm{acc}@1\) because a very large first refinement jump can indicate a formatting or first-turn protocol artifact rather than steady feedback use.

6.8 Prompt Templates↩︎

This section records the prompt templates used by the current implementation. Braced names such as {problem} denote task-specific fields inserted at runtime.

6.8.0.1 Shared teacher system prompt.

You are a careful tutor. Read the student's latest attempt, identify the most
important mistake, and give targeted feedback without revealing the final answer.

6.8.0.2 Shared teacher output format.

Pinpoint the step in which I am making a mistake.
Provide the most informative piece of information for me to succeed on the next
try, without telling me the final answer.
You may reason about the mistake first inside <think>...</think> tags.
Your final feedback message must appear exactly once inside
<feedback>...</feedback> tags.

Return only:
<feedback>
your feedback
</feedback>

6.8.0.3 Omni-MATH student prompts.

System:
You are a mathematician solving competition-level math problems.

Initial user prompt:
Problem:
{problem}

Solve the problem carefully.

You may reason in <think>...</think>. End with exactly one final line in this
format:
\boxed{your answer}

Refinement user prompt, feedback condition:
Feedback:
{teacher_feedback}

Revise your previous solution so it uses the feedback.

You may reason in <think>...</think>. End with exactly one final line in this
format:
\boxed{your answer}

Refinement user prompt, self-refinement/no-feedback condition:
Revise your previous solution and produce a corrected solution.

You may reason in <think>...</think>. End with exactly one final line in this
format:
\boxed{your answer}

6.8.0.4 Omni-MATH teacher prompts.

Initial teacher user prompt:
I am trying to solve this question:
{problem}

Here is my current attempt:
{previous_attempt}

{reference_context}

{teacher_output_rules}

Follow-up teacher user prompt:
Here is my current attempt:
{previous_attempt}

{teacher_output_rules}

The reference context is one of:

You do not have access to the ground truth solution or final answer.

Here is the ground-truth final answer:
{answer}

Here is the ground-truth solution:
{solution}

6.9 Example of interaction between student and teacher↩︎

This section provides an example of a complete dialogue between teacher gpt-oss 120B and student Gemma-4-31B. To make it more parsable outputs of student have been summarized. Teacher in this trajectory provided student with necessary refinement to complete the task.

Agent Interaction: Student-Teacher Correction

Student [Step 0] — Reasoning & Initial Hypothesis:
Analyzes the input grids and identifies the bounding box of the active shape (color 4). Hypothesizes that the target output is a \(3 \times 3\) grid of blocks forming a cross shape. Attempts to deduce the geometric transformations for each arm of the cross, but incorrectly formulates the policy: assuming the right arm is a horizontal reflection and the bottom arm is a vertical reflection, rather than pure rotations.

Student [Step 0] — Action:
Submits a \(17 \times 17\) output grid. (Result: Incorrect, Reward: 0.0)

Teacher [Step 1] — Feedback:
“You correctly noticed that the output is built from four copies of the input’s colored shape arranged in a cross, but the transformations you applied to the arms are not the ones used in the examples. In the training pairs, the top arm is the shape rotated \(90^\circ\) clockwise, the left arm is the original shape, the right arm is the shape rotated \(180^\circ\) (i.e., both axes flipped), and the bottom arm is the shape rotated \(270^\circ\) clockwise (or equivalently a \(90^\circ\) counter-clockwise rotation).

For non-square shapes, the rotated arms swap height and width, so the top and bottom blocks have the dimensions of the input \(W \times H\), while the left and right blocks keep the original \(H \times W\). Because you used a horizontal flip... several cells are placed in the wrong positions. Re-examine the transformations: left = original, top = \(90^\circ\) cw, right = \(180^\circ\), bottom = \(270^\circ\) cw, and assemble the output accordingly.”

Student [Step 1] — Reasoning & Correction:
Integrates the teacher’s feedback to update the transformation policy. Re-evaluates the active bounding box, explicitly noting it is non-square (\(H=5, W=6\)). Calculates the necessary dimensional swaps for the top and bottom arms (\(W=5, H=6\)). Methodically maps the pixel coordinates for \(S_{90}\), \(S_{180}\), and \(S_{270}\) applied to the base shape \(S\), and reconstructs the spatial layout of the final grid using the corrected rotational rules.

Student [Step 1] — Action:
Submits the corrected \(17 \times 17\) output grid. (Result: Success, Reward: 1.0)

7 Additional Results↩︎

The initial sections present detailed dense-matrix tables and performance visualizations for all model pairings across the tested environments, including Omni-MATH, Codeforces, BBEH Linguini, and ARC-AGI1. They do it for all types of our metrices we used: accuracy, cumulative accuracy, and performance gains.

Subsequent sections focus on specific ablation studies and deeper mechanistic insights, In them we are using gemma 4 dense matrices. We isolate the impact of relative teacher strength, information access and maximum history, providing a thorough analysis of the variables that govern reasoning improvements in multi-turn feedback settings.

7.1 Supplementary Dense-Matrix Tables↩︎

7.2 Supplementary Plasticity Table↩︎

7.3 Supplementary Model-Matrix Figures↩︎

7.4 Supplementary Trajectory Figure↩︎

7.5 Supplementary Gain-by-Role Figures↩︎

7.6 Supplementary Gemma4 Ablation Figures↩︎

7.7 Feedback-Control Tables↩︎

References↩︎