1 Introduction↩︎

Scientific and high-performance applications often need to move between programming models such as CUDA, OpenMP, OpenCL, and serial CPU implementations as hardware, compiler support, and deployment constraints change [1]. Parallel API translation is therefore an important but difficult code-generation problem because translations must preserve the source computation while satisfying target-specific requirements for parallel structure, memory movement, synchronization, and API usage [2], [3].

Large language models and autonomous code agents can propose flexible transformations beyond rigid source-to-source rules [4]–[6], but in executable parallel-code translation, even plausible looking outputs may fail to compile, crash, or violate reference behavior, pushing many systems toward generate\(\rightarrow\)test\(\rightarrow\)repair loops.

Small differences in intermediate reasoning can lead to different compile\(\rightarrow\)run\(\rightarrow\)validate outcomes, and poor high-level parallelization choices can be hard to repair after decoding. This motivates an earlier intervention point: can nearby reasoning trajectories lead to better executable outcomes before any token-level repair is attempted?

Existing guidance methods typically score decoded programs, textual reasoning traces, or completed candidates [7]–[10]. Latent reasoning offers an earlier intervention point by shifting some test-time computation into continuous hidden-state trajectories before standard decoding resumes [11]–[14]. However, most latent-reasoning systems still follow a single hidden trajectory and do not compare nearby alternatives before the final decoding.

We study a test-time latent guidance method for executable parallel API translation that keeps the primary model frozen, samples perturbed hidden-state branches during latent reasoning, scores partial latent prefixes with a smaller PRM, and resumes generation from the highest-scoring branch, placing task-specific adaptation in the compact guidance model rather than retraining the generator. Related work and positioning are discussed in Appendix 5.

Empirically, latent PRM guidance improves ParaTrans validation by 9.21 points over unguided latent reasoning, and the gain persists when inserted into the same three-attempt repair loop. A held-out branch-selection test further suggests that the PRM can learn a weak but useful local preference signal to avoid undesirable latent continuations.

We make three contributions. First, we identify pre-decoding latent branch selection as a distinct intervention point for executable parallel-code translation. Second, we instantiate this intervention with a smaller PRM and evaluate it on ParaTrans, where correctness requires compilation, execution, and validation rather than textual similarity. Third, we provide bounded mechanism-level evidence from a held-out branch-selection test that a smaller PRM can provide a conservative local preference signal over continuous latent branches before final code decoding.

2 Method↩︎

We propose a test-time optimization method for guiding latent reasoning without fine-tuning the main generator in parallel code translation. Specifically, we keep a large latent-reasoning primary model frozen and train a smaller PRM to score partial latent trajectories before text decoding, as shown in Figure 1.

2.1 Latent-reasoning Primary Model↩︎

Let \(M\) be a frozen autoregressive model equipped with continuous latent reasoning. In our main experiments, \(M\) is LLaMA-3.3-70B. Following the COCONUT-style latent-reasoning paradigm [11], the model performs a fixed number of latent steps before returning to standard token decoding. Let \(h_t \in \mathbb{R}^{d_M}\) denote the primary model’s hidden state at latent step \(t\), before the language-modeling head. During the latent phase, \(h_t\) is not decoded into a discrete token; instead, it is mapped back into the \(M\) input space and used as the next continuous input.

Instead of training the main model in the manner proposed in [11], we use the training-free alignment transform of [14]. Given the input embedding matrix \(W_{\mathrm{in}}\) and output unembedding matrix \(W_{\mathrm{out}}\), we compute \[W_a = (W_{\mathrm{out}}^\top W_{\mathrm{out}})^{-1} W_{\mathrm{out}}^\top W_{\mathrm{in}},\] the closed-form solution to \[\min_{W_a} \| W_{\mathrm{out}} W_a - W_{\mathrm{in}} \|_F^2 .\] At latent step \(t\), the aligned continuous input for the next step is \(z_{t+1}=h_t W_a\). We use \(K=6\) latent steps when collecting PRM training trajectories and \(K=12\) latent steps during inference.

2.2 Latent process reward model↩︎

We train a smaller reward model to estimate the potential of a partial latent trajectory of the reasoning process via the mean terminal quality score. The PRM uses a Qwen-Coder-7B backbone with a scalar value head. Because the PRM receives hidden states from the primary LLaMA-3.3-70B model rather than its own token embeddings, we prepend a learned adapter that maps this hidden dimension to the PRM embedding dimension. In our implementation, this maps 8192-dimensional generator states to the 3584-dimensional Qwen-Coder input space using a linear layer, GELU activation, and layer normalization.

For every latent reasoning step \(t\), the PRM \(V_\theta\) receives the prefix trajectory \[\tau_{\leq t} = (h_1,\ldots,h_t)\] and predicts a scalar value \[V_\theta(\tau_{\leq t}).\] The target is the empirical terminal reward obtained from rollout sampling.

For rollout supervision at each latent step \(t\), we sample \(B_{\mathrm{train}}=3\) candidate branch states: two perturbed resamples and the unperturbed hidden state.

\[\label{eq:perturbation} \tilde{h}_{t}^{(b)} = h_t + \epsilon^{(b)}, \qquad \epsilon^{(b)} \sim \mathcal{N}(0, 1.4^2 I).\tag{1}\]

For each branch \(b\), we roll out to terminal code \(y_t^{(b)}\) and score the completed program with an automatic reward function \(S(\cdot)\).

The prefix target is the average terminal score of its continuations: \[R_t = \frac{1}{B_{\mathrm{train}}} \sum_{b=1}^{B_{\mathrm{train}}} S(y_t^{(b)}),\]

The PRM is trained with mean squared error: \[\mathcal{L} = \frac{1}{N} \sum_{i=1}^{N} \left( V_\theta(\tau_{\leq t}^{(i)}) - R_t^{(i)} \right)^2 .\]

2.3 Training data and reward construction↩︎

We construct the PRM training set from primary-model reasoning trajectories on ParaTrans samples used by [3] and supplement with additional samples created with the same methodology, balanced across CUDA\(\rightarrow\)OpenMP, OpenMP\(\rightarrow\)CUDA, Serial\(\rightarrow\)OpenMP, and Serial\(\rightarrow\)CUDA. We use 60 development samples for hyperparameter selection and train the final PRM on the combined training and development set before evaluating on the held-out test tasks, as shown on the left side of Figure 1.

The terminal reward \(S(y)\) combines executable and semantic signals, providing denser supervision than executable feedback alone. Compilation and execution receive fixed weights of \(0.30\) and \(0.25\), respectively. The remaining \(0.45\) weight is distributed across the validators available for the sample, proportional to their base weights. These validators include pass/fail validation, checksum or output matching when available, benchmark metrics, code-embedding similarity [15], an LLM-as-a-judge score, and a variable-comparison heuristic inspired by [6]. Final benchmark evaluation uses executable validation, not this mixed training reward. Additional reward and hardware details appear in Appendix 7.

2.4 PRM-guided latent inference↩︎

At inference time, we perform greedy branch selection during the latent reasoning phase, as shown in the inference panel of Figure 1. For every latent step \(t\), we score \(B_{\mathrm{test}}=8\) candidate branch states: the unperturbed state \(h_t\) and seven perturbed variants from Eq. 1 .

Each candidate forms a possible branch \[\tau_{\leq t}^{(b)} = (h_1,\ldots,h_{t-1},\tilde{h}_t^{(b)}).\] The PRM scores each candidate branch and selects \[b_t^\star = \arg\max_{b} V_\theta(\tau_{\leq t}^{(b)}).\] Only the selected branch is retained. The selected hidden state \(\tilde{h}_t^{(b_t^\star)}\) is aligned with \(W_a\) and fed back into the primary model for the next latent step. After \(K=12\) latent steps, the primary model returns to standard autoregressive decoding and produces the final target code. While initial training is done with \(K=6\) latent steps for efficiency, we find that using more latent steps at inference can further improve performance, as shown in Appendix 7.

3 Experimental Setup↩︎

The main evaluation uses the 76-task ParaTrans test set, where a translation counts as successful only if it compiles, runs, and passes integrated validation against the reference behavior. We compare vanilla LLaMA-3.3-70B, the fine-tuned LLaMA-3.3-70B baseline from [3], unguided latent reasoning with the same frozen base model, random latent branch selection with the same perturbation budget, and latent PRM guidance. Appendix 7 gives the full evaluation protocol, including direction counts, overlap checks, the repair setting, and the held-out branch-selection benchmark; direction-wise results appear in Appendix [sec:appendix-direction].

4 Results↩︎

4.1 Main executable translation results↩︎

Table 1 reports executable validation rates on ParaTrans. The key comparison is between unguided latent reasoning and latent PRM guidance, since both use the same frozen base model and differ only in whether latent branches are scored and selected. Latent PRM guidance improves validation by 9.21 percentage points without repair (paired task-bootstrap 95% CI [+3.07, +15.79], two-sided paired sign-flip \(p=0.0081\)) and by 8.78 percentage points under the same three-attempt repair loop (95% CI [+2.63, +15.35], \(p=0.0064\)). A random branch-selection baseline with the same perturbation budget reaches only 27.28% validation, suggesting that many latent perturbations are harmful rather than automatically useful. Without repair, latent PRM guidance already exceeds all non-PRM baselines, including repaired latent reasoning. With repair, it improves further to 45.18%, reinforcing the notion that pre-decoding guidance remains useful alongside later token-level repair. Appendix 6 reports the corresponding test-time overhead comparison, task-level win/loss/tie counts, and a post-decoding best-of-8 ablation.

4.2 Latent-trajectory branch analysis↩︎

We test whether perturbed latent continuations can improve on the unperturbed trajectory. On the 952-sample PRM training set, 95% of trajectory samples contain at least one candidate branch with higher terminal reward than the original. 59% of candidate continuations improve over their corresponding unperturbed branch. Among samples with at least one trajectory that passes both validation and output comparison, 25% succeed only through branching: the original trajectory fails, while at least one perturbed continuation succeeds. These results show that better latent continuations often exist locally, though distinguishing them from harmful ones remains difficult.

4.3 Branch-selection analysis↩︎

The branch-selection test isolates whether the PRM can choose a useful latent branch from a fixed local candidate set. To avoid leakage, this PRM excludes the 120 training trajectories used to construct the 500 branch-selection instances.

Table 2 provides bounded support for the mechanism behind latent guidance. Relative to random selection, the PRM more often selects a branch that improves over the original continuation or at least preserves it. While only modestly improving strict best-branch accuracy, this local advantage is applied repeatedly across consecutive latent steps and multiple candidate branches, where choices can compound before decoding begins. Together with the poor random-branch result in Table 1, this suggests that the PRM mainly helps by filtering harmful latent perturbations rather than by reliably finding the single best branch.

Limitations↩︎

This study is limited to one base-model family, one PRM backbone, and one executable parallel-code translation benchmark. The ParaTrans test set contains 76 held-out tasks, so the results should be read as evidence for this setting rather than as a general claim about latent guidance across models or domains. We also do not evaluate transfer of the learned value model across base models. We leave these broader evaluations to future work.

Latent PRM guidance assumes access to intermediate hidden states, which limits its application to self-hosted models or otherwise white-box models unless black-box APIs expose these states.

As with other domain-specific adaptation methods, the approach also requires task-specific supervision for the guidance model training. In our setting, this supervision is costly because parallel-code translations are long and each labeled example requires multiple roll-outs followed by executable and semantic validation. This limits the scale of the present study and makes broader robustness and transfer evaluations more difficult.

Finally, although the branch-selection analysis is consistent with the filtering interpretation, it does not by itself explain all success and failure cases. Finer-grained analysis of these cases could inform more reliable latent intervention strategies beyond local branch filtering.

Risk Statement↩︎

We do not identify material new risks from this work. The study is limited to offline benchmark evaluation of parallel-code translation.

Acknowledgments↩︎

Computational support was provided by Code Metal.

5 Expanded Related Work and Positioning↩︎

Execution-based and compiler-guided code-generation work has established that behavioral tests are a stronger signal for program quality than string overlap [16], [17]. Program-translation work then extended that view to translation settings through supervised adaptation, back-translation, distillation, and dialogue-based data generation [18], [19], [31]. These papers are relevant because they define the broader code-generation landscape in which our task sits, but their intervention points remain at the level of training data or decoded text rather than pre-decoding latent guidance.

HPC and parallel language translation-specific work is closer to our evaluation domain. UniPar, LASSI, ParaCodex, HPC-Coder, and HPC-Coder-v2 study executable parallel-code generation, while adjacent work studies parallelization assistance, fine-tuning, repair, or agentic adaptation with LLMs [3], [6], [20]–[22], [32]–[40]. Source-to-source translation systems such as SYCLomatic, HIPIFY, cu2cl, and TransCL intervene through deterministic or compiler-style program transformation rather than latent guidance [23]–[26], [41], [42]. Our setting is complementary: we ask whether a frozen generator can be steered before code is decoded. Thus, it is independent of downstream repair and transformation methods and can be combined with downstream repair or optimization methods.

Most existing guidance methods intervene after the program text is produced. Verifier-guided reasoning, chain-of-thought prompting, search, and PRMs provide the conceptual template for allocating additional test-time computation with auxiliary signals [7], [8], [27], [43]–[46]. For code, CodePRM and process-supervision-guided policy optimization show that execution-aware supervision can improve program generation [9], [10]. The key difference is that these methods score textual reasoning steps, decoded programs, or execution feedback after the text has been produced, whereas our method intervenes earlier by scoring continuous latent prefixes before code is emitted.

Latent reasoning offers an opportunity for this different intervention point. These papers motivate continuous hidden states as a useful reasoning substrate and shift part of test-time computation from explicit token sequences into continuous hidden-state trajectories before returning to standard decoding [11]–[14], [28], [29].  [30] is especially relevant because it treats latent computation as a search problem. Yet most latent-reasoning systems still follow a single hidden-state trajectory. They do not ask whether nearby latent branches can be compared before the final program exists. Our contribution is more application-specific: we study the applicability of a smaller PRM that scores candidate latent prefixes for executable parallel-code translation before any final program is decoded. A more distant but relevant line of work is activation steering. These methods modify internal activations at inference time, often by adding a contrastively derived direction to control a specific behavior [47], [48]. Code-focused variants use similar interventions for type prediction or language and library control [49], [50] but not in parallel code. We draw only on the broader idea that hidden vectors can serve as intervention points. Unlike activation steering, however, we move the intervention from decoding-time control to the latent-reasoning phase, and we use a learned value model to compare task-conditioned continuations rather than applying a fixed steering direction.

6 Supporting Results↩︎

6.0.0.1 Paired task win/loss/tie analysis.

We report task-level paired win/loss/tie counts using ParaTrans tasks averaged over three runs, matching the unit used for the paired bootstrap analysis. For each task, we compute the guided-minus-unguided validation-rate difference. A task is a win if the difference is positive, a loss if it is negative, and a tie otherwise. Confidence intervals and p-values follow the task-level paired bootstrap and two-sided sign-flip tests described in Appendix C, using 100,000 resamples or Monte Carlo samples. Table 4 shows that most tasks are unchanged, but among tasks whose outcome changes, PRM guidance more often improves than degrades validation in both the no-repair and repair settings.

6.0.0.2 Direction Split.

Table 5 breaks down compilation and validation rates by translation direction. PRM guidance improves over unguided latent reasoning in three of the four directions, with the largest gain on Serial\(\rightarrow\)OpenMP, smaller gains on the two CUDA/OpenMP directions, and only compilation gain on Serial\(\rightarrow\)CUDA. We view this split as additional descriptive evidence that the aggregate result is not driven by only one direction, rather than as proof of direction robustness. This pattern suggests that the aggregate improvement reflects both syntactic and semantic effects, since some directions improve mainly in compilation, some in validation, and some in both.

6.0.0.3 Latent Search Budget Ablation.

Table 6 ablates the test-time latent search budget by varying the number of candidate branches per step, \(B_{\mathrm{test}}\), and the number of latent reasoning steps, \(K\). Due to time and compute constraints, we construct PRM training trajectories under the smaller latent-step budget, but the preliminary pattern suggests that the PRM benefits from a larger decision surface at inference. More reasoning steps and more candidate options give the scorer additional opportunities to avoid weak continuations and occasionally choose stronger latent branches before decoding.

6.0.0.4 Test-time compute cost.

We estimate the incremental cost of PRM guidance relative to unguided latent reasoning under the same \(K=12\) latent-step setting with the 70B generator and one final decoding pass. PRM guidance adds \(B_{\mathrm{test}}=8\) branch scores per step, including the unperturbed branch, or \(8 \cdot 12 = 96\) Qwen-Coder-7B prefix-scoring calls per sample. By parameter count, this is roughly \(9.6\) LLaMA-3.3-70B-sized forward-step calls, while avoiding additional 70B rollouts or extra decoded programs. This is only an approximate call-level normalization and does not account for implementation details. The decoded non-code portion is unchanged in our logs, with high variance around 160 tokens in both settings, and the final code is decoded once in each condition.

This overhead is small compared with post-decoding repair. A repair attempt requires another full 70B decoding pass over the program, which in our logs generates approximately 1,853 tokens per attempt on average. Thus, the three-attempt repair setting adds approximately \(3 \cdot 1{,}853\) additional 70B decoding steps, compared with a much smaller auxiliary PRM-scoring budget. Despite this much smaller extra budget, latent PRM guidance without repair reaches 42.1% validation, exceeding unguided latent reasoning with three repair attempts at 36.4%. This comparison suggests that the gains are not explained simply by adding test-time computation.

Wall-clock measurements are also similar within run-to-run variation: 145.67\(\pm\)​9.33s per sample with PRM guidance versus 147.33\(\pm\)​4.67s for unguided latent reasoning over three runs. This parameter-count normalization is approximate, but it indicates that PRM scoring is not a dominant overhead in our implementation.

These measurements should not be read as a full efficiency claim. They show only that PRM scoring adds modest overhead in our implementation and remains compatible with token-level repair.

6.0.0.5 Post-decoding text-PRM reranking.

We also evaluate a post-decoding best-of-8 control that applies the same reward supervision after complete programs have been generated. We sample eight complete outputs from unguided latent reasoning without latent perturbations, using temperature 0.8 during decoding, and train a text PRM over completed answers with the same reward function used for latent PRM training. Candidates selected by the text PRM achieve 25.00% validation, while an oracle@8 over the same candidate sets reaches 36.84%. The oracle result indicates that the sampled sets sometimes contain validating programs, while the gap between oracle@8 and the text PRM shows that this control does not reliably identify them. Moreover, even this oracle remains below latent PRM guidance at 42.10%. This suggests that the main gain is not explained by PRM supervision alone. Taken together, these results support the value of applying the reward model before decoding over latent prefixes in this setting.

7 Hyperparameters and Reproducibility↩︎

7.0.0.1 Evaluation protocol.

Main evaluation uses the 76-task ParaTrans test set, split across CUDA\(\rightarrow\)OpenMP (18), OpenMP\(\rightarrow\)CUDA (19), Serial\(\rightarrow\)OpenMP (20), and Serial\(\rightarrow\)CUDA (19). We report three-run mean executable validation rates with and without a three-attempt self-repair loop. In the repair setting, failed translations are regenerated from the previous code and the execution-environment feedback. We also report a separate 500-instance held-out branch-selection benchmark, where each saved latent prefix is paired with the original branch and two perturbed alternatives, and instances without a unique best branch are excluded.

For pairwise comparisons, we use task-level paired inference. For each task, we average the binary validation outcomes over the three runs for each method, then compute the guided-minus-unguided task difference. We report the mean of these task-level differences. Confidence intervals are computed using a paired nonparametric bootstrap over tasks with \(B=100{,}000\) resamples. P-values are computed using a two-sided paired sign-flip randomization test over the same task-level differences with \(100{,}000\) Monte Carlo samples.

We used two NVIDIA H200 GPUs for primary-model inference when creating the dataset and running experiments. For PRM training and code evaluations, we used two NVIDIA A40 GPUs. For the three-attempt repair setting, we feed the execution-environment error and the previously generated code back to the model, then ask it to regenerate the full code. The exact prompt templates are shown below.

We train the PRM with the AdamW optimizer [51] in two stages. In the first stage, we train only the adaptation layer, which maps the primary model hidden dimension of 8192 to the PRM backbone hidden dimension of 3584, together with the value head, for half an epoch. In the second stage, we unfreeze the remaining layers. We train with an effective batch size of 6 and a learning rate of \(1\cdot 10^{-6}\).

7.0.0.2 Split and leakage audit.

We define each translation task by the fingerprint \((\texttt{kernel\_name}, \texttt{from\_api}, \texttt{to\_api})\). The PRM development split contains the first 15 kernels in each translation direction, for 60 examples total. It is used only for hyperparameter selection. After hyperparameters are fixed, the final PRM is trained on the union of the PRM training and development pools. For the branch-selection experiment, we train a separate PRM after removing the 120 source trajectories used to construct the branch-choice instances. The 76-task ParaTrans test set is held out from all PRM training, development, trajectory collection, reward construction, and hyperparameter selection. As a stricter check, we also compare kernel names while ignoring translation direction; this overlap is also zero.

7.0.0.3 Reward construction.

The reward target mixes executable and proxy components. We keep that distinction explicit because the benchmark metric in the main paper is executable validation only. For a terminal program \(y\), let \(C(y)\in\{0,1\}\) indicate successful compilation, \(E(y)\in\{0,1\}\) indicate successful execution without runtime failure, and \(A_j(y)\in[0,1]\) denote optional semantic or proxy validators available for that sample. We compute \[{!}{ S(y)=0.30 C(y)+0.25 E(y)+0.45 \frac{\sum_{j\in\mathcal{A}(y)} w_j A_j(y)}{\sum_{j\in\mathcal{A}(y)} w_j} }\] where unavailable validators are omitted and the remaining validator weights are renormalized. The optional validators include pass/fail integrated validation, checksum or output matching when available, benchmark-specific metrics, Jina code-embedding cosine similarity to the reference target code [15], a GPT-5-mini judge score, and the variable-comparison heuristic described below. In the common case where pass/fail validation, Jina similarity, and the GPT-5-mini judge are available, their effective contributions to \(S(y)\) are \(0.2423\), \(0.1038\), and \(0.1038\), respectively, in addition to the fixed \(0.30\) compilation and \(0.25\) execution weights.

7.0.0.4 GPT-5-mini judge.

The judge is used only for PRM target construction and never for final benchmark scoring. For each candidate rollout, GPT-5-mini receives the source program, and the generated target-language program. It is asked to assess semantic equivalence, target-API correctness, and likely behavioral preservation, then return a bounded scalar score in \([0,1]\) with a short justification. We use the scalar score as one optional validator in the renormalized reward above.

7.0.0.5 Variable-comparison heuristic.

The variable-comparison heuristic is intended to catch semantic shortcuts not exposed by unit tests alone. GPT-5-mini first identifies variables in the source and generated programs that represent the core computed quantity for the built-in test. We then instrument the reference and generated programs by adding print statements for the selected variables, execute both programs on the same test input, and compare the observed final values.

7.0.0.6 Inference prompt.

The following prompt is used for the initial (non-repair) translation pass and during dataset creation. The system message establishes the model’s role as an HPC translation expert, and the user template supplies the source API, target API, expected output file count and names, and the source code.

7.0.0.7 3 Attempt repair prompt.

The repair prompt is used in the three-attempt regeneration setting after a failed compilation or execution attempt. It provides the previous error trace together with the broken translation and asks the model to emit a complete corrected translation in a strictly parseable format.

References↩︎

---

1. GitHub:https://github.com/Scientific-Computing-Lab/Parallax.git↩︎