1 Introduction↩︎

Greenhouse climate control is more than keeping one variable near one setpoint. Heating, ventilation, CO₂ dosing, screens, lighting, crop response, and operating cost all act through the same climate system. Classical optimal-control and model-based-control studies make this coupling explicit by specifying the greenhouse and crop model before optimizing operation [1]–[4]. Energy-aware control depends on the same modelling choices: conclusions change with the representation of heat exchange, humidity, CO₂, crop growth, and weather forcing [5]. Reinforcement learning (RL) is attractive because simulation can generate many policy rollouts without putting a crop at experimental risk. Recent greenhouse studies have therefore compared RL with model predictive control (MPC), combined RL and MPC, tested robust or model-based variants, and released benchmark environments for controller development [6]–[12].

This coupling also makes a high simulator return difficult to interpret. A grower or control engineer does not only ask whether the return increased; they ask what the controller did to obtain it. Heating, screen closure, CO₂ enrichment, ventilation, humidity control, and supplemental lighting have different physical meanings and different failure modes. CO₂ enrichment, for example, depends on light and crop state, and high light under unsuitable temperature can create stress rather than useful assimilation [13]–[16]. Once these effects are collapsed into a single profit-like reward, a small return gain may reflect a physiological response, a shifted setpoint, a screen-use shortcut, a favorable weather episode, or a scaling artifact. This paper therefore asks a practical reporting question: can the reward terms used for simulator training remain interpretable and comparable when they are checked against calibrated simulator rollouts and logged greenhouse records?

Process-based greenhouse simulators such as GreenLight [17] provide a useful starting point because they expose climate states and actuator channels. GreenLight-Gym [12] wraps GreenLight as an OpenAI Gym environment with six controls: heating, CO₂, thermal screen, ventilation, lighting, and blackout screen. Its default reward is still a scalar that combines growth revenue and energy cost, so it does not show which physiological or actuator terms drive the policy. Reward shaping can make learning easier, but it can also change what the agent optimizes unless the shaping assumptions remain visible [18]–[21].

1.1 Related work and gap↩︎

Greenhouse RL has begun to move from proof-of-concept policy learning toward benchmarked comparisons with MPC and robust optimization. GreenLight-Gym provides fixed weather inputs, an actuator interface, and a rule-controller baseline for simulator studies [12]. Recent work also makes clear that nominal return is not enough. A controller must respect climate constraints, remain usable under model mismatch, and be interpretable enough for growers to inspect or adjust [8], [9], [22]. These expectations are familiar in model-predictive control, where constraints and forecasts are explicit, and in safe RL, where deployment limits are part of the learning problem [2], [23].

Two prior lines of work motivate treating reward design as an audit target. The first is sim-to-real caution. Robotics and control studies often use simulation for safe data generation, but policies can fail when simulator dynamics, actuation, sensing, or aggregation differ from the target system [24]–[26]. The same issue appears in greenhouse simulation: a reward component computed every 15 minutes in simulation cannot be assumed comparable with a daily aggregate score from logged greenhouse data. The second line is explainable RL. Policy extraction, decision-tree distillation, and broader XRL taxonomies provide tools for inspecting learned behavior without treating the neural policy itself as transparent [27]–[29]. In agricultural modelling, explainable AI is also used to show which measured variables support a prediction rather than treating black-box accuracy as sufficient [30]. In greenhouse control, this leads to a channel-wise question: can the learned screen, lamp, CO₂, or ventilation action be approximated by a shallow rule under the same simulator distribution?

Reward shaping can encode domain knowledge, but the shaping assumptions also need to remain inspectable. Potential-based shaping preserves policy invariance under clean conditions [18], [20]. Greenhouse reward terms rarely meet such conditions because growth, weather, climate comfort, and actuator costs are coupled. Decomposed rewards can therefore serve a diagnostic role: physiological terms can be inspected individually, and the resulting actions can be checked for shallow-rule approximability [23], [31], [32].

The missing step is a practical reward check. Existing greenhouse RL studies often report aggregate simulator return, but rarely show whether reward terms have consistent semantics across simulator rollouts and logged greenhouse records, whether an energy term is a measured quantity or an action proxy, or whether a small return margin can be traced to recognizable actuator behavior. This study uses the same named components for three connected tasks: training a GreenLight-Gym policy, scoring AGC greenhouse logs, and inspecting actuator behavior with shallow rules. The audit asks three bounded questions: whether the reward remains learnable, whether action changes occur in interpretable regimes, and whether logged data point to crop-specific calibration targets.

1.2 Contribution and scope↩︎

This study contributes a calibration-first reward-component audit protocol for greenhouse RL, designed to test whether a reward remains learnable, comparable across simulator and logged records, and interpretable at actuator level before controller-transfer claims are made.

The framework has three steps (Fig. 1). First, the scalar GreenLight-Gym objective is decomposed into named components for temperature, CO₂, humidity/VPD, screen use, and weather-normalized actuation. Second, reward interpretation is gated by adapting GreenLight to AGC2 logged climate traces before simulator rollouts are used as evidence. Third, the same components are recomputed on AGC1, AGC2, and AGC4 greenhouse logs [33]–[35], and learned actuator channels are distilled into shallow rules so screen, lamp, heating, ventilation, and CO₂ behavior can be inspected directly rather than inferred from scalar return alone.

2 Materials and methods↩︎

2.1 Simulator environment and action interface↩︎

Experiments use GreenLight-Gym, a reinforcement-learning benchmark built around the GreenLight process model. GreenLight represents greenhouse climate, crop growth, lighting, heat exchange, and weather forcing. GreenLight-Gym exposes these dynamics through state observations, a continuous actuator vector, and the official rule-based controller distributed with the benchmark. The action vector contains six normalized controls: pipe heating, CO₂ dosage, thermal screen, ventilation, supplemental lighting, and blackout screen. Because these controls are normalized simulator actions, action-cost analyses are reported as actuation-proxy diagnostics rather than as physical resource accounting.

The simulator is used here as a controlled test bed for reward design and calibration analysis. Process-based models are useful because they expose state variables and actuator channels, but their numerical outputs still depend on equations, timestep resolution, weather forcing, and unit conventions. The analysis keeps three quantities separate: native simulator reward, component reward terms used for inspection, and physical greenhouse resource use.

The baseline controller is the official rule-based policy distributed with the benchmark. A learned policy must clear this floor before its component behavior is interpreted. This requirement keeps the analysis tied to a policy that is both learnable and inspectable, rather than to traces that are easy to explain but poor controllers. Evaluation summaries report native simulator reward together with component contributions, radiation-conditioned actions, and rule-fidelity scores.

2.2 Conditional component reward↩︎

The training reward keeps the native GreenLight-Gym reward visible while adding a diagnostic physics-prior term: \[r_t = 0.7 r_t^{\mathrm{native}} + 0.3 r_t^{\mathrm{diag}}, \qquad r_t^{\mathrm{diag}} = \sum_{k \in \mathcal{K}} w_k r_{t,k}.\] Here \(t\) indexes the 15-min simulator timestep, \(r_t^{\mathrm{native}}\) is the original GreenLight-Gym scalar reward, \(r_t^{\mathrm{diag}}\) is the diagnostic physics-prior term, \(\mathcal{K}\) is the set of named reward components, \(r_{t,k}\) is the timestep score for component \(k\), and \(w_k\) is its component weight. The 0.7/0.3 mixture was used as a fixed audit-oriented design choice to keep the native GreenLight-Gym objective dominant while exposing diagnostic component traces; it was not tuned against AGC outcomes. The mixture is not proposed as an optimal greenhouse-control reward. Sensitivity to actuation-proxy weighting is reported in Additional file 1, and broader mixture optimization is outside the present audit claim. All component weights were fixed at \(w_k=1\) unless explicitly varied in the actuation-proxy sweep. No component weight was fitted to AGC yield, AGC team ranking, or any logged greenhouse outcome.

The diagnostic term records seven components: temperature comfort, temperature direction, CO₂ direction, humidity/VPD, light, screen, and actuation proxy (Table 1). Daytime in simulator rollouts is defined as outside radiation \(>50\) W m\(^{-2}\). VPD is computed as \[\mathrm{VPD}(T,RH)=0.6108\exp\left(\frac{17.27T}{T+237.3}\right)(1-RH/100),\] with \(T\) in \(^\circ\)C, \(RH\) in percent, and VPD in kPa.

4pt

2.3 Reward-component implementation↩︎

The component reward is implemented as a wrapper on top of the scalar GreenLight-Gym reward. At each timestep, component scores are computed from the simulator state, weather context, and normalized action vector. Their sum enters the training reward, while the individual component values are retained for ablation, one-out ranking, and comparability checks.

For AGC logs, the same component names are recomputed from daily climate summaries and actuator proxies. Cucumber uses \(24.0/19.0^\circ\)C day/night setpoints and a radiation threshold of 3 in the daily CO₂ gate; tomato and cherry tomato use \(21.5/18.5^\circ\)C and threshold 5. When direct physical flow is unavailable, logged heating and CO₂ values are treated as setpoint-derived proxies, allowing the same reward map to locate calibration targets in public greenhouse records.

2.4 Training protocol↩︎

Policies are trained with Proximal Policy Optimization (PPO) [21]. The official rule-based controller provides the base action, and the neural policy learns a bounded residual (\(\pm 0.15\) scale). VecNormalize is applied to observations and rewards. Training uses three start days (120, 180, 240) and three evaluation seeds (42, 123, 456). The start days evaluate the controller across different seasonal radiation and temperature regimes rather than tuning it for a single weather episode. Evaluation rollouts are summarized by native reward, action traces, and component scores.

The rule-controller floor is the official GreenLight-Gym rule-based policy executed in the same simulator, weather, start-day, and evaluation setting as the learned policies. It is retained as a sanity check: a learned policy must beat this floor before component behavior is interpreted. Native reward is used for learnability and floor-clearing, whereas component terms are used for inspection. This separation matters because component traces can be informative even when the native-reward margin is small.

2.5 Configurations↩︎

Three configurations are evaluated. The tomato component-reward wrapper uses a \(21.5/18.5^\circ\)C day/night set-point pair, the cucumber component-reward wrapper uses a \(24.0/19.0^\circ\)C pair, and the scalar-reward tomato baseline uses the same tomato temperature pair.

2.6 Diagnostic analyses↩︎

The evaluation uses six reward-audit checks instead of one performance score. First, each reward wrapper must clear the rule-controller floor after 500k training steps. Second, rollout actions are stratified by hour and solar-radiation bin to test whether the gated CO₂ term changes behavior in a physiologically plausible regime. Third, one-out component ablation checks whether any single component dominates the normalized reward scale; the full ranking is reported in Additional file 1, the supplementary PDF accompanying this manuscript.

The remaining checks ask whether the reward can still be read after learning and across domains. A reward–actuation-proxy sweep evaluates tradeoffs in normalized action space, where the horizontal axis is a simulator action proxy rather than measured energy. Policy distillation fits shallow decision trees to each actuator channel [27]–[29]. Finally, the same named components are scored on logged AGC greenhouse data [33]–[35] and compared with team-level yield rankings using Spearman correlation. Before that cross-domain comparison, a component-comparability audit checks aggregation, units, and semantics.

For the calibration-first branch, AGC2 logged weather, actuator proxies, and indoor climate targets are converted into compact replay tables with blocked validation windows. Candidate GreenLight parameter sets are evaluated by replaying logged actions and comparing simulated and measured \(T_{\mathrm{air}}\), relative humidity, and CO₂. A candidate passes the quality gate if it improves the blocked validation objective across action-supported contexts without degrading validation temperature RMSE by more than \(1^\circ\)C. Only after this gate is passed are reward components recomputed on the default simulator, the candidate simulator, and the corresponding real-log rows at matched aggregation scales.

The AGC2 calibration data contain 47,809 weather rows and 23,910 action/target rows in the compact hourly contract. The raw time-block split assigns the first 80% of each team trace to calibration and the last 20% to validation, but the C3c search uses stricter action-supported 3-day replay windows. Within each selected 3-day window, the first half is used for candidate scoring and the second half for blocked validation. Both imputed and boiler/CO₂-complete action policies are considered; windows must retain at least 75% of expected rows and sufficient boiler/CO₂ support. The selected candidate is one shared GreenLight parameter set across all selected AGC2 contexts, not a separate per-team fit.

The validation objective is the sum of normalized RMSE values for air temperature, relative humidity, and CO₂, using scale factors of \(5^\circ\)C, 20 percentage points, and 500 ppm, respectively: \[J=\frac{\mathrm{RMSE}(T_{\mathrm{air}})}{5} +\frac{\mathrm{RMSE}(RH_{\mathrm{air}})}{20} +\frac{\mathrm{RMSE}(CO2_{\mathrm{air}})}{500}.\] The local search budget was 24 candidates with seed 41, initialized around a C3b warm start and filtered by the \(1^\circ\)C validation temperature-degradation guard. Table 2 lists the searched parameter bounds and the selected C3c candidate values.

3pt

2.7 Reproducibility and release-asset map↩︎

The release package separates the submission-facing evidence from reusable code. The main manuscript reports the diagnostics needed for the argument, Additional file 1 preserves supplementary ablation and artifact checks, and the code repository provides reproduction scripts and component definitions for reuse on new greenhouse traces.

3 Results↩︎

The decomposed reward first had to remain learnable under PPO. Subsequent checks test whether selected actuator changes occur in interpretable regimes, whether component scales are comparable, and whether AGC2 calibration changes rollout accuracy. The final checks score the same components on AGC logs, measure action-proxy tradeoffs, and distill actuator decisions into shallow rules.

3.1 The decomposed reward remains learnable under PPO↩︎

All nine 500k-step PPO runs clear the official GreenLight-Gym rule-based controller floor [12] (Table 3). The component reward matches or slightly exceeds the scalar baseline: tomato component \(169.54 \pm 0.78\) vs scalar \(168.82 \pm 0.99\) (\(+0.72 \pm 0.45\)), cucumber component \(169.47 \pm 0.42\) (\(+0.66 \pm 0.72\)). In this training setting, the physics-prior wrapper adds interpretable component traces without reducing native simulator reward.

After facility adaptation, all six PPO runs in the AGC2-adapted simulator also cleared the rule-controller floor. Scalar-reward tomato achieved \(89.526 \pm 0.663\) and component-reward tomato \(89.392 \pm 0.540\). Matched-seed margins were mixed (\(+0.554\), \(-0.912\), \(-0.042\); mean \(-0.133\)), so neither reward design dominates in the adapted environment. The adapted simulator supports stable policy training under both scalar and component rewards.

A matched 1M-step budget check in the adapted simulator also remained above the rule-controller floor for all seeds and gave a small directional component-reward margin (mean \(+0.662\) native reward; Additional file 1). This result is retained as learnability and budget-sensitivity support rather than reward-dominance evidence.

3.2 CO₂ actions shift modestly toward radiation-supported regimes↩︎

Time-conditioned rollout analysis shows that the tomato component reward modestly increases day/night CO₂ action separation in two saved-policy replays. In replay A, the day/night contrast is \(0.452\) under the component reward and \(0.437\) under the scalar baseline (margin \(+0.015\)). In replay B, the contrast increases from \(0.378\) to \(0.464\) (margin \(+0.086\); Fig. 2). Radiation-bin margins are less stable: the 100–200 W m\(^{-2}\) bin is positive in replay A (\(+0.067\)) but slightly negative in replay B (\(-0.014\)), even though this is the regime where the gated CO₂ term is physiologically motivated [13]. The two replays correspond to training seeds 42 and 456 for reproducibility. The supported statement is therefore narrow: the component reward modestly strengthens day/night CO₂ separation in the available replays, while radiation-bin differences remain a simulator-side diagnostic rather than a stable mechanism claim.

3.3 Component scores require scale and semantic matching↩︎

One-out component ablation shows that all component indices are close to zero at the normalized reward scale (largest: temperature direction \(+0.0056\); see Additional file 1 for the full ranking table). The greenhouse reward operates as an interacting system in which temperature, radiation, CO₂, humidity, screen, and cost terms shape actuator behavior jointly, consistent with the coupled physics of greenhouse climate control.

Before cross-domain comparison, simulator and AGC component scores were matched for aggregation scale. The simulator sums 96 timestep-level component values per day, whereas AGC logs yield one daily row. After scale matching, the apparent temperature-comfort gap fell from approximately five reward units to \(0.07\)–\(0.10\) units. The remaining CO₂-direction difference reflects a semantic gap between a timestep radiation gate and a daily proxy. Additional file 1 gives the full comparability audit table and figure.

3.4 AGC2 calibration improves state rollout and component comparability↩︎

The facility-adapted GreenLight candidate improved state-rollout accuracy across four unique AGC2 team/window traces represented by seven preprocessing validation contexts (Table 4 and Fig. 3)¹, with no context exceeding the \(1^\circ\)C temperature-degradation guard. Mean paired improvements over default GreenLight [17] were \(+0.184^\circ\)C in temperature RMSE, \(+4.274\) pp in relative-humidity RMSE, and \(+562.529\) ppm in CO₂ RMSE across the seven preprocessing rows (positive = lower error).

3pt

Reward-component comparability also improved after adaptation. Mean Wasserstein-1 distance between real-log and simulator component distributions decreased from 0.01677 to 0.01389 at the timestep scale and from 1.19748 to 0.97708 at the daily-sum scale. The aggregate component improved across all paired validation contexts, though the row-level improvement fraction was 62/112 (55.4%), reflecting uneven gains across individual component channels.

3.5 Logged AGC scores identify crop- and channel-specific alignment↩︎

Computing the same physics-prior components on logged AGC greenhouse data [33]–[35] links the simulator reward design to observed production records (Fig. 4). On AGC1 cucumber, the conditional composite prior gives a positive but underpowered team-level rank correlation (\(\rho=0.829\); asymptotic \(p=0.042\); exact permutation \(p=0.058\); six teams), improving over an earlier physics-prior variant by \(\Delta\rho=+0.172\). Across all three AGC datasets (18 team-level points), within-dataset standardized scores give a pooled correlation of \(\rho=0.468\) (asymptotic \(p=0.050\); Monte Carlo permutation \(p=0.053\)).

The alignment is crop- and channel-specific. At the composite level, AGC2 cherry tomato (\(\rho=0.200\)) and AGC4 dwarf tomato (\(\rho=0.029\)) are weak. Individual component channels are more informative: AGC2 CO₂ direction (\(\rho=0.657\)), AGC2 actuation proxy (\(\rho=0.771\)), and AGC4 CO₂ direction (\(\rho=0.829\); exact permutation \(p=0.058\)). Bootstrap confidence intervals and permutation checks reflect the small team-level sample sizes (AGC1 composite 95% CI [0.000, 1.000]; pooled CI \([-0.016, 0.858]\); BH-FDR \(q=0.101\) for the strongest signals). These correlations are used to locate likely calibration targets, not to validate the reward as an agronomic objective.

3.6 Actuation-proxy tradeoffs remain action-space diagnostics↩︎

The actuation-proxy sweep reports normalized control burden as an engineering diagnostic. The component reward includes action penalties, and the available quantities are normalized simulator actions rather than audited resource meters. The sweep therefore tests whether reward gains coincide with changed action magnitudes inside the simulator. In this action space, the physics-prior reward creates favorable reward–actuation tradeoff points relative to the scalar baseline (Additional file 1, Table S4). At actuation-proxy weight \(0.5\), the component reward achieved \(0.5663\) vs scalar \(0.5596\), showing that the proxy can reveal whether a reward improvement is being bought through actuator shortcuts.

3.7 Rule fidelity provides the strongest actuator-level evidence↩︎

Rule-fidelity analysis provides the clearest behavioral evidence for the physics-prior reward. Decision-tree distillation [27] shows higher shallow-rule approximability for thermal screen, lighting, and blackout screen actions under the component reward than under the scalar baseline (Table 5; Fig. 5). Thermal-screen \(R^2\) rises from 0.652 to 0.835, lamp from 0.631 to 0.838, and blackout screen from 0.592 to 0.750. These are actuator channels where growers expect rules tied to radiation, temperature, and weather. In this setting, the physics-prior reward makes the main actuator decisions easier to summarize as shallow greenhouse rules.

In the facility-adapted simulator, both reward designs produce highly rule-approximable policies: mean shallow-tree \(R^2\) across six actuators was 0.945 (scalar) and 0.943 (component), with random-forest \(R^2\) at 0.951 and 0.956. The convergence in rule-fidelity after facility adaptation indicates that the adapted GreenLight dynamics constrain actuator behavior toward physically structured rules regardless of the training reward. Channel-level differences between reward designs are small and mixed in the adapted environment.

A depth-sensitivity replay on the same three-seed calibrated traces supports the same interpretation (Additional file 1). Using local CART surrogates with minimum leaf size 50, the mean actuator \(R^2\) across six channels was already about 0.92 at depth 2 and increased smoothly to about 0.95 at depth 5 for both scalar and component rewards. The calibrated-domain rule-fidelity result is therefore not an artifact of one depth-4 tree choice; it also shows that much of the post-calibration interpretability gain comes from the adapted simulator distribution shared by both reward designs.

Additional file 1 retains the full component ablation, pairwise interaction, actuation-proxy sweep, policy-distillation, and component-comparability tables.

4 Discussion↩︎

The central contribution is a reward-component audit for greenhouse RL. Rather than relying on a single simulator return, the study follows the same named components through policy learning, AGC2 calibration, AGC log scoring, and shallow-rule actuator analysis. This gives greenhouse RL studies a practical way to report how reward is earned, which parts of the simulator need calibration, and which actuator channels are suitable for rule-based inspection.

The results separate protocol validation, simulator-side behavior, and logged-data alignment. Component rewards remained trainable, AGC2 calibration improved blocked state rollout and reward-component comparability, and default-simulator screen and lighting actions became easier to approximate with shallow rules. The AGC yield-ranking result serves as calibration-target evidence: useful for deciding what to check next, but too small to validate the reward as an agronomic objective.

4.1 Why AGC log scoring matters↩︎

Scoring the same reward components on AGC logs moves the reward check outside the simulator. It allows a candidate RL reward to be compared with observed crop-management records before it is adopted as a training objective. On AGC1 cucumber [33], the crop-adjusted composite prior gives a positive but underpowered team-level rank correlation (\(\rho=0.829\)), with bootstrap intervals, exact permutation testing, and BH-FDR correction limiting the strength of this six-team result. AGC2 and AGC4 show crop- and channel-specific alignment. CO₂-direction channels are informative in tomato logs even when the composite score is weak.

AGC1, AGC2, and AGC4 data are used here for retrospective reward-component scoring and calibration-target identification [33]–[35]. This use is valuable because it shows which reward terms can be evaluated on public greenhouse records and which crop-specific channels need improved calibration or measurement support.

4.2 Interpretability versus transfer↩︎

Rule fidelity and sim-to-real transfer are different claims. The comparability check ensures that simulator and real-log component scores share aggregation, units, and semantics before cross-domain conclusions are drawn, following standard sim-to-real caution [24]–[26]. The AGC2 facility-adaptation branch showed that a candidate parameter set can improve both state rollout and aggregate reward-component comparability, while the method also identifies which components still require calibration.

The convergence of rule-fidelity scores after facility adaptation (\(R^2 \approx 0.94\) for both reward designs) is informative. In the default simulator, the physics-prior reward produces more rule-approximable actuator behavior. After adaptation narrows the simulator–facility gap, the adapted dynamics themselves constrain both reward designs toward similar actuator structure. This suggests that the physics-prior reward is most valuable when simulator–facility mismatch is largest.

4.3 Implications for greenhouse RL reporting↩︎

The results argue for reporting greenhouse RL policies with more than aggregate simulator reward. In this study, the useful reporting items are the rule-controller floor, the distinction between physical quantities and action proxies, shallow-rule summaries of actuator channels, and the ability to score the same reward components on logged greenhouse data. This reporting style connects RL studies back to the older greenhouse-control literature, where classical optimal-control studies made modelling assumptions explicit [1], [2], [4]. The reward wrapper provides a practical link between agronomic knowledge and RL evaluation: the same named terms train a simulator policy, score production data, identify calibration targets, and summarize actuator behavior for engineering inspection.

4.4 Interpretation limits↩︎

Three interpretations remain outside the current evidence. The CO₂ diagnostic strengthens day/night separation in two saved-policy replays, but mixed radiation-bin margins do not support a stable CO₂ mechanism claim. Scoring AGC logs identifies crop-specific calibration targets, but the small team-level samples and permutation/FDR results do not validate the reward as an agronomic objective. The AGC2-adapted PPO runs clear the rule-controller floor under both scalar and component rewards, but their mixed 500k margins do not support component-reward dominance.

4.5 Future calibration path↩︎

A stronger sim-to-real study would need matched aggregation on both sides of the comparison and a simulator model that remains stable after adaptation [25]. On the simulator side, component scores can be summarized as timestep means, daily aggregates computed after first aggregating state and action variables, and season-level summaries. On the logged-data side, action proxies can be replaced where possible by audited physical quantities such as heating energy, lamp energy, CO₂ dosage, and screen position. The comparison can then be repeated under weather regimes shared by simulation and data, with sensitivity analyses for start date, radiation distribution, and actuator support.

Here, GreenLight is adapted to logged AGC2 climate trajectories and the reward and policy diagnostics are repeated in the adapted candidate simulator. With richer facility data, the same loop can support a practical deployment pathway: first calibrate the process model to logged weather, actions, and climate states; then add residual correction where the process model is biased; then use the audited reward components to select candidate policies for greenhouse trials. A production-first extension would treat climate states, VPD, CO₂, and light as intermediate variables for yield or biomass prediction, while keeping cost and energy terms as optional diagnostics rather than headline outcomes. A residual-hybrid simulator is a natural next direction because it can preserve process-model structure while learning facility-specific climate residuals.

4.6 Limitations↩︎

• All policy training and rollout results are simulator-based. Prospective greenhouse trials would be needed to evaluate closed-loop operation.

• AGC log scoring uses small team-level samples (6–10 teams per dataset). Bootstrap confidence intervals and BH-FDR correction bound statistical power accordingly.

• Facility adaptation was selected and validated on AGC2 only; generalization to other facilities remains untested.

• Several selected AGC2 calibration parameters lie at or near the local-search bounds. The selected vector should therefore be read as a facility-adaptation coefficient set, not as a uniquely identifiable estimate of greenhouse equipment properties.

• Actuation-proxy tradeoffs reflect normalized simulator actions, not measured physical energy.

• GreenLight-Gym is a simplified greenhouse model; residual-hybrid extensions require further validation before stronger transfer claims.

5 Conclusions↩︎

This study establishes reward-component auditing as a reproducible intermediate step between simulator-only greenhouse RL and facility-specific validation. It shows that greenhouse RL reward design can be inspected across policy learning, calibrated rollouts, AGC log scoring, and actuator-rule distillation. The supporting evidence is that the component reward remained trainable, AGC2 facility adaptation improved state-rollout accuracy across four unique team/window traces represented by seven preprocessing validation contexts, and real-log–simulator reward-component distances decreased. The behavioral evidence is strongest in the default simulator, where thermal-screen, lamp, and blackout-screen decisions became more rule-approximable under the physics-prior reward; after facility adaptation, both scalar and component rewards became similarly rule-approximable, indicating that adapted dynamics dominate that comparison. Together, these results support reward-component auditing as a pre-deployment diagnostic layer for greenhouse RL; field-level yield gains and closed-loop controller transfer require dedicated greenhouse trials.

Data availability↩︎

GreenLight and GreenLight-Gym are cited as external model and benchmark resources [12], [17]. The AGC logged data used for real-log reward-component scoring are available from the original 4TU.ResearchData releases for AGC1 [33], AGC2 [34], and AGC4 [35]. We release the reward-audit workflow developed in this study, including reward-component definitions, the physics-prior reward wrapper, and multi-component scoring and validation code. The repository URL is https://github.com/shennongwm/rl-reward-audit.

CRediT authorship contribution statement↩︎

Yuhui Bie: Conceptualization, methodology, software, formal analysis, visualization, writing – original draft, writing – review and editing. Guowei Xu: Data curation, validation, investigation, visualization, writing – original draft, writing – review and editing. Yaojun Wang: Conceptualization, resources, supervision, project administration, writing – review and editing.

Declaration of competing interest↩︎

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Funding↩︎

This research is supported by the Joint Fund Project of the National Natural Science Foundation of China (U25B2019-1).

Acknowledgments↩︎

The authors thank the GreenLight and GreenLight-Gym developers, and the Wageningen Autonomous Greenhouse Challenge organizers and participants for making the simulation environment and logged greenhouse datasets available.

Declaration of generative AI and AI-assisted technologies in the manuscript preparation process↩︎

During the preparation of this work the authors used OpenAI Codex/GPT and Anthropic Claude for language polishing, code-checking assistance, and organization of revision notes. After using these tools, the authors reviewed and edited the content as needed and take full responsibility for the content of the published article.

Supplementary diagnostic tables↩︎

This file contains the detailed diagnostic evidence behind the preprint. The tables document reward components, actuation proxies, rule fidelity, logged-data component scoring, and component-comparability checks used to interpret the physics-prior reward.

Table S1a. Reward-component definitions used for training and AGC-log scoring↩︎

Table S1. Ordinary 100k-step component ablation↩︎

Supplementary Figure S1. 500k-step simulator learnability↩︎

All configurations clear the GreenLight-Gym rule-controller floor. The figure documents the learnability gate used before interpreting reward components; controller ranking is addressed separately in the main text.

Table S2. One-out component ranking↩︎

Table S3. Top pairwise component interactions↩︎

Table S4. Simulator reward–actuation-proxy tradeoff↩︎

The proxy axis is a normalized residual/action proxy. The component-reward weight 0.5 point has the highest reward in this sweep, showing how the audit can track whether reward changes are accompanied by changed simulator action burden. Physical resource accounting would require matched energy, lamp, and CO₂ flow measurements.

Table S5. Policy-distillation fidelity↩︎

Table S6. Component-comparability audit↩︎

Supplementary Figure S2. Original component-gap artifact↩︎

This plot is retained as an audit artifact. The original comparison mixed timestep-summed simulator values with daily aggregate real-log scores, so it is used to document the aggregation mismatch that motivated the component-comparability audit.

Scope of supplementary diagnostics↩︎

These tables support simulator-side reward decomposition, logged-data component scoring, and component-comparability auditing. They provide detailed evidence for reward design and calibration planning, including which quantities would need matched-scale resource accounting or prospective greenhouse trials in a later validation study.

Table S7. Calibration-first AGC2 gates↩︎

Supplementary Figure S3. AGC2 calibration deltas across unique contexts↩︎

Positive deltas indicate that the facility-adapted GreenLight candidate is closer to logged AGC2 indoor climate than the default GreenLight simulator. The four rows are unique team/window contexts; preprocessing variants that produced identical action tables are not counted as independent facility traces. Objective, relative-humidity, and CO₂ RMSE improve in all four contexts, while temperature RMSE improves in two contexts and remains within the no-worse-than-\(1^\circ\)C guard in the two boundary contexts. This figure is included as anti-cherry-picking support for the AGC2 calibration gate, not as evidence of generalized facility adaptation.

Table S8. Calibrated-candidate PPO learnability↩︎

Table S9. 1M-step PPO budget-sensitivity↩︎

This matched-seed 1M-step check reports learnability and budget-sensitivity in the AGC2-adapted GreenLight-Gym environment. All scalar and component PPO runs clear the adapted rule-controller floor without early termination. The component-reward runs are directionally ahead at 1M steps, but the margin is small and simulator-only; stronger claims about reward dominance, policy transfer, deployment readiness, or measured resource savings require prospective evidence beyond this table. The compact source table is provided as e2_1m_budget_sensitivity.csv.

Table S10. Calibrated-candidate rule-fidelity summary↩︎

Table S11. Calibrated rule-fidelity depth sensitivity↩︎

Depth sensitivity was recomputed from the same six calibrated trace samples used for Table S10, using local CART surrogates with minimum leaf size 50 and the same feature and actuator target columns. It is included only as a stability check for the tree-depth choice: both reward designs remain highly rule-approximable from depth 2 to 5, and component–scalar differences remain small and channel-mixed after facility adaptation. The compact source tables are provided alongside this supplement as the depth-sensitivity summary, delta, macro, and raw CSV files.

Table S12. CO₂ time-conditioned seed replication↩︎

The time-conditioned CO₂ replay supports a narrow simulator-side statement: the tomato component reward increased the day/night CO₂ contrast in the two available saved-policy time-conditioned replays. It does not support a three-seed radiation-bin mechanism claim. Seed 123 is available in the start-day action-bin audit, but that audit explicitly bins only by evaluation start day and cannot support daylight-gated CO₂ timing. Compact source files are provided alongside this supplement as CO₂ seed-replication summary and radiation-bin CSV files.

Table S13. Real-log bridge permutation audit↩︎

The exact permutation check was added because the AGC bridge uses small team-level samples. It confirms that the strongest logged-data rows are best treated as directional calibration signals. These rows support calibration-target discovery and component-prior auditing; prospective greenhouse trials would be the next step for confirmatory reward validation and policy-transfer assessment. The complete permutation table is included in the package CSV and JSON audit files.

Table S14. AGC2 selected parameters at effective search bounds↩︎

Boundary-selected values are reported as transparency evidence for the AGC2 facility-adaptation search. They should be interpreted as a compact coefficient set that improves replay fidelity under the stated gate, not as uniquely identifiable greenhouse equipment estimates. The full compact source table is provided in the SAT experiment-closure parameter-boundary CSV.

Tables S7–S14 support the calibration-first manuscript update. They establish the supplement-level evidence for simulator adaptation, component-comparability, learnability, rule-fidelity, and time-conditioned action diagnostics, and they define the measurements needed for later prospective greenhouse-trial evaluation.

References↩︎