This work has been accepted to the 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2026). © 2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses.

1 Introduction↩︎

Autonomous navigation is a fundamental capability for mobile robots operating in agricultural environments, enabling tasks such as crop monitoring, pest detection, precision spraying, and harvesting. Over the past decade, visual navigation has been extensively studied in the agricultural robotics community to allow robots to perceive their surroundings and traverse diverse crop fields safely and reliably [1], [2]. Despite these advances, most existing systems have been developed and evaluated under daytime conditions. In contrast, nighttime navigation has been widely investigated in general on-road and off-road environments [3], [4], while it remains relatively underexplored in agricultural settings.

In fact, nighttime deployment of mobile robots can expand the operational bandwidth of agricultural automation [5], enabling more continuous and efficient food production. For example, extended operational availability would allow robots to complete harvesting within a narrow time window ensuring optimal fruit quality, even if their harvesting speed is slower than that of human workers [6]. Furthermore, cooler nighttime temperatures can help preserve the firmness and post-harvest quality of soft fruits [7]. Nighttime navigation would also enable robots to continuously monitor fields, detect nocturnal pests, and perform timely interventions.

A key challenge for developing nighttime agricultural navigation systems is, however, the limited availability of annotated nighttime data, as state-of-the-art learning-based approaches require large volumes of labeled training data. Existing datasets [1], [2] primarily consist of daytime RGB images and thus do not adequately represent nighttime field conditions. Collecting nighttime data is particularly challenging in agricultural fields due to limited visual feedback for operators and heightened safety concerns, while reduced illumination further complicates annotation by requiring additional effort for precise object identification.

To address this limitation, we propose an unsupervised image translation framework that converts daytime plant-row RGB images into realistic near-infrared (NIR) nighttime counterparts without requiring pixel-to-pixel supervision. As shown in 1, this enables the direct reuse of daytime semantic labels for training nighttime perception models. The framework incorporates a pre-trained “Contrastive Language-Image Pretraining” (CLIP) model [8] to preserve semantic and structural consistency during translation. Additionally, a visibility mask is introduced to account for the limited effective range of NIR illumination in nighttime scenes. For evaluation, we train semantic segmentation models only with the translated images and validate them on real nighttime images, aiming to integrate the resulting models into a navigation controller. We further demonstrate their effectiveness through physical robot experiments under nighttime conditions.

We utilize AgriNight—a novel dataset consisting of \(428\) daytime RGB and \(549\) nighttime NIR images collected using our night-vision-equipped mobile robot from strawberry and carrot fields and manually annotated with pixel-wise semantic labels—and introduce it as the first benchmark for nighttime agricultural visual navigation. In particular, we adopt \(850\) nm NIR illumination for nighttime imaging rather than visible light, considering the following advantages. First, it lies beyond plant photoreceptors’ sensing range [9], minimizing disruption to growth [10]. Second, unlike visible light, it attracts fewer nocturnal insects that could disrupt on-board perception. Third, it yields high soil-vegetation contrast, which is critical for navigation, since chlorophyll-bearing foliage reflects more strongly than soil [11].

Our contributions are summarized as follows:

• A CLIP- and visibility mask-enhanced unsupervised image translation framework for cross-modal (RGB-to-NIR) day-to-night domain transfer in agricultural fields;

• Comprehensive evaluation demonstrating superior performance over state-of-the-art image translation methods, including real-world robot experiments; and

• The AgriNight dataset for benchmarking nighttime agricultural field navigation.

To the best of our knowledge, this work is the first to present a perception pipeline specifically designed for nighttime agricultural robot navigation, along with a day/night dataset.

2 Related Works↩︎

2.1 Unsupervised Cross-Modal Image Translation↩︎

Image translation between RGB imagery and other sensing modalities (e.g., thermal infrared (TIR)) has been extensively studied to leverage abundant RGB annotations for training models in target modalities where labeled data are scarce [12], [13]. In particular, unsupervised image translation approaches do not require paired images captured from the source and target modalities of the same scene. For instance, SSL-RGB2IR [13], similar to [12], performs unsupervised learning to convert RGB images into corresponding TIR images. These pseudo RGB-TIR pairs are then subsequently used to train a refined image translator in a supervised manner. This approach enables the supervised learning of downstream perception tasks, such as object detection and semantic segmentation, as demontrated in urban driving scenarios by using the translated TIR imagery.

While our downstream task also involves semantic segmentation, several key distinctions set our work apart: 1) we jointly address cross-modal (i.e., RGB to NIR) and “cross-illumination” (i.e., day-to-night) translation to enable nighttime agricultural operation; 2) our focus is on relatively unstructured agricultural environments—characterized with non-uniform crop spacing and diverse plant morphologies—rather than structured urban scenes (e.g., vehicles, buildings); and 3) we validate the framework through a physical mobile robot operating in crop fields at night.

2.2 Unsupervised Domain Adaptation in Agriculture↩︎

In agricultural applications, unsupervised domain adaptation (UDA) is a crucial capability for adapting perception models to novel environmental conditions, such as varying soil types, crop growth stages, and lighting. Previous works [14], [15] performed unsupervised domain transfer from one field to another to enable semantic segmentation of crops, weeds, and soil without requiring target-domain labels. In particular, the framework in [14] is designed to enhance CycleGAN-based generative models by jointly training a semantic segmentation network to preserve semantic consistency during translation. However, such joint optimization may lead to unstable supervision, particularly in early training stages when segmentation predictions are unreliable. To mitigate this issue, our framework instead leverages a fixed, pretrained CLIP model [8] to compute a semantic preservation loss in a shared embedding space.

In addition, diverse agricultural contexts have been considered for UDA, including harvesting [16], leaf localization [17], growth stage prediction [18], and plant row detection [19]. These studies, however, omitted physical robot demonstrations and primarily focused on a single data modality (i.e., RGB imagery) for daytime scenarios. In contrast, our work addresses cross-modal translation (RGB to NIR) and domain adaptation (Day to Night) simultaneously, validated also through physical robot deployment in the field.

2.3 Visual Navigation in Low-light Condition↩︎

While visual navigation under low-light conditions has been investigated extensively in urban streets (e.g., illumination enhancement [3] and day-to-night translation [20], [21]), agricultural research remains underexplored. The M2P2 dataset, collected in forested environments [4], represents a step toward off-road low-light navigation. However, its primary focus lies in utilizing multi-modal sensory inputs, such as stereo RGB, event, and thermal cameras, along with high-resolution LiDAR, to supply high-throughput geometric information to the robot controller. Our approach instead aims to enable robust navigation in dark agricultural fields using only a single NIR night-vision camera.

2.4 Nighttime Operation in Agricultural Robotics↩︎

Most existing nighttime perception approaches in agriculture have relied on RGB sensors paired with high-intensity light sources from bulky external rigs to ensure target objects remain visible within the RGB spectrum [22], [23] As an alternative, researchers in [24] employed TIR imagery with depth estimation, mitigating the effects of varying lighting conditions. Still, all these studies were predominantly conducted in the context of nighttime fruit harvesting robots.

In contrast, our work addresses autonomous navigation in agricultural fields [1], [2] by leveraging NIR night-vision sensors. As an alternative to TIR cameras, NIR cameras offer a low-cost, high-resolution solution, preserving fine texture and edges. Recently, [25] deployed NIR cameras mounted on a mobile robot platform in agricultural fields at night. However, their focus was on classifying terrain conditions while the operator manually controlled the robot.

3 Unsupervised Day-to-Night Cross-Modal Image Translation↩︎

Our primary goal is to learn a mapping \(G: X \rightarrow Y\) that translates daytime RGB images \(x\in X\) into nighttime NIR images \(y\in Y\) without requiring paired examples of the same scene. The semantic labels available in \(X\) are subsequently leveraged to train a semantic segmentation model \(\mathcal{S}\) using the translated images \(y\) (cf. 1), enabling robust nighttime operation. As in [14], our framework builds upon CycleGAN [26], utilizing an adversarial setting and cycle consistency constraints. To ensure semantic preservation in translated imagery, we integrate pre-trained CLIP image encoders [8]. “Visibility” masks are also introduced to simulate the restricted effective sensing range of NIR night-vision cameras. We denote the daytime and nighttime data distributions as \(x \sim p_X\) and \(y \sim p_Y\), respectively.

3.1 Adversarial Training & Masked Cycle Consistency↩︎

In the adversarial setting [27], a discriminator \(D_Y\) is trained to distinguish translated images \(G(x)\) from real nighttime images \(y\). Simultaneously, the generator \(G\) is optimized to produce realistic NIR images to deceive \(D_Y\). Inspired by Pix2Pix [28], the discriminator \(D_Y\) and generator \(G\) are trained to minimize the following losses: \[\begin{align} \mathcal{L}_{D_Y} = \mathbb{E}_{y \sim p_Y} \Big[ \frac{1}{HW}\|D_Y(y) - \mathbf{1}\|^2_F \Big] \\ + \mathbb{E}_{x \sim p_X} \Big[ \frac{1}{HW}\|D_Y(G(x))\|^2_F \Big], \end{align} \label{eq:adversarial1}\tag{1}\] and \[\mathcal{L}_{G} = \mathbb{E}_{x \sim p_X} \Big[ \frac{1}{HW}\|D_Y(G(x)) - \mathbf{1}\|^2_F \Big], \label{eq:adversarial2}\tag{2}\] where \(D_Y(\cdot) \in \mathbb{R}^{H \times W}\) denotes the patch-level output from the discriminator, \(\mathbf{1}\) represents an \(H \times W\) matrix of ones, and \(\|\cdot\|_F\) denotes the Frobenius norm. To be specific, the discriminator \(D_Y\) evaluates realism over \(HW\) subpatches, assigning values close to one for realistic patches and close to zero for fake ones, while the generator \(G\) aims to produce images that fool the discriminator into classifying generated patches as real. Following LSGAN [29], we also adopt the least-squares (Frobenius norm-based) adversarial loss to alleviate the gradient vanishing problem during training.

To add a cycle consistency constraint, a secondary generator \(F: Y \rightarrow X\) and discriminator \(D_X\) are also trained in parallel to handle the inverse mapping (night to day), using adversarial losses \(\mathcal{L}_{D_X}\) and \(\mathcal{L}_{F}\) analogous to Eq.@eq:eq:adversarial1 and 2 , respectively. Both generators \(G\) and \(F\) are then trained to ensure an image translated through either \(X \rightarrow Y \rightarrow X\) (cf. 2 (a)) or \(Y \rightarrow X \rightarrow Y\) remains similar to its original input. To achieve this, \(G\) and \(F\) are optimized to minimize the cycle consistency loss [26]: \[\mathcal{L}_{cyc} = \mathbb{E}_{x\sim p_X}[\| F(G(x)) - x\|_1] + \mathbb{E}_{y\sim p_Y}[\| G(F(y)) - y\|_1].\]

As shown in 3, however, NIR night-vision sensors operating near an \(850\) nm wavelength provide limited visibility, depending on the intensity and beam divergence of the infrared illumination [30]. Consequently, distance regions in NIR nighttime images may lack sufficient information to reconstruct a full daytime scene. To account for this, we apply binary “visibility” masks (\(m\) and \(m'\)) to the cycle consistency loss, ignoring pixels lying outside the sensor’s reliable effective range in nighttime images. Specifically, the visibility mask \(m\) is created based on translated nighttime images \(G(x)\), while another mask \(m'\) is based on real nighttime images \(y\). The resulting masked cycle consistency loss is defined as: \[\begin{align} \mathcal{L'}_{cyc} = \mathbb{E}_{x\sim p_X}[\| m \odot F(G(x)) - m \odot x\|_1] \\ + \mathbb{E}_{y\sim p_Y}[\| m'\odot G(F(y)) - y\|_1], \end{align} \label{eq:cycle95consistency2}\tag{3}\] where \(\odot\) denotes element-wise multiplication. Conceptually, this approach is similar to Mask CycleGAN [31]. However, our objective is not to stitch a partially style-transferred image back into the original image.

3.1.1 Visibility Mask Generation↩︎

Each visibility mask (\(m, m' \in \{0,1\}^{H'\times W'}\)) is a binary matrix with the same spatial dimensions as the corresponding image to which it is applied, where \(H'\) and \(W'\) denote the image height and width, respectively. To simulate the limited effective range of the NIR illumination, we set the “visible” region based on pixel intensity.

Given a nighttime image (either a translated image \(G(x)\) for \(m\) or a real image \(y\) for \(m'\)), we apply a heuristic algorithm that scans each column \(c_i\) to identify the uppermost row—i.e., the minimum row index \(r_j\)—where the intensity exceeds a predefined threshold \(\tau\), representing the boundary of the illuminated, visible region. We then define the mask elements \(b(r, c_i)\) to be \(0\) if \(r<r_j\) and \(1\) otherwise. This process is repeated for all columns to generate the final mask.

Though the estimated mask may be imperfect during early training epochs when the translated image \(G(x)\) is still unstable, we observe that the inferred visibility boundary becomes gradually more realistic as adversarial training converges. By incorporating this spatial constraint into the loss function, optimization is focused on regions where reliable NIR observations are available, avoiding penalization for distant areas where the night-vision sensor lacks valid signals. 5 also discusses alternative designs.

3.2 Semantic Consistency Enhancement↩︎

To preserve the underlying semantic structure during translations, we utilize a fixed, pre-trained CLIP image encoder \(\mathcal{E}\) [8]. As illustrated in 2 (b), a cosine-similarity loss is computed in the shared CLIP embedding space to encourage the generators \(G\) and \(F\) to create translated images with semantic consistency with the original real images \(x\) or \(y\). In this loss calculation, specifically, each input image is first converted to grayscale to reduce the influence of color information. The generators \(F\) and \(G\) are then optimized to minimize the semantic consistency loss below: \[\begin{align} \mathcal{L}_{sem} = &\quad \mathbb{E}_{x\sim p_X} \Bigl[1 - \cos_\mathcal{E} \Bigl( m \odot G(x), m \odot x \Bigr) \Bigr] \\ &\quad + \mathbb{E}_{x\sim p_X} \Bigl[1 - \cos_\mathcal{E}\Bigl( m \odot F(G(x)), m \odot x \Bigr) \Bigr] \\ &\quad + \mathbb{E}_{y\sim p_Y} \Bigl[1 - \cos_\mathcal{E}\Bigl( m' \odot F(y), y \Bigr) \Bigr] \\ &\quad + \mathbb{E}_{y\sim p_Y} \Bigl[1 - \cos_\mathcal{E}\Bigl( m'\odot G(F(y)), y \Bigr) \Bigr],\\ \end{align} \label{eq:semantic95consistency}\tag{4}\] where \(\cos_\mathcal{E} (a, b)\) represents the cosine similarity between the embeddings \(\mathcal{E}(a')\) and \(\mathcal{E}(b')\), where \(a'\) and \(b'\) are the grayscale versions of \(a\) and \(b\), respectively. Note here that the visibility binary masks (\(m\) and \(m'\)) are also applied.

Finally, the ultimate objective function is: \[\begin{align} \mathcal{L} = \lambda_{GAN}(\mathcal{L}_{D_Y} + \mathcal{L}_{G} + \mathcal{L}_{D_X} + \mathcal{L}_{F}) \\ + \lambda_{cyc}\mathcal{L'}_{cyc} + \lambda_{sem}\mathcal{L}_{sem} + \lambda_{id}\mathcal{L}_{id}, \end{align}\] where \(\mathcal{L}_{id} = \mathbb{E}_{y\sim p_Y}\|G(y) - y\|_1 + \mathbb{E}_{x\sim p_X}\|F(x) - x\|_1\)—i.e., the identity mapping loss to regularize the generators by ensuring target-domain samples are preserved when passed through their respective generators [26]. The hyperparameters \(\lambda\) control the relative weight of each loss component.

4 Experiments↩︎

4.1 AgriNight Dataset↩︎

As summarized in 1, the AgriNight dataset contains \(428\) daytime RGB and \(549\) nighttime NIR images with pixel-wise semantic labels across three classes: Traversable (inter-row space), Non-Traversable (crop rows and plant regions), and Other. The pixel distribution for each class is reported in 2, where Traversable regions occupy the smallest proportion of pixels. Note that due to limited visibility during nighttime (cf. 3), the proportion of the Other class increases, while the proportions of the Traversable and Non-Traversable classes decrease.

Data were collected through multiple sessions conducted during daytime and nighttime conditions across two large-scale commercial strawberry farms in Georgia, USA, and one carrot field located at the KSU Field Station—a research farm operated by Kennesaw State University in the same state. 3 shows example images and labels.

4.1.1 Strawberry Fields↩︎

In Strawberry Farm A, both daytime and nighttime images were collected from five crop rows, each approximately \(40\) meters in length. In Strawberry Farm B, data were sampled from nine crop rows, each approximately \(70\) m long and \(0.8\) m wide. As shown in 3 (a) and 3 (c), the fields in Farm A were predominantly flat, while crop rows in Farm B exhibited a shallow longitudinal depression centered along each row. This structural difference introduced visual variability to the dataset, which may improve model generalization to novel field environments. In both fields, natural weeds were frequently observed within crop rows and along inter-row regions. All strawberry fields were cultivated using plastic-mulched raised beds, a common plasticulture system in commercial production.

During each session, a Rover Robotics \(4\)WD Rover Zero \(3\) platform was manually operated, with the operator relying solely on natural moonlight for visibility. To ensure diverse visual observations, the robot was driven along crop rows using both straight and zigzag trajectories. The platform was equipped with an Arducam night-vision camera featuring six integrated \(850\) nm NIR LEDs and a switchable infrared-cut filter that automatically toggles based on light conditions. The camera was mounted approximately \(33\) cm above the ground at the front of the robot to capture both daytime and nighttime images at a resolution of \(1280 \times 720\) pixels.

A total of \(331\) daytime and \(370\) nighttime images were collected over two days and two nights in mid-May during the harvesting season, when strawberry plants were mature, densely foliated, and bearing fruit. Daytime sessions were conducted under clear weather conditions. Nighttime data collection was performed after sunset (9PM to midnight) to capture realistic low-light field environments.

4.1.2 Carrot Fields↩︎

The carrot field was located in a greenhouse environment, where six crop rows were covered, each extending approximately \(24\) m in length and \(0.6\) m in width. As shown in 1, a total of \(97\) daytime and \(179\) nighttime images were collected in late February, approximately two months after planting. At the time of data collection, each carrot plant was approximately \(19\) cm tall.

We utilized a similar robot and camera setting to that used in the strawberry fields. To better capture these relatively small plants, however, the camera was mounted at a lower height, approximately \(22\) cm above ground level, along with an Andoer portable LED, featuring \(49\) NIR lamps, installed above the camera to provide additional active illumination during nighttime sessions. Both daytime and nighttime images were captured at a resolution of \(640 \times 480\) pixels.

4.2 Implementation Details↩︎

Inspired by [14], each of our generators (i.e., \(G\) and \(F\)) follows an encoder-transformer-decoder architecture. To process an input image of size \(256 \times 256\), the encoder utilizes a \(2\)D convolutional layer with \(64\) filters of size \(7\times 7\), followed by a second \(2\)D convolutional layer with \(128\) filters of size \(3 \times 3\) and stride \(2\) for spatial downsampling. The transformer module consists of \(16\) sequential residual blocks, each equipped with instance normalization layers instead of batch normalization and without additional spatial downsampling. The decoder employs a \(2\)D transposed convolution layer with \(64\) filters of size \(3\times 3\) kernels and stride \(2\) to restore the original spatial resolution through learnable upsampling, followed by a \(2\)D convolutional layer with three \(7 \times 7\) filters to produce a three-channel output image.

Each discriminator (\(D_X\) and \(D_Y\)) comprises five \(2\)D convolutional layers, with the first four using \(64, 128, 256\) and \(512\) filters of size \(4 \times 4\) with stride \(2\), respectively. Each layer applies instance normalization. The final layer uses a \(4 \times 4\) convolution to produce a single-channel \(16 \times 16\) subpatch prediction map, which is used for the computation of \(\mathcal{L}_{GAN}\). During training, standard image augmentation methods such as random cropping and horizontal flip were performed with a probability of \(25\%\).

To utilize the embedding space from a pretrained CLIP model, we utilized the ViT-B/\(32\) Transformer image encoder². Additionally, a semantic segmentation model was designed with a ResNet-\(18\) backbone [32] as the encoder, followed by a decoder with symmetric upsampling layers and U-Net–style skip connections [14]. The model performs pixel-wise classification over three semantic classes: Traversable, Non-Traversable, and Other. Training is conducted using the standard cross-entropy loss for multi-class segmentation [33]. The model was trained for \(100\) epochs, and we selected the checkpoint that achieves the highest mean Intersection-over-Union (mIoU) for evaluation. During training, data augmentation methods, including random cropping, horizontal flipping, Gaussian blurring, and color jittering, were applied with a probability of \(25\sim50\%\).

All training was conducted with a single NVIDIA RTX A5000 GPU (\(24\) GB VRAM). Our image translation model was trained for \(500\) epochs with a learning rate of \(10^{-5}\), and the checkpoint achieving the lowest FID score—designed to assess overall image quality [20]—was selected for evaluation. The loss weights were set to \(\lambda_{cyc}=10\), \(\lambda_{GAN}=1\), \(\lambda_{id}=5\), and \(\lambda_{sem}=1\) based on empirical validation.

4.3 Evaluation Protocols↩︎

For evaluation, we focus on the downstream nighttime semantic segmentation task, which is a critical component for achieving reliable nighttime visual navigation. As shown in 1, the segmentation model is trained exclusively on translated nighttime images and masked daytime annotations, and validated on real nighttime images. In particular, the visibility mask \(m\) (cf. 3.1.1) is applied to the daytime annotations to emulate reduced nighttime visibility. Following prior work [33], [34], we report mIoU and average pixel-wise accuracy as the primary evaluation metrics. In addition, we compute precision and recall [15] for each class.

We primarily utilize data from the Strawberry Fields dataset (4.1) due to its relatively large volume, while reserving the Carrot Field dataset for physical robot experiments. Specifically, the nighttime NIR images from the Strawberry Fields dataset are organized into three predefined splits, each containing mutually exclusive Train (\(247\) images) and Val (\(123\) images) subsets, enabling \(3\)-fold cross-validation. In each fold, all daytime images together with the nighttime images from Train are used for unsupervised training of the day-to-night image translation model. The translated nighttime images from all daytime images and their masked semantic labels are then used to train the downstream segmentation model (1). The trained segmentation model is subsequently evaluated on Val.

To prevent temporal leakage between Train and Val subsets, image frames collected from each farm are first chronologically ordered according to sampling time and then partitioned into three equally sized sequential segments. The first split then assigns the first temporal segment from each farm to the Val set, while the remaining two segments are used for Train. The other splits are constructed by rotating the validation segment accordingly. As a result, we report the average performance across the three distinct Val sets.

4.4 Comparative Evaluations↩︎

The proposed framework is compared against representative image-to-image translation and domain adaptation models discussed in 2:

• UoB [14]: An unsupervised domain adaptation framework developed at the University of Bonn for crop, weed, and soil semantic segmentation in unseen fields.

• SSL-RGB2IR [13]: A RGB-to-TIR image translation framework designed to support object detection and semantic segmentation in urban environments.

• CycleGAN-Turbo [20]: A CycleGAN-style framework that leverages a pretrained Stable Diffusion-Turbo [35] backbone as the generator for image translation.

• Real Nighttime: The segmentation model is trained using annotated real nighttime NIR images in Train to demonstrate the achievable upper-bound performance.

Both SSL-RGB2IR and CycleGAN-Turbo were trained and validated using their official implementations with task-specific modifications. In contrast, UoB was reimplemented based on the published description due to the absence of publicly released code. The primary modification in our implementation is the use of nine residual blocks, instead of the six, to increase model capacity for cross-modal translation.

For a fair comparison, the hyperparameters of each baseline model (e.g., learning rates and loss weights) were tuned to minimize the FID score [20]. All adopted hyperparameters, as well as the text prompts used for CycleGAN-Turbo, are available in the code repository. In addition, the FID score was used to select the best-performing checkpoint during training. Similarly, for the downstream segmentation task, the checkpoint achieving the highest mIoU on translated nighttime images was selected for evaluation. The final configurations are documented in our code repository.

4.4.1 Result Analysis↩︎

¿tbl:tab:comparison? presents the three-fold cross validation results of the semantic segmentation task evaluated on real nighttime data from the AgriNight Strawberry Fields dataset. The proposed model with \(16\) residual blocks achieves the highest performance across most evaluation metrics, compared to all baseline methods. Moreover, it exhibits lower standard deviations across different Val sets, indicating more consistent performance. This result suggests that the translated nighttime images used for training effectively preserve structural and visual characteristics of real nighttime NIR data. Although the model achieves approximately \(9\%\) lower mIoU (\(4\%\) lower accuracy) than the upper-bound scenario trained directly on annotated real nighttime images, the results remain impressive given that training relied solely on translated images derived from daytime RGB inputs.

In particular, all methods tend to struggle with the Traversable class, likely due to its relatively small pixel proportion in the dataset (cf. 2), which limits effective learning. Additionally, visual ambiguities caused by weeds, debris, and irregular soil textures may further confuse the segmentation models. However, the performance gain of our proposed model is particularly pronounced for this class, which is critical for reliable robot navigation.

Based on mIoU and pixel-wise accuracy, UoB, which also employs a dedicated semantic consistency loss, produces higher-quality NIR translations than SSL-RGB2IR and CycleGAN-Turbo. Yet, it remains inferior to our model using nine residual blocks—the same number used in UoB. This indicates that integrating a pretrained CLIP model, rather than jointly training domain-specific segmentation models, contributes to producing higher-fidelity NIR translations.

Although CycleGAN-Turbo leverages pretrained Stable Diffusion Turbo, its poor performance and high variance imply that transferred knowledge from training on large-scale data does not generalize well to agricultural NIR imagery.

4.5 Qualitative Analysis↩︎

Given the daytime scene in 4 (a), the translated image (4 (c)) presents a well-structured nighttime appearance, leading to precise semantic segmentation with clear boundaries of the Traversable region (4 (e)). While soil textures and some nearby leaves are not rendered in detail, trees in the daytime background become invisible in 4 (c).

However, 4 (d) shows missing strawberry plants on the raised bed and artifacts in the left background, causing the segmentation model to misclassify weeds within the Traversable region as Non-Traversable plants and parts of the raised bed as Traversable in 4 (f). Using a larger CLIP model might offer better semantic alignment.

4.6 Ablation Study↩︎

¿tbl:tab:ablation? summarizes our ablation study on the impacts of two core components: visibility masking mechanism and CLIP-based semantic consistency constraint. The naïve model without either component yields the worst performance. Using a CLIP results in a more substantial improvement than applying visibility masking alone, highlighting the importance of semantic alignment in cross-modal translation.

Finally, combining both components achieves the best overall performance, validating the proposed design choice. The complementary effects of NIR-specific visual regularization and semantic guidance demonstrate significant improvement in day-to-night cross-modal image translation.

4.7 Physical Field Testing↩︎

The robot platform introduced in 4.1 was later deployed in the same carrot field used to acquire the Carrot Field dataset. The \(16\) residual block-model, pre-trained on the Strawberry Fields data, was further fine-tuned using all Carrot Field data to learn the RGB-to-NIR translation. The robot was equipped with an NVIDIA Jetson Orin for onboard control, running the trained segmentation model in real time.

For simplicity, the robot employed a “pure pursuit” controller to follow paths generated from the segmentation output, keeping the robot centered within the Traversable region. Our framework is, however, controller-agnostic and can be readily integrated with other controllers.

During autonomous navigation at a speed of \(0.2\) m/s, the onboard system processed \(14\) frames per second. Over a total travel distance of \(96\) m, the robot ran without any collisions. This experiment highlights the generalizability of our approach, as testing was conducted in previously unseen crop rows, and plants showed increased heights (\(25\) cm) and denser foliage than those in the training data. A visual demonstration is provided in the supplementary video.

5 Conclusion, Limitations, & Future Work↩︎

An unsupervised RGB-to-NIR translation framework is proposed for nighttime agricultural navigation. Built upon CycleGAN [26], it leverages a pre-trained CLIP model [8] for semantic consistency and a visibility mask for limited NIR illumination range. Segmentation models trained on translated images perform reliably on real nighttime data, demonstrating high-fidelity translation, and are deployed alongside a robot controller to enable autonomous nighttime navigation. The AgriNight dataset, comprising daytime/nighttime images with pixel-wise semantic labels, is also introduced.

However, crop-row switching [36], which is essential for fully autonomous field robots, is not addressed here, and mask realism warrants further investigation. Future work may explore learning-based masking, depth estimation for improved physical consistency, and larger CLIP models.

Acknowledgment↩︎

This work was partially supported by USDA-NIFA (2023-38640-39572) through the Southern SARE program (GS24-301) and by the Georgia Peanut Commission (KSU-1-26/26).

References↩︎

---

1. \(^{1}\)LaSER Lab, Kennesaw State University, Marietta, GA 30060, USA. rmamo@students.kennesaw.edu, tchoi@kennesaw.edu.
   \(^{2}\)Lincoln Institute for Agri-Food Technology, University of Lincoln, Lincoln LN2 2LG, UK. {ODeSilva, GCielniak}@lincoln.ac.uk.↩︎

2. https://huggingface.co/openai/clip-vit-base-patch32↩︎