1 Introduction↩︎

Person re-identification (ReID) [1]–[3] is a fundamental computer vision task that aims to recognize the same individual across non-overlapping camera views. The core scientific challenge lies in achieving accurate and robust cross-view matching under complex conditions such as background clutter, pose variations, occlusions, and illumination changes [4]. As a core component of intelligent surveillance systems, ReID facilitates the efficient retrieval of persons of interest from massive-scale video streams, thereby providing essential technical support for critical public safety applications such as urban security, traffic management, and societal governance. Early studies on person ReID primarily focused on the supervised learning paradigm within the visible spectrum, where high-quality RGB images with explicit identity labels were utilized to achieve cross-camera pedestrian matching through deep feature extraction and metric learning. Such approaches [5]–[10] have achieved remarkable success on standardized datasets such as Market-1501 [11]. However, such approaches suffer from substantial limitations when applied to complex real-world scenarios. In low-light environments or adverse weather conditions, insufficient reflection of visible light frequently leads to severe degradation in image quality, manifesting as blurring, color distortion, and the loss of contour information. These issues consequently compromise the stability and reliability of identity feature extraction.

In response to the aforementioned limitations, researchers have progressively extended person ReID from single-modal recognition to cross-modal and, subsequently, tri-spectral and multi-modal settings. This evolution reflects the increasing emphasis on exploiting heterogeneous information to address more realistic and complex retrieval scenarios. Fig. 1 summarizes this developmental trajectory, illustrating the progression of person ReID from single-modal recognition to cross-modal, tri-spectral, and multi-modal paradigms.

To provide a clearer understanding of this evolution, we organize existing person re-identification tasks according to their retrieval protocols and primary learning objectives, rather than merely the number of involved modalities. Specifically, cross-modal person re-identification performs identity retrieval between heterogeneous query and gallery domains. Its primary learning objective is heterogeneous modality alignment, which aims to reduce the modality gap while preserving identity consistency across different representations. Representative tasks include visible–infrared ReID (VI-ReID), text–image ReID (TI-ReID), and sketch–image ReID (Sketch-ReID). We also discuss NLOS-related ReID, particularly emerging signal-to-visual settings that associate wireless, radar, or sparse geometric observations with visual identity representations. These settings share the challenge of heterogeneous alignment.

Meanwhile, tri-spectral person re-identification has gradually developed into a dedicated research direction. Its distinction is not determined solely by the use of three sensing modalities. Existing tri-spectral methods place particular emphasis on spectral-aware representation learning by modeling the spectral characteristics, inter-spectral consistency, and complementary identity evidence across RGB, near-infrared (NIR), and thermal-infrared (TIR) observations. Accordingly, we review tri-spectral ReID in a dedicated section to summarize its methodological characteristics, technical challenges, and open research questions.

In contrast, multi-modal person re-identification jointly exploits multiple available information sources or flexible modality combinations for identity representation and retrieval. Its primary learning objective is multi-source information fusion, with an emphasis on exploiting modality complementarity while controlling cross-source redundancy. Representative settings include joint fusion of RGB and infrared observations, as well as more flexible frameworks that combine RGB, infrared, text, sketch, and color-pencil representations. Cross-modal, tri-spectral, and multi-modal ReID therefore represent different paradigms for exploiting heterogeneous modalities, distinguished primarily by their retrieval protocol and learning objectives while sharing the common goal of robust cross-modal identity representation. Based on these perspectives, Fig. 2 illustrates the taxonomy adopted throughout this survey.

The Re-ID task is increasingly responsive to complex perception demands, real-world deployment challenges, and multi-modal integration. Several surveys have focused on Re-ID and cross-modal Re-ID. D’Orazio and Cicirelli [12] provide an early systematic survey on person re-identification and tracking across non-overlapping camera views, emphasizing appearance-based techniques. Bedagkar et al. [13] categorize person re-identification into Contextual and Non-contextual Methods based on visual features. Satta et al. [14] offer a comprehensive review of methodologies for appearance descriptor construction. Zheng et al. [4] trace the evolution of person re-identification, differentiating image-based from video-based approaches and reviewing both handcrafted and deep learning methods. Leng et al. [15] present the first survey of open-world Re-ID, aligning closely with real-world applications. Ye et al. [16] categorize Re-ID systems into open-world and closed-world settings and survey closed-world Re-ID through deep feature representation, metric learning, and ranking optimization. Zheng et al. [17] provide an overview of VI-ReID research, comparing traditional Re-ID and VI-ReID. Chang et al. [18] categorize VI-ReID studies based on application scenarios. Nguyen et al. [19] delivers a review of domain shift challenges in three settings: Unsupervised Domain Adaptation ReID, Domain Generalizable ReID, and Lifelong ReID. Chen et al. [20] analyze recent Re-ID research through specific application scenarios, while Wang et al. [21] systematically classify heterogeneous person re-identification across various cross-modal contexts. Huang et al. [22] survey VI-ReID from four technical perspectives: modality-shared feature learning, modality-specific compensation, auxiliary information utilization, and data augmentation.

Through an examination of existing surveys, we find that existing reviews lack a comprehensive research landscape that spans diverse cross-modal and multi-modal tasks. In response, this paper presents a systematic overview covering three representative cross-modal tasks: VI-ReID, TI-ReID, Sketch-ReID and NLOS ReID, as well as tri-spectral scenarios (e.g., RGB-NIR-TIR) and multi-modal ReID. We analyze the technical evolution and key challenges across these settings, aiming to construct a more holistic research map that offers insights and guidance for future studies. We present in Table 1 a comparison between our survey and existing surveys, highlighting the differences in covered tasks, modalities, and research perspectives. In this work, we also establish a transformer-based baseline for VI-ReID. This baseline serves as a simple yet effective reference framework, offering insights into how Transformer architectures can be adapted to cross-modal person re-identification.

The major contributions of this article are outlined as follows:

• Comprehensive coverage of diverse scenarios. To the best of our knowledge, this is the first survey that systematically reviews cross-modal ReID across multiple scenarios, including visible–infrared , text-image , sketch-image and Non-Line-of-Sight settings, as well as multi-spectral and multi-modal extensions.

• Detailed analysis of representative methods. We provide an in-depth review of related studies in each subfield, discussing representative methods and datasets, thereby offering a structured understanding of the research landscape.

• Introduction of a Transformer-based baseline. We establish a simple yet effective Transformer-based framework tailored for VI-ReID, which serves as a reference model and highlights the potential of Transformer architectures in addressing cross-modal person re-identification.

2 Cross-modal Person Re-Identification↩︎

In contrast to conventional Re-Identification (ReID) methods that operate within a single modality, cross-modal person re-identification performs identity retrieval between heterogeneous query and gallery domains. Its central objective is to reduce the representation discrepancy induced by different sensing or descriptive modalities while preserving identity-discriminative information. This section systematically reviews these representative cross-modal ReID tasks, focusing on task definitions, dataset construction, key challenges, and notable methodologies.

2.1 Visible-Infrared ReID↩︎

Visible-Infrared person re-identification (VI-ReID) is a cross-modal task that aims to identify the same individual by matching images captured by visible and infrared cameras [24]. This task expands the scope of traditional single-modal person re-identification (ReID) by tackling the significant performance degradation that occurs under challenging lighting conditions, such as nighttime or low-illumination scenarios. The core challenge of visible-infrared person re-identification (VI-ReID) resides in mitigating the feature distribution shift induced by distinct imaging mechanisms: visible images deliver abundant color and texture information but are highly susceptible to degradation under low-light environments, while infrared images, which rely on thermal radiation, exhibit robust imaging performance under adverse illumination conditions yet lack fine-grained appearance details. Depending on whether the training process leverages cross-modal matching pairs and identity annotations, existing research on VI-ReID can be generally classified into supervised and unsupervised paradigms. In this section, we summarize the key challenges confronted by each paradigm and conduct a comprehensive review of representative methodologies.

2.1.1 Datasets↩︎

Public benchmark datasets specifically designed for VI-ReID have played a crucial role in advancing this field. The details of the datasets are summarized in Table 2.

SYSU-MM01 [24] is the first large-scale VI-ReID dataset, containing 491 identities with a total of 287,628 visible images and 15,792 infrared images captured by six cameras (labeled 1 to 6) across diverse scenes. It supports two evaluation modes: full search and indoor search. In full search mode, visible images from cameras 1, 2, 4, and 5 form the gallery, while infrared images from cameras 3 and 6 are used as the query set. In indoor search mode, only visible images from cameras 1 and 2 are included in the gallery, with the query set remaining unchanged.

RegDB [25] contains 412 identities, with each individual associated with 10 visible images and 10 infrared images captured in real-world daytime and nighttime environments. It includes two evaluation modes: Visible To Infrared, where visible images serve as the query set and infrared images as the gallery, and Infrared To Visible, where the roles are reversed.

LLCM (Large-scale Low-light Condition Multi-modal) [26] dataset is the first large scale VI-ReID benchmark specifically constructed for nighttime low-light scenarios commonly encountered in real-world surveillance. It is collected using a 9-camera network deployed in low-light environments and contains 46,767 images of 1,064 identities, with 30,921 images from 713 identities in the training set (16,946 VIS and 13,975 IR) and 13,909 images from 351 identities in the testing set (8,680 VIS and 7,166 IR). The dataset covers both daytime and nighttime conditions, various climates, and diverse clothing styles, while also including real-world challenges such as motion blur, pose variation, occlusion, and low resolution. Similar to the RegDB dataset, two evaluation modes are adopted, i.e., visible to infrared and infrared to visible.

Summary:These benchmarks provide complementary evaluation conditions rather than a single unified measure of VI-ReID robustness. SYSU-MM01 emphasizes scale and camera diversity, RegDB offers a more balanced and controlled bidirectional setting, and LLCM focuses on challenging low-light surveillance. To obtain a more reliable assessment of cross- environment generalization, future studies should report cross-dataset evaluation, camera- or environment-disjoint testing, and robustness under degraded or unavailable modalities, rather than relying solely on within-dataset Rank-1 accuracy and mAP.

2.1.2 Supervised Visible-Infrared Person ReID↩︎

Supervised Visible-Infrared Person Re-ID aims to develop discriminative, modality-invariant representations from labeled cross-modal identity annotations, facilitating accurate identity matching across diverse modalities. To tackle challenges encountered during various training stages, existing research can be classified into three main directions: data augmentation, feature learning, and metric learning. This section systematically reviews prominent techniques within each direction, with an overview of major approaches and representative studies presented in Fig. 3, and tabulate a comprehensive performance comparison of representative methods in Table 3.

Data Augmentation for cross-modal Robustness: In supervised VI-ReID, substantial modality discrepancies, limited training samples, and complex environmental conditions are major factors constraining model performance and generalization. To mitigate these challenges, data augmentation has been widely employed as an image-level optimization strategy. By expanding the sample distribution, increasing image diversity, and simulating complex scenarios, data augmentation enhances model robustness to modality variations and facilitates more effective cross-modal feature alignment and discrimination. Early augmentation approaches in VI-ReID primarily relied on basic image-level perturbations, such as random horizontal flipping, cropping, and erasing [27], aiming to improve generalization under limited training data. However, as the demand for higher cross-modal alignment accuracy and more discriminative feature learning has grown, these conventional augmentation strategies have proven insufficient. Consequently, researchers have proposed more targeted cross-modal augmentation methods to address these limitations.

Inspired by single-modal ReID, some studies employ generative adversarial network (GAN)-based methods to map visible and infrared images into a unified modality domain, thereby simplifying cross-modal matching into a single-modal problem. Kniaz et al. [28] developed Thermal-GAN, a color-to-thermal re-identification framework that models thermal appearances conditioned on visible inputs. Despite its effectiveness, direct visible-to-thermal translation loses critical channel cues, including textures and color distributions essential for discriminative learning. Zhong et al. [29] proposed a reverse modeling approach that generates pseudo-color images from infrared inputs (i.e., image colorization), making semantic and texture details in infrared data explicitly visible in the color space. Wang et al. [30] employed dual GANs for bidirectional mapping between visible and infrared modalities, embedding the translated results into a shared feature space to support cross-modal matching. Zhu et al. [31] synthesized new modality samples and jointly trained models with both synthetic and real images to enhance alignment. Qi et al. [32] further incorporated paired information from the complementary modality and introduced a part-based cross-modal contrastive loss to provide consistent supervision for modality transition. Conversely, Qian et al. [33] presented a pose-guided augmentation strategy that leverages attention mechanisms to generate pose-diverse, identity-consistent image pairs, thereby enriching the structural diversity of training data.

In contrast to methods that focus solely on modality translation, another research direction investigates the fusion of visual information from both visible and infrared modalities at the image level to create training samples with mixed-modality characteristics. This approach allows models to concurrently capture discriminative cues from both modalities during training. For example, Zhang et al. [34] develop a nonlinear intermediate-modality generator to synthesize images with unified intermediate features, which are employed alongside original images for training. Huang et al. [35] horizontally divide RGB and IR images into multiple local regions and apply linear interpolation within each region using a modality-adaptive mixing strategy. Qian et al. [36] partition visible and infrared images into small patches and reassemble them to create novel training samples that integrate both modalities.

7pt

GAN-based generative methods and image-level mixing strategies, though demonstrating effectiveness in alleviating modality discrepancies, both exhibit inherent limitations. The former often introduces artifacts, distortions, and uncontrollable noise, whereas the latter may struggle to preserve fine-grained structural consistency. These shortcomings can ultimately undermine the reliability of model learning.

Recent studies have explored more controllable and stable augmentation techniques. Among them, channel-level augmentation directly manipulates image color channels or spectral responses to suppress modality-specific color cues and guide the model to focus on more discriminative structural features. This approach offers a more robust solution for modality alignment while avoiding the artifacts commonly introduced by generative models.

In earlier work, Ye et al. [56] design the Homogeneous Augmented Tri-modal (HAT) framework, which converts visible images into grayscale images, introduces intermediate modality images combining visible and infrared styles, and jointly trains the three modalities to reduce modality discrepancy. Liu et al. [57] extend the grayscale idea but simplify the processing pipeline by directly replacing the original visible image with its grayscale counterpart for model training, thereby preserving structural features while weakening modality gaps. Zhang et al. [58] propose a staged strategy: in the first stage, auxiliary modality pre-training is conducted using grayscale-balanced images; in the second stage, original images are reintroduced to mitigate sensitivity to color features during model training. Subsequently, Ye et al. [59] propose an effective random channel exchange strategy, which disrupts the original color-channel relationships and reduces modality-specific correlations. Due to its simplicity and effectiveness, this method becomes a common augmentation technique in cross-modal ReID, as illustrated in the Fig. 3.

In addition, some studies explore various innovative approaches for channel-level data augmentation. Zhang et al. [60] map single-channel infrared images into pseudo-RGB three-channel format to achieve channel-wise dual semantic alignment: intra-modality consistency is enforced by guiding each channel to learn coherent semantics within the infrared modality, while inter-modality alignment is achieved by minimizing the distribution gap between corresponding channels (e.g., R-to-R) across visible and infrared modalities. From the perspective of spectral modeling, Fan et al. [61] propose a cross-spectrum image generation method that synthesizes multiple spectral styles (e.g., blue, red, yellow, and grayscale) to encourage the model to learn shared features under diverse spectral conditions, thereby enhancing cross-modal generalization. Tan et al. [62] further explore the local linear response of materials to spectral variations and propose a dual-stage augmentation strategy that combines global channel perturbation with localized random linear enhancement to simulate realistic modality shifts. Alehdaghi et al. [63] point out that the lack of explicit objective modeling in intermediate image generation limits its adaptability. To address this, they propose an adversarial training framework that not only generates modality-bridging intermediate images but also explicitly guides the model to focus on learning identity-discriminative features.

Feature Learning for Modality-Invariant Representation:

Motivated by the objective of isolating modality-related variations while preserving discriminative identity cues, feature disentanglement methods aim to explicitly separate modality-specific and modality-invariant factors, thus enabling more effective modality alignment. From a broader perspective, these methods address a fundamental challenge in VI-ReID: visible and infrared images contain both shared identity information and modality- dependent factors, but the two types of information are difficult to separate perfectly. Existing studies therefore explore different forms of factorization, including identity-related and identity-irrelevant information, camera-specific and camera-agnostic features, and appearance-related and structural representations.

Choi et al. [38] first introduce this idea into VI-ReID by disentangling identity- relevant factors (e.g., body shape, clothing) from identity-irrelevant ones (e.g., pose, illumination). Pu et al. [64] employ a Dual Gaussian-based Variational Auto-Encoder (DG-VAE) to divide features into identity-discriminative and identity-ambiguous subspaces. To mitigate intra-class discrepancies caused by style variations (e.g., background, exposure, color shifts) across cameras, Li et al. [65] explicitly decompose features into two branches: a camera-specific branch that learns background features tied to camera attributes, and a camera-agnostic branch that removes style interference while retaining primary identity information. Although these methods differ in their decomposition mechanisms, they share the same principle of preventing modality-specific variations from dominating the identity representation. Their main difference lies in the source of the nuisance factors being modeled: some methods focus on illumination and appearance, whereas others explicitly account for camera style or ambiguous identity information. This distinction is important because camera-related variations may remain within the same modality, while illumination and spectral differences are particularly prominent across modalities. Building on this concept, Zhang et al. [47] utilize disentangled identity features to generate missing-modality representations, training these in conjunction with original features. This approach, distinct from image-level generation, circumvents artifacts and structural distortions, thereby enhancing discriminability and stability in feature learning. This feature-level completion strategy also reveals an important evolution of disentanglement-based methods. Instead of using factorized features only to suppress irrelevant information, the learned identity-related representation is further used to compensate for missing modality information. In this way, feature disentanglement serves both as an alignment mechanism and as a basis for cross-modal feature reconstruction.

To address the over-reliance on body shape in cross-modal representation learning, Feng et al. [66] introduce a shape-erased feature learning paradigm that eliminates body- shape information, compelling the model to capture a broader range of modality-shared features, which results in improved performance in visible-infrared person re-identification. Huang et al. [67] expand upon the extraction of modality-shared appearance features by incorporating modality-invariant relational features among different body parts, thereby enhancing the robustness and generalization of cross-modal representations while maintaining discriminative appearance information. More recently, Zhou et al. [68] proposed a progressive discriminative feature learning strategy that implements varying alignment schemes at different stages to effectively extract discriminative features. These studies further suggest that modality-invariant information is not limited to global appearance or body shape. Relational structures among body parts and progressively learned semantic cues can also provide stable identity evidence. However, explicitly removing a potentially useful factor, such as body shape, may reduce identity discrimination for identities that are strongly characterized by body structure. Therefore, disentanglement methods need to balance the removal of modality-related bias with the preservation of identity-specific information.

Beyond feature disentanglement, another line of research improves modality- invariant representation by optimizing the spatial and relational structure of feature learning. Early studies primarily aim at learning global shared representations across modalities [24], [69]–[71]. Global features provide a holistic description of person images and capture modality-shared semantics. However, they are highly sensitive to background noise and non-target regions, which reduces the precision of identity representations. In challenging scenarios such as occlusion, complex backgrounds, or pose variations, global representations often fail to highlight the most discriminative regions, thereby limiting the robustness and discriminability of cross-modal matching. The limitation of global representation learning is therefore not simply insufficient feature capacity, but the difficulty of distinguishing identity-related regions from background and modality-dependent regions. This motivates local feature learning as a complementary strategy. To address these limitations, local feature learning is introduced as a complementary strategy, since it focuses on fine-grained and region-specific cues that are less affected by background interference. Within this direction, Ye et al. [42] propose a Dynamic Tri-level Relation Mining (DTRM) framework that models cross-modal semantic relations at three levels, namely channel, part, and graph structure, in order to achieve joint optimization. Zhu et al. [72] introduce the Two-Stream Local Feature Network (TSLFN) to emphasize fine- grained local feature learning. Zhang et al. [73] further integrate global and multi-scale local features, enabling complementary and mutually reinforced representations across modalities, which enhance both robustness and discriminability in cross-modal matching. The progression from global modeling to local and relational modeling reflects an increasing effort to capture identity cues at multiple spatial scales. DTRM emphasizes structured relations among feature elements, TSLFN focuses on fine-grained body regions, and the global-local fusion strategy combines holistic identity semantics with detailed local evidence. Nevertheless, local alignment may be affected by pose variation, occlusion, and inaccurate body-part correspondence, which limits the reliability of purely local matching.

Park et al. [43] establish pixel-level correspondences between RGB and IR images through a module called CMAlign, which avoids noisy matches caused by conventional coarse image-level alignment and guides the model to learn more discriminative identity features. Yang et al. [49] address the simultaneous presence of noisy identity annotations and incorrect cross-modal correspondences, collectively referred to as Twin Noisy Labels (TNL), by integrating sample cleanliness estimation and a pairwise correction mechanism. This approach effectively mitigates the negative impact of both annotation noise and matching noise. These methods show that improving feature quality also requires improving the reliability of cross-modal correspondence. Pixel-level alignment attempts to obtain more precise spatial matching, whereas TNL explicitly considers the fact that both identity labels and modality correspondences may be unreliable. Thus, the effectiveness of local and relational feature learning depends not only on the representation architecture but also on the quality of the supervision used to establish cross-modal relationships.

Additional research, such as Zhang et al. [74], proposes to fuse the fine- grained advantages of visible images with the stable characteristics of infrared images for joint representation learning. Based on the fused features, the model further performs refined enhancement by exploiting the complementary strengths of both modalities. These studies [46], [48], [75] realize modality confusion at the feature level by incorporating rich semantic information from both visible and infrared images. Unlike image-level generation, feature-level modality confusion directly constrains the representation used for retrieval. Its underlying assumption is that modality identity should be weakened in the shared feature space while identity information should remain discriminative. This reduces the risk of generation artifacts and avoids the additional computational cost of synthesizing images. However, overly strong modality confusion may also remove useful modality-specific cues, resulting in a trade-off between modality invariance and identity discrimination.

Compared with traditional GAN-based image generation methods, such feature-level confusion mechanisms avoid generation noise and additional computational overhead, thereby enabling more efficient learning of modality-invariant representations. Therefore, feature-level fusion and modality confusion can be viewed as complementary to feature disentanglement: disentanglement explicitly separates different factors, whereas modality confusion directly regularizes the shared representation to reduce modality predictability.

With the rise of Transformer architectures, their potential in VI-ReID has been increasingly investigated. Liang et al. [76] are the first to introduce Transformer networks into the cross-modal Re-ID task, leveraging their strong capability for global modeling to extract richer and more discriminative features. Subsequently, Zhao et al. [77] demonstrate that Transformers are not only effective in modeling global interactions among image patches, but also exhibit strong potential in capturing sequence correlations at both the pixel and channel levels. Accordingly, they propose a method that models long-range dependencies across spatial and channel dimensions, thereby enhancing the discriminative power of features in cross-modal person re-identification. The dual-stream Transformer architecture proposed in [78] addresses the limitation of CNNs in modeling global dependencies, enabling the capture of long-range dependencies in pedestrian images. These studies indicate that Transformer-based architectures provide a flexible way to integrate global context, local patch interactions, and cross-modal dependencies within a unified representation. However, the self-attention mechanism alone does not guarantee modality invariance. Without an appropriate modality-sharing or modality-confusion strategy, long-range modeling may also propagate modality-specific noise across the feature space. Thus, the key issue is not only whether Transformers can model global dependencies, but also how shared and modality-specific information should be controlled during this process.

Most methods for identity representation primarily focus on visual information, relying heavily on appearance features. However, this reliance renders models vulnerable to occlusion, pose variations, and background clutter, which impedes the extraction of identity-discriminative features. To mitigate these challenges, an increasing number of studies incorporate auxiliary information such as pose estimation, body keypoints, attribute annotations, or semantic segmentation to provide structural guidance for feature learning. The motivation of this direction is to introduce relatively stable structural or semantic cues that are less sensitive to illumination and spectral changes. Such auxiliary information does not replace visual appearance, but provides additional constraints for identifying body structure, local regions, and semantic attributes across modalities. For example, the Pose-guided Modality-invariant Feature Alignment method [79] uses human posture as a structural clue to uniformly align the features of visible light and infrared images; in addition, auxiliary posture estimation tasks [80] further verify the effectiveness of key point position information as a modality sharing clue in cross-modal ReID. On the other hand, Wei et al. [81] propose a method called FBP-AL (Flexible Body Partition with Adversarial Learning), which automatically partitions human body parts and integrates structure-assisted supervision with weighted feature fusion to guide the model in capturing identity-related local features more effectively. These pose- and body-structure-based methods are particularly useful when color and texture information becomes unreliable. Nevertheless, their performance may depend on the accuracy of pose estimation or body-part partitioning, which can be degraded by occlusion, low resolution, and imperfect infrared imaging. This illustrates the general trade-off of auxiliary information: stronger structural guidance can improve alignment, but inaccurate auxiliary cues may introduce additional noise.

In addition to pose information, researchers also explore other forms of auxiliary information. Wang et al. [82] employ the CLIP model with learnable bimodal text tags to encode modality-specific semantics in visible and infrared images and fuse them into a unified representation, thereby enhancing the modeling of modality complementarity. Yu et al. [83] propose a Semantic Margin-guided Feature Decoupling (SMFD) module that decomposes image features into identity-related and style-related components, with text embedding constraints imposed to strengthen the semantic consistency of the former and achieve more accurate cross-modal alignment. Lee et al. [84] develop a camera-aware progressive adaptation framework that leverages camera-specific labels of pedestrian images to incrementally transfer knowledge from the source domain to the target domain, thereby improving generalization across cameras and modalities. Jiang et al. [85] innovatively introduce natural language guidelines to learn both global and local contextual information for VI-ReID, uncovering additional latent cues. These methods extend auxiliary guidance from physical structure to semantic and contextual knowledge. Text-based methods provide high-level descriptions that can supplement incomplete visual evidence, whereas camera-aware methods explicitly model the domain factors responsible for feature shifts. Their common objective is to constrain the learned representation with information that is more stable or interpretable than raw appearance. At the same time, the dependence on pretrained models, semantic annotations, or camera labels may limit their applicability in settings where such information is unavailable.

In summary, the above studies address modality-invariant representation learning from complementary perspectives. Feature disentanglement separates identity-related factors from modality- or camera-specific variations; global-local and relational modeling improves the spatial precision of identity representations; feature-level modality confusion directly reduces the modality discrepancy in the embedding space; Transformer architectures enhance long-range dependency modeling; and auxiliary information provides additional structural or semantic constraints. These strategies are not mutually exclusive, but their combination requires a careful balance between modality invariance, identity discrimination, correspondence reliability, computational cost, and dependence on external priors.

Metric Learning for cross-modal Alignment: Alongside feature learning, metric learning also plays a vital role in VI-ReID by enforcing discriminative constraints in the embedding space, encouraging intra-class compactness and inter-class separability, and thereby enhancing identity discrimination across modalities. Traditional metric learning in ReID primarily relies on distance-based constraints between samples. Identity loss ensures correct classification of each instance, contrastive loss encourages cross-modal alignment by pulling intra-class samples closer and pushing inter-class samples farther apart, and triplet loss constructs anchor–positive–negative triplets to guarantee that positive samples of the same identity are closer to the anchor than negative samples of different identities. To further improve discriminability, advanced extensions such as quadruplet loss [86] , center loss [87] and hetero-center triplet loss [88], and cluster contrast loss [89] are also introduced. In addition to these classical losses, recent studies design novel loss functions tailored for cross-modal scenarios. For instance, Jia et al. [90] observe that existing works often overemphasize modality discrepancies while ignoring intra-modality similarities, and thus propose the Similarity Inference Metric (SIM) to leverage intra-modality similarity for alleviating the modality gap. Zhang et al. [91] introduce a hybrid constraint framework that jointly optimizes intra- and inter-modality feature distributions by enforcing both class-level and modality-level similarity constraints. Gao et al. [92] further present a novel approach that integrates Perceptual Edge Feature (PEF) loss with cross-modal Contrastive Center (CMCC) loss, enabling joint optimization to enhance feature discriminability. Kong et al. [93] propose Dynamic Center Aggregation Loss, which fuses mixed-modality information and dynamically adjusts center distances between different modalities, thereby achieving effective constraints on multimodal distribution relationships. Cai et al. [94] strengthen category aggregation and separation by learning category centers in visible and infrared modalities separately, and construct triplets based on a dynamic hard sample mining strategy. Subsequently, Cheng et al. [89] design a deep comparative learning framework with an aggregated memory mechanism to enhance inter-modal alignment by modeling cross-modal semantic consistency. Most recently, Kim et al. [95] introduce Positive/Negative Enhancement Loss, which expands the representation space of the same identity within a single modality to improve feature diversity and discrimination, thereby further promoting generalization across modalities.

2.1.3 Unsupervised Visible-Infrared Person ReID↩︎

Recent advancements in supervised visible-infrared person re-identification (VI-ReID) have achieved high recognition accuracy, demonstrating the effectiveness and maturity of cross-modal modeling. However, practical scenarios often involve infrared images captured under nighttime or low-light conditions, leading to low clarity and making manual annotation challenging. Additionally, the inherent costs of cross-modal pairing limit scalability across heterogeneous devices and diverse real-world environments, thereby constraining the generalizability and deployment feasibility of supervised methods. To mitigate dependence on labeled data and enhance real-world applicability, there is increasing focus on unsupervised visible-infrared person re-identification (US-VI-ReID). This approach aims to learn pedestrian representations that bridge the gap between visible and infrared modalities without the need for manually annotated identity labels, facilitating cross-modal image retrieval. By circumventing the costly and time-consuming process of cross-modal annotation, US-VI-ReID methods are better suited for deployment in large-scale, open scenarios and represent a crucial direction for advancing cross-modal ReID applications. We illustrate the pipeline for unsupervised VI-ReID in Fig. 4 and provide a comparative analysis of existing methods in Table 4.

Based on these challenges, existing studies mainly explore three key dimensions: (1) cross-modal invariant feature learning, which aims to directly model modality-shared identity features and reduce inter-modal discrepancies; (2) cross-modal pseudo-label generation and enhancement, which focus on producing more reliable pseudo-labels to guide feature learning while mitigating the adverse effects of noisy labels; and (3) cross-modal label association, which establishes identity correspondences between visible and infrared samples to provide cross-modal supervisory signals and enable the learning of robust identity representations.

Cross-modal Invariant Feature Learning: Modality-invariant feature learning centers on the core objective of extracting identity-discriminative yet modality-agnostic representations from heterogeneous inputs. The key challenge lies in reducing the distributional discrepancy between visible and infrared images in the feature space, while preserving the structural and semantic information necessary for person identification. Liang et al.  [96] pioneers the first unsupervised framework H2H, which employs a two-phase strategy. A homogeneous learning phase generating intra-modality pseudo-labels and a heterogeneous learning phase aligning cross-modal features to extract invariant representations. Yang et al. [97] advances this study by proposing the ADCA framework, which enhances feature invariance through augmented dual-contrastive learning and cross-modal memory aggregation. To tackle hierarchical discrepancies (intra-camera, inter-camera, and cross-modal variations) in cross-modal retrieval, the GUR framework [98] achieves discrepancy disentanglement and robust matching via hierarchical representation learning. More recently, SDCL [99] extends modality-invariant feature learning by jointly exploiting shallow and deep feature representations. Instead of relying solely on the final-layer representation, its shallow–deep collaborative learning strategy leverages complementary structural details and high-level semantic cues, leading to more robust modality-invariant representations for unsupervised VI-ReID.

Cross-modal pseudo-label generation and enhancement: In unsupervised visible-infrared person re-identification, pseudo labels function as surrogate supervisory signals, critically influencing the feature space structure, cross-modal alignment, and the stability of label propagation and matching. Consequently, substantial research has concentrated on the reliable generation and refinement of pseudo labels. Given that clustering underpins pseudo label generation, several approaches aim to enhance reliability by optimizing the clustering process, promoting feature discriminability and structural consistency to mitigate error accumulation during training. For instance, Lin et al. [100] propose a bottom-up clustering (BUC) method that initially treats each sample as an independent identity to maximize inter-identity diversity, subsequently merging similar samples to enhance intra-identity similarity. Similarly, CHCR  [101] introduces a hierarchical clustering strategy that constructs stable local structures within each modality and performs cluster-level alignment across modalities, progressively generating more reliable cross-modal positive pairs. Despite these advancements, existing methodologies often overlook the impact of intra-modality camera differences. Variations in appearance due to multi-camera setups (cam-style variation) can exacerbate feature shifts within a modality, resulting in inconsistent clustering. To address this challenge, several studies have integrated camera labels to facilitate more accurate clustering [65], [98], [102]–[105].

In traditional methods (e.g., [96], [97], [106], [107]), pseudo-label assignment typically relies on offline clustering, i.e., clustering is performed periodically on extracted features rather than being updated continuously during training. However, such static label associations are prone to error accumulation during training, which can hinder model optimization. To alleviate this issue, Yang et al. [108] proposes a collaborative pseudo-label refinement framework that integrates both offline and online mechanisms. Specifically, an online cross-modal feature memory module is introduced, comprising Instance Memory, Identity Memory, and Domain Memory. This dynamic update strategy mitigates the impact of noisy labels by smoothing label transitions and has inspired subsequent research. Shi et al. [109] further extends the memory mechanism by decomposing the conventional unified memory into multiple sub-memories, enabling multi-view feature representation. Traditional cross-modal clustering-based matching strategies adopt a one-to-one scheme, where each visible cluster is matched to a single infrared cluster. Given the limited information in infrared images, this often results in information asymmetry and insufficient matching. To address this issue, Cheng et al. [110] introduces an innovative Many-to-Many Bilateral Matching mechanism, which alleviates mismatches caused by inaccurate cluster boundaries.

Following the generation of initial pseudo-labels, several studies have focused on evaluating their posterior reliability and refining their utilization [111]–[113]. These methods typically assess pseudo-label confidence after clustering to distinguish clean samples from noisy ones, and adopt quality-aware training strategies accordingly, thereby enhancing the robustness of pseudo-supervision and improving overall training stability. Treating all hard samples—those distant from cluster centers—as noisy may lead to the misjudgment and discard of valuable supervisory signals. MIMR [114] argues that some hard samples are actually “ambiguous samples” located near the decision boundaries of multiple clusters, which can confuse model training. Therefore, it proposes to further categorize hard samples into “normal hard samples” and “ambiguous samples”, and explicitly remove the latter to enhance the accuracy of pseudo-label learning. Shi et al. [115] categorizes sample prototypes into three types—Centroid Prototype, Hard Prototype, and Dynamic Prototype—each serving different optimization objectives. A progressive contrastive learning strategy is employed: Centroid Prototypes are utilized in the early training stages to capture commonality, while Hard and Dynamic Prototypes are gradually incorporated later to model discriminative and diverse features.

In evaluating the importance and reliability of pseudo-labels, neighbor labels are often treated as a structural prior that provides auxiliary cues for local semantic consistency. Specifically, many studies [116], [117] leverage the neighborhood consistency assumption, which posits that if a sample’s predicted label is consistent with those of its neighbors in the feature space, it is more likely to be correct and reliable. Based on this principle, an increasing number of methods have introduced neighborhood structure modeling to assess pseudo-label confidence via neighbor label distributions and refine training strategies accordingly, thereby improving the stability and robustness of pseudo-supervised learning. In addition, Pang et al. [101] propose an inter-channel pseudo-label refinement strategy based on the assumption that all three channels of the same visible image should share a consistent pseudo-label. This approach filters noisy labels and enhances pseudo-label reliability. N-ULC [118] replaces hard pseudo labels in homogeneous and heterogeneous spaces with neighbor-derived soft labels and dynamically downweights unreliable samples, thereby treating label confidence as a continuous quantity.

Cross-modal Label Association: In US-VI-ReID, the development of cross-modal pseudo-labels is vital for modality alignment and feature learning. The PGM method [106] utilizes modality-specific clustering graphs and cost matrices for cross-modal graph matching, but its cluster-level labeling can lead to incorrect groupings and neglect fine-grained instance-level semantics. Ye et al. [119] integrate instance-level matching and outlier filtering. Conversely, OTLA [107] reformulates label assignment as an optimal transport problem, employing Sinkhorn optimization for semantic alignment and balanced label distribution, thus mitigating visible modality dominance. Cheng et al. [116] build upon this with a bidirectional transport mechanism that facilitates mutual projections of visible and infrared images into each other’s pseudo label space. Addressing the limitations of single memory modules for identity association, Shi et al. [109] propose a multi-memory framework to capture subtle identity differences. Additionally, Yang et al. [120] introduce a progressive learning paradigm that combines intra-modal adversarial learning, cross-modal neighbor expansion clustering, and modality-invariant contrastive optimization to refine pseudo labels and reduce noise propagation. Finally, Teng et al. [121] apply a bidirectional consistency rule to stabilize matching between modalities, minimizing early training noise accumulation. MCL [122] studies an unpaired setting in which visible and infrared identity populations only partially overlap, rather than assuming full cross-modal identity correspondence. It uses Cross-modality Feature Mapping to construct a pseudo-identity space and Static-Dynamic Collaborative learning to align cluster and instance-level correspondences under identity mismatch, this setting reduces implicit pairing supervision.

7pt

2.2 Text-Image ReID↩︎

In real-world surveillance and public security scenarios, complete visual information of the target individual is often unavailable as a query reference. In contrast, verbal descriptions provided by eyewitnesses—such as clothing attributes, body shape, and carried items—are more common and accessible. Therefore, how to accurately retrieve the corresponding pedestrian image from a large-scale gallery based solely on such textual information has become a research problem of both practical importance and urgency. This has led to the emergence of the Text-Image Person Re-Identification (TI-ReID). TI-ReID aims to retrieve target pedestrians by using natural language descriptions as queries under cross-modal conditions. Compared with conventional image-based ReID, TI-ReID encounters more severe cross-modal discrepancies. The text modality is typically abstract, flexible, and highly subjective in nature, while the image modality provides concrete, continuous, and structurally rich visual information. The inherent differences in representation form and semantic level between the two modalities give rise to a significant modality gap, posing considerable challenges for achieving effective cross-modal feature alignment and consistent identity matching. We illustrate the primary challenges encountered in TI-ReID in Fig. 5 and compare the performance of existing methods in Table 6.

2.2.1 Datasets↩︎

CUHK-PEDES [124] is a text-based person retrieval dataset proposed by the Chinese University of Hong Kong and is one of the most widely used benchmarks in the field of Text-Image ReID. It is constructed by integrating image resources from CUHK03 [125], Market-1501 [126], SSM [127], VIPeR [128], and CUHK01 [129], supplemented with natural language descriptions collected via the Amazon Mechanical Turk (AMT) platform. The dataset contains 40,206 images of 13,003 identities paired with 80,412 textual descriptions (each with no fewer than 23 words), and is divided into a training set (34,054 images, 11,003 identities) and validation/test sets (about 3,000 images and 6,100 descriptions associated with 1,000 identities, respectively). This dataset is characterized by rich semantic diversity, with descriptions covering various aspects such as appearance, posture, actions, and interactions with objects.

ICFG-PEDES [130] is a person description dataset constructed by South China University of Technology, designed to facilitate research in fine-grained person description and image-text matching. The dataset comprises 54,522 pedestrian images corresponding to 4,102 distinct identities, all collected from the MSMT17 [131] dataset. Each image is annotated with a single natural language description, with an average length of 37.2 words. The training set contains 34,674 image-text pairs associated with 3,102 identities, while the test set consists of 19,848 pairs covering 1,000 identities. Compared to CUHK-PEDES, ICFG-PEDES places greater emphasis on fine-grained descriptions of pedestrian appearance, and deliberately removes irrelevant background and redundant information to improve semantic clarity in image-text alignment. Furthermore, the dataset construction introduces greater appearance diversity to mitigate the background consistency bias observed in previous datasets. As a result, ICFG-PEDES serves as a more challenging benchmark for evaluating model performance under complex and detail-sensitive matching scenarios. Table 5 summarizes the detailed statistics of the datasets.

RSTPReid [132] is a text-image person re-identification dataset constructed by Nanjing University of Science and Technology, aiming to address the limitations of CUHK-PEDES, where each pedestrian is typically captured by the same camera under similar time-space conditions—an unrealistic setting for real-world applications. Built upon the MSMT17 [131] dataset, RSTPReid contains 20,505 images of 4,101 identities, captured by 15 distinct cameras under diverse viewpoints, lighting conditions, locations, and weather scenarios, offering significantly higher environmental variability and realism. Each identity in the dataset is represented by five images captured from different camera perspectives, accompanied by two natural language descriptions, each containing no fewer than 23 words. The dataset is partitioned into a training set comprising 3,701 identities, while the validation and test sets each consist of 200 identities. Despite the relatively limited number of identities, RSTPReid encompasses both indoor and outdoor environments, highlighting significant variations in camera angles and illumination conditions. This characteristic renders it a more rigorous and representative benchmark for Text-Image ReID tasks. Comprehensive details regarding the datasets are presented in Table 5.

Summary:Overall, these benchmarks differ substantially in data provenance and annotation strategy, leading to distinct benchmark biases. CUHK-PEDES is constructed by integrating multiple ReID datasets, resulting in rich semantic descriptions but also introducing background and scene consistency biases. In contrast, ICFG-PEDES and RSTPReid, both built upon MSMT17, place greater emphasis on fine-grained appearance descriptions and cross-camera diversity, respectively, providing more realistic and challenging evaluation settings. These differences should be taken into account when comparing model performance across benchmarks.

2.2.2 Methods↩︎

At the early stage of research on Text-Image ReID (TI-ReID), most approaches employed a global matching paradigm [133]–[139], where entire images are mapped to comprehensive textual descriptions within a unified semantic space, thereby enhancing the efficacy of cross-modal retrieval.

However, natural language descriptions are typically concise and semantically dense, containing numerous fine-grained cues that are crucial for identity discrimination (e.g., “carrying a black shoulder bag”, “wearing a blue hat”). When relying solely on global representations, models tend to overemphasize semantically strong but low-discriminative words (e.g., “man”, “person”), while neglecting key descriptions that truly convey identity differences. Moreover, in real-world scenarios such as surveillance, eyewitness recollections of suspects often focus on salient attributes and local details, further highlighting the importance of fine-grained alignment modeling [[140]][141]. Moreover, [[135]][142] jointly leverage global identity-level descriptions and local phrase-level region features to construct more fine-grained text-image alignment. Shao et al. [143] propose a unified pre-training framework specifically designed for the TI-ReID task. To support this framework, they also construct a large-scale text-annotated dataset, LUPerson-T, which achieves consistent alignment between image and text modalities at both data and training levels. Han et al. [144] propose a novel cross-modal momentum contrastive learning framework to better learn more discriminative feature representations on small-scale datasets.

In addition, images and text differ significantly in structural representation and semantic granularity. For instance, the word “man” in text may correspond to the entire human figure in an image, while a local image region (e.g., a backpack) may align with a more detailed phrase such as “carrying a blue backpack.” Moreover, images typically contain rich visual information, whereas textual descriptions are often coarse, leading to granularity mismatches that hinder precise cross-modal alignment and may result in semantic drift or retrieval failures. To address this issue, some studies introduce explicit granularity alignment mechanisms. For example, Jing et al. [145] align full-sentence descriptions with relevant image regions and leverage human pose information to guide the alignment between body parts in images and corresponding noun phrases in text, thereby bridging semantic granularity gaps. Niu et al. [146] propose a multi-granularity alignment framework that sequentially aligns image and text at three levels: global-global (entire image with entire text), global-local (entire image with text phrases), and local-local (image regions with text segments), progressively refining the matching process. Shao et al. [147] propose the LGUR framework, which employs a dictionary-guided alignment module (DGA) and a prototype-guided unification module (PGU) to explicitly align visual regions with textual semantic units in the feature space, effectively mitigating granularity discrepancies and enhancing both matching accuracy and cross-domain generalization. Additionally, Huang et al. [148] extract multi-scale semantic features from both text and image using variable-sized windows and project them into a shared space to capture alignment across different semantic levels.

Most existing studies leverage uni-modal pre-trained models as backbone encoders; for example, Long Short-Term Memory (LSTM) networks or BERT [149] are employed for textual encoding, while ResNet [150] or Vision Transformer (ViT) [151] are utilized for visual encoding. However, these approaches often fail to fully exploit the synergistic potential inherent in the interaction between different modalities. In recent years, Visual-Language Pre-training (VLP) methods have garnered increasing attention due to their superior capability in learning cross-modal representations. Among them, CLIP[152] stands out as a representative framework, constructing a unified multimodal semantic space via large-scale contrastive learning on image-text pairs. A large number of studies have fine-tuned CLIP for various downstream tasks, such as text-video retrieval [153]–[155], referring image segmentation[82], and general video recognition [156], achieving remarkable performance across these applications. Inspired by this trend, researchers have begun to explore the potential of CLIP in the TI-ReID task, aiming to leverage its strong cross-modal alignment capabilities to enhance text-based person retrieval performance.

Li et al. [157] represents the first systematic attempt to introduce CLIP into ReID tasks, in which the authors develop a two-stage training framework named CLIP-ReID. Yan et al. [158] propose CFine, a fine-grained information mining framework based on CLIP, which aims to transfer the knowledge of CLIP to the TI-ReID task and achieve fine-grained text-image alignment. It is noteworthy that CLIP’s pretraining objective and data domain are inherently misaligned with the requirements of the TI-ReID task. Cao et al. [159] conduct a comprehensive study on CLIP’s performance in TI-ReID from the perspectives of data augmentation and loss design. By incorporating tailored training heuristics, augmentation strategies, and customized loss functions, they establish a strong TBPS-CLIP baseline tailored to the TIReID setting. CLIP is trained on large-scale image-text datasets; however, its training domain differs significantly from that of TI-ReID, with image scenes and textual description styles being entirely distinct. Directly applying CLIP to TI-ReID may restrict the model’s discriminative ability and generalization performance in identity-level retrieval. To address this issue, Li et al. [160] propose a two-stage training strategy. In the first stage, only prompts are optimized with contrastive loss to achieve domain adaptation, enabling the model to better align with the TI-ReID data distribution. In the second stage, with prompts fixed, the encoders are fine-tuned to focus on fine-grained identity features, thereby accomplishing task adaptation. This approach facilitates a more effective transfer of CLIP’s knowledge to the downstream TI-ReID task.

Moreover, the CLIP model focuses on constructing instance-level matching between image-text pairs, with its pre-training objective aimed at learning semantic correlations between images and their corresponding texts, thereby emphasizing one-to-one correspondence. In contrast, TI-ReID requires identity-level retrieval based on natural language descriptions, highlighting cross-modal identity consistency modeling. This shift from “semantic matching” to “identity discrimination” directly limits the transferability of CLIP to TI-ReID tasks. To address this challenge, Yan et al. [161] construct a multi-image multi-text support set during training, perform semantic enhancement for alignment, and finally distill the model into a lightweight version that requires only a single image or text at inference. Gou et al. [162] propose a Structure-level Distribution Guided feature learning method, which introduces more stable category-level feature distributions to calibrate the unstable instance-level distributions. Furthermore, Yan et al. [163] establish an end-to-end Prototypical Prompting framework (Propot) that simultaneously models instance-level and identity-level matching, ensuring precise text-image alignment while enhancing cross-image/text identity consistency modeling.

Although vision-language pre-trained models exhibit strong semantic understanding and achieve impressive performance in image-text alignment tasks, they often rely on an implicit yet unrealistic assumption: all image-text pairs in the training data are perfectly aligned, clean, and error-free. In the context of TIReID, where most image-text pairs share high semantic consistency but may contain subtle mismatches in fine-grained details, conventional methods struggle to detect such local misalignments, as shown in Fig. 1, ‘D’,thereby degrading discriminative performance. To address this, Xu et al. [164] explicitly model “mismatched word–region pairs” and incorporate them as negative signals during training, enhancing the model’s sensitivity to imperfect matches. Building upon this idea, Qin et al. [165] formally define such mismatched correspondences as "Noisy Correspondence (NC)" and propose a Robust Dual-branch Embedding (RDE) framework, which mitigates the impact of noisy alignments on model optimization through a noise-tolerant training paradigm.

In TI-ReID, modality information is often incomplete due to subjective or ambiguous textual descriptions and occluded or low-quality images. To enhance the robustness of models under such incomplete information, existing studies commonly adopt cross-modal semantic inference mechanisms to mitigate alignment difficulties arising from modality incompleteness. Both Fujii et al. [166] and Xiang et al. [167] introduce masked modeling strategies over image patches and text tokens, employing bidirectional masking and reconstruction to enhance the model’s robustness against local semantic omissions. Alternatively, Du et al. [168] construct a semantic-aware graph structure to guide the available modality in semantically completing the missing one, thereby achieving cross-modal reconstruction and alignment from a graph-structural perspective.

Beyond the above discussions, several recent works explore alternative perspectives to further enhance TIReID performance. For instance, Si et al. [169] introduce an interactive cross-modal learning framework (ICL), which supports multi-turn interaction between users and multi-modal large language models (MLLM) via visual question answering, enabling dynamic query refinement with a focus on person-specific attributes. In addition, Du et al. [170] identifie the model’s over-reliance on color cues when image and textual color information align, and proposes a color-robust learning framework by incorporating color-perturbed images as an auxiliary modality, along with fine-grained alignment at the minimal semantic unit level across image, perturbed image, and text to mitigate color bias and enhance semantic discrimination.

Recent studies extend vision-language adaptation for TI-ReID beyond fixed identity prompts and full-model fine-tuning. HAM[171] learns prompts that represent diverse human annotation styles and uses them to guide multimodal language models in generating varied pedestrian descriptions. Niu et al.[172] adopts hierarchical progressive tuning to transfer a vision-language model from pedestrian-attribute understanding to fine-grained retrieval and instruction-driven reasoning. From the perspective of parameter-efficient transfer, DM-Adapter[173] inserts a sparse mixture of domain-aware adapters into the frozen visual and textual branches of CLIP, allowing different experts to capture complementary aspects of fine-grained person knowledge. Compared with full fine-tuning, prompt and adapter strategies update fewer task-specific parameters and can better preserve pretrained knowledge; however, their efficiency and retrieval value depend on whether the adapted components capture identity-level details rather than only generic pedestrian semantics.

4pt

2.3 Sketch ReID↩︎

In real-world security and criminal investigation scenarios, acquiring high-quality photographs of suspects or witnesses is often extremely difficult. In many cases, investigators have to rely on sketches drawn from eyewitness memory, which gives rise to the Sketch-ReID task that aims to enable cross-modal identity retrieval between sketches and photographs.

2.3.1 Datasets↩︎

PKU-Sketch [198] is a prominent dataset for sketch-based person re-identification, encompassing 200 identities, each with one hand-drawn sketch and two photographs. The photographs, captured in daylight from two cross-view cameras, are manually cropped to isolate individual subjects. Sketches, created by five professional artists, were iteratively refined from eyewitness descriptions, resulting in diverse drawing styles with an imbalanced distribution. The dataset adheres to a structured protocol that designates 3/4 of the identities per style for training purposes and the remaining 1/4 for testing, resulting in a 150/50 identity split. Each sketch and photograph within the dataset is uniquely annotated, ensuring clarity and precision in matching. This organization mirrors real-world scenarios encountered in surveillance and forensic sketch-photo matching, making PKU-Sketch a crucial benchmark for assessing the efficacy of sketch-based Re-identification (ReID) methods. For additional information and insights, please refer to Table 7.

CUFSF database (CUHK Face Sketch FERET) [199] is constructed by the Chinese University of Hong Kong based on the FERET dataset, primarily for face sketch synthesis and recognition research. It contains 1,194 identities, each with a face photo under lighting variations and a corresponding sketch. The sketches are drawn by professional artists while viewing the photos, featuring certain shape exaggerations to realistically capture the modality gap between sketches and photos.Detailed information regarding the datasets is provided in Table 7.

Summary:Overall, these sketch-based benchmarks exhibit distinct acquisition and annotation biases. PKU-Sketch is designed for person retrieval under surveillance scenarios but is limited by its relatively small scale and style imbalance among hand-drawn sketches, whereas CUFSF focuses on face sketch-photo matching with sketches generated under controlled conditions. Consequently, existing sketch-based benchmarks only partially capture the diversity and complexity of real-world sketch-based ReID applications.

2.3.2 Methods↩︎

Sketches are inherently abstract, preserving only structural contours and sparse texture details, while discarding critical visual cues such as color, illumination, and background context. Furthermore, sketches are typically drawn from standard frontal views, whereas real images often exhibit large variations in viewpoint, pose, occlusion, and imaging conditions. Additional semantic divergence arises from the subjectivity of eyewitness descriptions and stylistic differences in sketch rendering. These factors collectively exacerbate the modality gap between sketches and photos, making cross-modal alignment and identity discrimination particularly challenging. As a compelling extension of traditional person Re-ID, it offers substantial real-world relevance and has garnered increasing attention within the broader field of cross-modal visual understanding. We compare the performance of existing methods in Table 8.

Pang et al. [198] first formally introduce the Sketch Re-ID task and propose a cross-domain adversarial feature learning framework consisting of a domain discriminator and two modality-specific generators. Sketches and photos are encoded by separate generators and then fed into the discriminator, which enforces modality confusion through adversarial learning. Meanwhile, identity classifiers in both modalities provide supervision to ensure discriminative identity representations. This approach effectively filters out low-level interference (e.g., texture and color) while preserving high-level semantic information. Meanwhile, Pang et al. release the PKU-Sketch dataset, which plays an important role in driving subsequent studies on Sketch Re-ID. Li et al. [200] introduce a Cross-Compatible Embedding (CCE) framework combined with a Semantic Consistent Feature Construction (SCFC) scheme, which enables local-level cross-modal feature exchange in a Transformer and constructs semantically consistent global representations to alleviate the modality gap in Sketch Re-ID. Gui et al. [201] propose a multi-level domain-invariant feature learning framework for sketch-based ReID with a staged training strategy. It model first extracts shared global identity features, then incorporates an adversarial domain discriminator to reduce modality discrepancies at the mid-level, and finally applies attention mechanisms to align fine-grained local features across modalities. To mitigate the inherent information asymmetry between sketches and photos, Chen et al. [202] first generate an auxiliary sketch modality from the photo modality, then employ an asymmetric disentanglement mechanism to decompose photo features into sketch-relevant and sketch-irrelevant parts, transferring the latter to the sketch modality to compensate for missing information, and further introduce a prototype contrastive learning scheme to capture shared modality information. Liu et al. [203] introduce an intermediate modality that integrates photo appearance cues with sketch structural abstractions to bridge the modality gap, and further design a tri-modal collaborative learning strategy across sketches, the intermediate modality, and photos to enhance semantic consistency and discriminative capability under limited sketch information. In Sketch Re-ID, sketches are often influenced by eyewitness memory bias and stylistic variations, leading to significant semantic inconsistencies with real photos. To address this subjectivity issue, Lin et al. [204] construct a large-scale, multi-style sketch-photo dataset to systematically model and analyze the challenges in sketch-based re-identification. The dataset contains 4,763 sketches and 32,668 photos, with each identity represented by multiple sketches drawn by different eyewitnesses and artists, thereby realistically capturing perceptual discrepancies and stylistic diversity. Based on this dataset, the authors introduce two modules to mitigate subjectivity: the NL module, which aggregates multiple sketches of the same identity to counteract discrepancies caused by different eyewitnesses and drawing styles; and the AttrAlign module, which leverages attributes such as gender and clothing as references to facilitate cross-modal alignment between sketches and photos. Recently, Hu et al. [205] propose adaptive incremental prompt tuning to progressively transfer drawing-style knowledge from generic sketch datasets to Sketch-ReID, addressing the limited availability of identity-labeled sketch data while improving domain adaptation to sketch-specific representations. Gong et al. [206] propose KTCAA, a few-shot cross-modal framework that combines alignment augmentation with a knowledge transfer catalyst in a meta-learning paradigm, aiming to reduce domain discrepancy and improve robustness to modality perturbations with limited labeled sketches. These studies extend Sketch-ReID beyond conventional feature alignment by considering efficient style adaptation and few-shot cross-modal generalization.

2.4 NLOS ReID↩︎

In recent years, Non-Line-of-Sight Re-identification (NLOS ReID) has emerged as a significant research area that integrates wireless signal perception with computer vision, addressing the limitations of traditional visual sensors for pedestrian identification in occluded or non-line-of-sight scenarios. This approach utilizes various wireless signal technologies, such as radar, Wi-Fi, and ultrasonic systems, to enhance identification capabilities.

Non-Line-of-Sight ReID extends cross-modal identity retrieval beyond conventional visual observations by exploiting sensing signals that remain informative under occlusion, poor illumination, or limited camera visibility. Existing studies provide important foundations for this direction by investigating whether non-visual observations contain stable identity evidence.

Guo et al. [212] introduce ReID3D, the first LiDAR-based person re-identification framework, together with the LReID dataset and the synthetic LReID-sync dataset. By exploiting geometric features and pre-training for point-cloud completion and shape learning, their work demonstrates that sparse 3D observations can contain identity-relevant structural cues beyond conventional image appearance. However, the sparsity and viewpoint dependence of point clouds make it difficult to establish fine-grained correspondence with dense RGB images, while the synthetic-to-real gap may further affect the transferability of the learned representations. Beyond static geometric observations, Liu et al.[213] investigate cross-modal ReID using millimeter-wave (mmWave) radar and RGB cameras. Their method aligns radar and image features in a shared representation space and introduces 3D pose estimation as an auxiliary task. This design highlights the value of radar for capturing motion- and geometry-related identity cues under occlusion or poor illumination. It is worth noting that radar measurements are strongly affected by signal reflection, body orientation, multipath propagation, and temporal misalignment, making direct pixel-level correspondence with RGB images inappropriate. Cascio et al.[214]introduce Wi-PER81, the first public benchmark for radio-signal-image-based ReID, containing 162,000 wireless signal packets from 81 individuals across two sessions, together with a Siamese baseline for processing signal-magnitude heatmaps. The dataset and baseline establish an important foundation for evaluating identity cues in wireless signals. A key challenge is that the observed Wi-Fi signal patterns are highly dependent on the sensing environment. Variations in devices, rooms, propagation conditions, and body–sensor configurations can substantially alter the measured signals, making stable identity representation considerably more difficult than in vision-based ReID. Sheng et al. [215] explored person ReID using mmWave radar by leveraging sparse 3D point clouds instead of visual appearance. Their framework combines an Anchor-based Orientation Descriptor (AOD) with a Multi-scale Gait Catch (MGC) module to jointly model local geometric structures and multi-scale gait dynamics for robust identity representation.Zhao et al. [216] introduced RF-ReID, which utilizes radio frequency (RF) reflections to extract intrinsic body characteristics rather than appearance cues, enabling long-term, privacy-preserving person re-identification that is robust to clothing changes and occlusions. More recently, Cascio et al. [214] further advanced this direction by introducing the Wi-PER81 benchmark, the first public Wi-Fi signal image-based ReID dataset, together with a Siamese baseline for learning discriminative representations from wireless signal heatmaps.

Beyond sparse radar point clouds, recent mmWave ReID studies explore temporal and image-like signal representations. Han et al. [217] extracts multi-frequency micro-Doppler patterns to characterize gait dynamics, whereas Wang et al. [218] constructs radio images from mmWave echoes to retain more spatial structure. These approaches demonstrate that radar identity cues can be represented as geometry, motion spectra, or radio images. Their alignment with RGB data nevertheless remains nontrivial: micro-Doppler features require gait-phase and sequence-level correspondence rather than pixel matching, while radio images differ from optical images in resolution, reflection mechanism, and sensitivity to body orientation and multipath. Mission [213] directly addresses the cross-modal case by matching a radar query to an RGB gallery, but broader cross-device and cross-environment validation remains necessary.

Acoustic sensing provides a related but less mature direction for identity perception. SonicID [219] uses active ultrasonic reflections for near-field user authentication, while acoustic footstep studies  [220] identify people from gait-related sound patterns. These results indicate that echoes and footsteps can encode identity-related shape or motion information, but they should not be regarded as established acoustic-to-visual ReID methods. Footstep signatures vary with footwear, floor material, reverberation, background noise, and microphone placement, and their temporal gait cues have little direct correspondence with appearance-rich RGB images. Establishing this direction therefore requires synchronized acoustic–visual datasets, identity-level query–gallery protocols, and cross-room and cross-device evaluation.

Despite recent progress, cross-modal alignment between non-visual observations and dense visual imagery remains the primary challenge in NLOS ReID. Unlike RGB images, non-visual modalities provide sparse or indirect observations with fundamentally different representations, making identity correspondence difficult to establish. Learning modality-invariant yet identity-discriminative representations therefore remains a key direction for future research.

3 Tri-spectral Person Re-identification↩︎

Most existing studies on cross-modal visible–infrared person re-identification (ReID) focus on dual-spectral settings, which alleviate some limitations of single-modal approaches but do not fully address the diverse requirements of extreme conditions. This shortcoming drives the investigation of tri-spectral ReID, which utilizes complementary information from RGB, NIR, and TIR modalities to better meet the challenges of complex surveillance environments. RGB images capture visible light, offering rich color and texture information but are highly sensitive to illumination variations. Infrared images can be classified by their imaging principles: NIR images are generated from near-infrared reflection, providing stable structural data under low light but suffering detail loss in bright conditions; TIR images, derived from heat radiation, facilitate effective human–background separation in darkness or adverse weather, though they lack fine-grained appearance details. Leveraging these complementary properties, tri-spectral person re-identification research aims to integrate RGB, NIR, and TIR data to enhance the robustness and discriminative power of identity representations.

The selection of RGB, near-infrared (NIR), and thermal-infrared (TIR) as the primary tri-spectral configuration is motivated by their complementary sensing characteristics under varying illumination conditions. RGB provides rich appearance details, NIR preserves structural information under challenging lighting, and TIR captures thermal patterns in dark environments. Therefore, existing tri-spectral ReID studies combine these modalities to exploit complementary visual cues across diverse scenarios. Nevertheless, this configuration represents a practical choice supported by existing sensing systems and benchmarks rather than a universally optimal solution.

3.1 Datasets↩︎

RGBNT201 [221] is a benchmark dataset designed for tri-spectral person re-identification. It contains synchronized RGB, NIR, and TIR observations of 201 identities captured by four cameras, providing an experimental basis for studying spectral complementarity under challenging imaging conditions. Nevertheless, its relatively limited identity and camera coverage constrains the diversity of populations, scenes, and sensing conditions represented in the benchmark. Moreover, the use of synchronized and complete modality triplets favors fixed three-stream fusion, whereas practical systems may encounter imperfect registration or unavailable modalities. Performance on RGBNT201 should therefore be interpreted as evidence of fusion effectiveness under its acquisition protocol, rather than as a complete measure of robustness across different sensors and deployment environments.

3.2 Methods↩︎

Wang et al. [222] propose TOP-ReID, a novel multi-spectral ReID framework, which performs token-level permutation to achieve feature alignment and aggregation across spectra, thereby enhancing discriminative feature interaction while preserving modality-specific details. To address the issue of missing spectral information in real-world scenarios, a token-level reconstruction constraint is further introduced to effectively reduce distribution gaps among different spectra. In a subsequent work, Wang et al. [223] further introduce the DeMo framework to address the challenges of dynamic imaging quality variations and the weakening of modality-specific information in multi-modal object re-identification. The framework employs a Hierarchical Decoupling Module (HDM) to separate modality-specific and shared information, and introduces an Attention-Triggered Mixture of Experts (ATMoE) mechanism to perform dynamic weight allocation.

4 Multi-modal Person Re-identification↩︎

Cross-modal person re-identification continuously drives the field toward greater adaptability and practical utility across diverse scenarios, significantly enhancing retrieval performance under challenging conditions such as complex illumination and missing visual information. Nevertheless, existing research and datasets are predominantly limited to specific modality combinations, whereas real-world applications require ReID systems capable of accurately identifying individuals across both single-modality and multi-modality inputs.

Building on this practical objective, research has gradually expanded from binary cross-modal settings to multi-modal configurations, enabling more flexible inputs and more comprehensive information integration. Its primary objective is to integrate complementary identity evidence across modalities while controlling cross-source redundancy and accommodating variations in modality availability. Although effective multi-modal fusion still requires cross-modal alignment, its principal emphasis lies in combining complementary information rather than performing retrieval between a fixed pair of heterogeneous query and gallery domains. Existing multi-modal ReID frameworks mainly encompass two categories: tri-modal settings (image, text, and sketch) and five-modal settings (visible image, infrared image, text, sketch, and color sketch). In this chapter, we systematically review related methods and datasets, summarizing their technical characteristics and application scenarios.

4.1 Datasets↩︎

Zhai et al. [232] construct a multi-modal ReID dataset that encompasses text, sketch, and RGB images by extending the existing PKU-SketchReID dataset, where an image captioning model is employed to automatically generate textual descriptions for the RGB images. Chen et al. [233] propose and construct three large-scale multi-modal person re-identification datasets, namely Tri-CUHK-PEDES, Tri-ICFG-PEDES, and Tri-RSTPReid, to address the lack of multi-modal benchmarks in existing research. These datasets are extended from the text-based datasets CUHK-PEDES[124], ICFG-PEDES[130], and RSTPReid[132], respectively, by introducing the sketch modality, thereby forming a tri-modal structure that includes RGB images, textual descriptions, and synthesized sketches. Zuo et al. [234] propose the ORBench dataset, which is constructed on the basis of existing visible–infrared benchmarks SYSU-MM01 [24] and LLCM [26]. Specifically, representative pedestrian samples are first selected from RGB and infrared images, while discarding those with poor imaging quality or weak clothing representativeness. Then, an online image generation model [235] combined with manual supervision is introduced to produce color pencil drawings for each pedestrian, which are further transformed into sketch-style images using the Meitu software [236]. In addition, human annotators are employed to provide detailed textual descriptions for each RGB sample, thereby enriching the dataset with semantic information. In total, ORBench contains 1,000 identities with 45,113 RGB images, 26,071 infrared images, 18,000 color pencil drawings, 18,000 sketches, and 45,113 textual descriptions. Table 10 summarizes the detailed statistics of the datasets.

4.2 Methods↩︎

As two representative descriptive modalities, text and sketch each possess unique advantages. Text, despite its abstract nature, can convey rich semantic information such as body pose, clothing details, and carried items. Sketches, on the other hand, while belonging to the visual modality, excel at depicting contours, structural shapes, and local textures, but often lack color information and exhibit limitations in conveying fine-grained details and complete semantic content. Therefore, text and sketch are inherently complementary in terms of semantic representation. Effectively integrating these two descriptive modalities holds the potential to compensate for the weaknesses of each individual source, enabling the construction of more comprehensive and discriminative person representations, and offering valuable support for cross-modal retrieval in complex real-world scenarios. Zhai et al. [232] are the first to use both text and sketch as query modalities for multi-modal person re-identification. Their method extracts features separately for textual and sketch inputs, and then employs an attention-based fusion mechanism to generate a comprehensive semantic representation. To further enhance cross-modal matching, they introduce a generative adversarial learning framework, where multiple loss functions progressively align descriptive and image features. Subsequent studies point out that independently pre-training text and image modalities may limit generalization capability. Moreover, in real-world scenarios, the availability of both text and sketch modalities is not always guaranteed, which highlights the need for practical systems capable of handling diverse query modalities with consistent performance. To address this issue, Chen et al. [233] propose the UNReID framework, which unifies Text-to-RGB, Sketch-to-RGB, and Text+Sketch-to-RGB retrieval into a single paradigm. This framework supports both cross-modal and multi-modal retrieval, and the authors further construct three multi-modal datasets—Tri-CUHK-PEDES, Tri-ICFG-PEDES, and Tri-RSTPReid—based on existing text–image datasets. Ha et al. [237] introduce MP-ReID, which contains RGB, infrared, and thermal observations acquired from ground and aerial platforms, together with Uni-Prompt ReID. The method combines modality-aware and platform-aware prompts with visual context to condition a shared representation on different sensing and acquisition settings. These studies illustrate two complementary forms of modality flexibility: adapting retrieval to different combinations of available queries and adapting a unified model to heterogeneous sensors and platforms.

With the continuous advancement of research, multi-modal person ReID has been further extended to a five-modality setting. Zuo et al. [234] introduce infrared images and color pencil drawings as additional query modalities on top of RGB, text, and sketch, and propose a unified multi-modal person re-identification framework named ReID5o. This model not only supports single-modality inputs but also flexibly handles combinations of multiple modalities, thereby significantly enhancing the adaptability and robustness of both cross-modal and multi-modal retrieval. Specifically, ReID5o first employs a multi-modal tokenizing assembler to project heterogeneous inputs into a shared embedding space; then applies a unified encoder to extract modality-shared features, together with a multi-expert router to explicitly capture modality-specific representations; meanwhile, a feature mixture mechanism is adopted to fuse information from multiple sources, and cross-modal alignment strategies are integrated to optimize the training objective.

5pt

ReID5o [234] provides a controlled setting for examining modality complementarity because multiple query-modality combinations are evaluated using the same model and ORBench protocol. For example, taking the text-only query as a reference, its combination with infrared, color-pencil, or sketch information increases mAP from 54.88% to 70.77%, 81.14%, and 74.01%, respectively. This comparison does not imply that ReID5o relies on a fixed text-centered modality configuration; rather, it provides one illustrative subset of its flexible query combinations for measuring the incremental contributions of different modalities. As summarized in Table 11, these contributions are not uniform. Adding infrared to the text+color-pencil query improves mAP by 5.61 percentage points, whereas adding sketch to the same pair improves it by only 1.91 points. Moreover, the full text+infrared+color-pencil+sketch query reaches 87.46% mAP, only 0.71 points above text+infrared+color-pencil. These controlled comparisons provide task-level evidence of modality complementarity while also revealing redundancy and diminishing returns. Therefore, increasing the number of modalities does not necessarily constitute the best deployment choice when sensing cost, input quality, and modality availability are jointly considered.

30pt

5 Proposed baseline↩︎

Drawing on the comprehensive review presented in this survey, we observe a clear evolution of VI-ReID research from early CNN-based architectures toward Transformer-based paradigms. While CNN-based methods have achieved remarkable success in feature extraction and modality alignment, they primarily rely on local texture cues and often suffer performance degradation when processing infrared images that lack color and fine-grained appearance details. This observation motivates us to adopt a Transformer-based baseline as a representative modern architecture for validating the design principles summarized throughout the survey. Compared with CNNs, Transformers are better suited for modeling global structural and shape information, providing stronger representational capacity under the infrared modality and facilitating the learning of modality-invariant identity features. Moreover, their patch-token representation naturally enables fine-grained cross-modal semantic interactions, allowing more effective local feature alignment and improved discriminative capability.

Recent advancements in Transformer architectures have spurred significant progress in person re-identification. Various studies have sought to exploit the structural advantages of Transformers to overcome the limitations of traditional CNNs. For instance, Li et al. [238] introduce multiple class tokens with self-diverse constraints to generate compact and diverse embedding subspaces, enhancing discrimination for similar identities. Zhu et al. [239] propose an auto-alignment mechanism that aligns patch-level features to mitigate the impact of misalignment and occlusion. Tan et al. [240] further explore fine-grained representation by disentangling attention heads via a Head Disentangling Block and enforcing diversity constraints. Li et al. [241] incorporate a pyramid architecture with convolutional patch embedding and auxiliary embedding to model multi-scale and non-visual biases

These advances not only demonstrate the potential of Transformers in ReID tasks but also provide new directions for further exploration. Inspired by this, we propose a unified baseline for visible-infrared person re-identification (VI-ReID) that supports both supervised and unsupervised learning paradigms. The architecture is built upon a dual-path Transformer, where visible and infrared images are processed through separate modality-specific branches to extract shallow appearance features. Additionally, we incorporate the random channel augmentation strategy introduced by [50] into the visible stream to enhance joint training. This dual-path design enables the model to capture modality-specific representations and leverage the complementary nature of visible and infrared inputs. It is particularly effective under challenging conditions such as blur, low resolution, occlusion, illumination changes, and viewpoint variations, enhancing the system’s robustness and accuracy.

On top of the dual branches, a shared Transformer encoder is used to learn modality-invariant identity embeddings in a unified feature space. Inspired by DC-Former [238] , we introduce the class token as a global identity aggregator, which interacts with patch tokens to capture holistic identity information and obtain more discriminative and robust identity representations. To further bridge the modality gap, we design a simple yet effective modality confusion strategy. Initially, we employ a feature-level weighted confusion mechanism to compel the model to learn modality-invariant representations. Subsequently, we maintain a shared memory bank to integrate multi-modal information. During the optimization phase, we utilize cluster contrastive learning to achieve a more compact clustering distribution in the feature space by pulling samples closer to their assigned cluster centroids while pushing away heterogeneous centroids. In the context of unsupervised learning, pseudo-labels generated via clustering inevitably contain noise, which detrimentally affects training efficacy. Recognizing that the IR modality often retains more stable structural information under adverse conditions (e.g., illumination variations and color interference), we propose leveraging the IR clustering results to assist in updating the RGB pseudo-labels. Specifically, we propagate the cluster labels from the IR modality to RGB samples based on a cross-modal affinity graph to reconstruct the RGB pseudo-labels, thereby enhancing their accuracy. This strategy effectively mitigates the semantic bias induced by clustering and bolsters the robustness and discriminability of cross-modal pseudo-supervised learning.

The architecture of our proposed baseline is shown schematically in Fig. 6. Our method demonstrates superior overall performance on both the RegDB and SYSU-MM01 datasets. Under both supervised and unsupervised settings, the results significantly surpass mainstream baseline methods, fully highlighting the potential and competitiveness of the Transformer framework for the VI-ReID task. Overall, this baseline provides a strong starting point for Transformer-based VI-ReID and holds considerable value as a unified evaluation platform for subsequent approaches. The comparison with the current classical baselines on two datasets is shown in Table 12.

3pt

5.1 Implementation Details↩︎

The proposed baseline is implemented in PyTorch using a ViT-Base/16 backbone initialized with pretrained weights. RGB and infrared images are processed by separate patch embedding layers and subsequently passed through a shared Transformer encoder comprising 12 blocks, 12 attention heads, and a 768-dimensional embedding space. The output of the shared class token is processed by a Batch Normalization neck to obtain the identity representation. All images are resized to \(288\times144\). Training augmentation includes random cropping, horizontal flipping, color jittering, random erasing, and channel augmentation. The model is trained for 50 epochs using SGD with a batch size of 256, eight instances per identity, an initial learning rate of \(3.5\times10^{-4}\), a momentum of 0.9, and a weight decay of \(5\times10^{-4}\). The learning rate is reduced by a factor of 0.1 every 20 epochs. Under supervised training, the objective combines the RGB and infrared memory-based contrastive losses with a weighted regularized triplet loss. For unsupervised training, RGB and infrared pseudo-labels are generated separately using DBSCAN on the re-ranked Jaccard distance. We set the DBSCAN threshold to 0.6, the minimum number of samples to 4, and \(k_1=30\) and \(k_2=6\). The contrastive temperature and memory momentum are set to 0.05 and 0.1, respectively. Cross-modal similarities are subsequently used to associate infrared clusters with RGB samples and refine the RGB pseudo-labels.

5.2 Ablation Study Design↩︎

To further justify the key design choices of the proposed Transformer-based baseline, we conduct an ablation study in both supervised and unsupervised VI-ReID settings. The ablation focuses on three components that are central to the proposed framework: the modality-shared Transformer encoder, the feature-level modality confusion strategy, and the IR-guided RGB pseudo-label refinement module. These components correspond to the main motivations discussed in the survey, namely learning modality-invariant identity representations, reducing the heterogeneous modality gap, and improving cross-modal label association under noisy pseudo-supervision.

Table 13 summarizes the ablation protocol. In the supervised setting, we evaluate the contributions of the modality-shared Transformer encoder and the feature-level modality confusion strategy. In the unsupervised setting, we further evaluate the IR-guided RGB pseudo-label refinement module, while retaining the other training components and optimization settings of the proposed baseline. All ablation variants are trained and evaluated under the same protocols as the corresponding full model. The experimental results are reported on SYSU-MM01 and RegDB using Rank-1 accuracy and mAP.

3pt

This ablation study is designed to clarify the individual and joint contributions of the proposed modules. In the supervised setting, we evaluate the effects of the modality-shared Transformer encoder (MSTE) and the feature-level modality confusion (FMC) strategy by comparing the full model with variants containing neither module or only one of them. MSTE examines whether sharing high-level Transformer representations across modalities facilitates visible- infrared alignment, while FMC evaluates the benefit of explicitly reducing modality discrepancies at the feature level. In the unsupervised setting, we further introduce IR-guided RGB pseudo-label refinement (IR-PR). By comparing single-module, pairwise-combination, and full- model variants, we investigate whether IR-PR improves the reliability of RGB pseudo-labels and whether it complements MSTE and FMC. Overall, the results demonstrate the individual contributions and complementary effects of these modules.

6 Future Research Perspectives↩︎

This section further discusses the future research potential and development directions of VI-ReID, aiming to provide insights for advancing both model performance and practical applicability.

Identity-aware Foundation Models Vision-language foundation models, such as CLIP, BLIP, and GPT-4o, have demonstrated remarkable capabilities in cross-modal representation learning. However, these models are primarily optimized for semantic alignment rather than identity-level discrimination, which limits their direct applicability to person re-identification. Future research should investigate identity-aware adaptation strategies that preserve fine-grained identity cues while maintaining the strong semantic priors of foundation models. Possible directions include identity-centric prompt learning, modality-aware token interaction, instance-level contrastive objectives, and lightweight parameter-efficient adaptation. Such designs could enable foundation models to better distinguish visually similar individuals while maintaining robust cross-modal generalization.

Causal Representation Learning for Modality Disentanglement A fundamental challenge in VI-ReID lies in disentangling identity information from modality-specific variations, such as illumination, sensor characteristics, and environmental conditions. Most existing approaches rely on feature alignment or adversarial learning, which often fail to explicitly distinguish causal identity factors from nuisance modality attributes. Future research may benefit from causal representation learning, which aims to separate invariant identity semantics from modality-dependent characteristics by modeling the underlying causal relationships. Such a framework could improve robustness to unseen sensors and environments while enhancing the interpretability and generalization ability of cross-modal ReID systems.

Neuro-symbolic Sketch Understanding. Sketch-ReID is challenging because sketches are abstract, subjective, and structurally sparse. Existing methods mainly rely on end-to-end feature alignment, which may overlook explicit human body structure, part relations, and attribute logic. Neuro-symbolic learning offers a concrete direction by combining neural representation learning with symbolic reasoning over body parts, attributes, and spatial relations. For example, a model can first extract neural features from sketch contours and photo regions, and then reason over symbolic structures such as “upper clothing-lower clothing”, “carried object–body part”, or “part adjacency”. This can help bridge the gap between subjective sketch strokes and identity-related visual evidence in real photos, especially when color and texture cues are missing.

World Models for Long-term Identity Reasoning Recent advances in world models have demonstrated impressive capabilities in learning dynamic environments and predicting future observations. Instead of merely generating synthetic training samples, world models could enable VI-ReID systems to reason about identity evolution across viewpoints, temporal changes, and sensor modalities. By modeling long-term pedestrian trajectories, behavioral consistency, and environmental context, world models may facilitate more reliable identity association under severe occlusion, missing observations, or long-term re-identification scenarios. Integrating predictive reasoning with identity representation learning therefore represents a promising research direction beyond conventional image matching.

Accuracy-aware Privacy Protection. Current cross-modal person re-identification datasets primarily rely on real-world captured images, raising concerns regarding identity exposure and privacy protection. As privacy regulations become increasingly stringent, developing privacy-preserving ReID systems without substantially compromising retrieval performance has become an important research direction. One promising direction is to jointly optimize identity discrimination and privacy preservation by learning repre- sentations that remain highly discriminative for identity matching while minimizing the leakage of sensitive personal information. For example, representation disentanglement could be employed to separate retrieval-relevant identity representations from privacy-sensitive visual information, while differential privacy may be incorporated during feature learning to reduce the memorization of in- dividual samples without destroying the relative feature structure required for retrieval. In addition, generative models could be leveraged to synthesize privacy-preserving training samples that retain identity consistency while concealing sensitive visual attributes. Achieving an effective balance be- tween privacy protection and identity discrimination will be crucial for the practical deployment of cross-modal ReID systems.

Next-Generation ReID Benchmarks Existing benchmarks have significantly advanced person re-identification by enabling standardized evaluation and fair comparison. However, they inevitably contain dataset-specific biases caused by sensing configurations, acquisition environments, and annotation strategies, making high performance not always indicative of real-world robustness. Future benchmarks should emphasize greater diversity in sensing devices, environments, annotation strategies, and modality sources. More comprehensive evaluation protocols, including cross-dataset testing, source-/device-disjoint evaluation, and robustness assessment under missing or degraded modalities, are also needed. These efforts can promote the learning of transferable identity representations beyond benchmark-specific correlations.

Identity-aware Foundation Models Vision-language foundation models, such as CLIP, BLIP, and GPT-4o, have demonstrated remarkable capabilities in cross-modal representation learning. However, these models are primarily optimized for semantic alignment rather than identity-level discrimination, which limits their direct applicability to person re-identification. Future research should investigate identity-aware adaptation strategies that preserve fine-grained identity cues while maintaining the strong semantic priors of foundation models. Possible directions include identity-centric prompt learning, modality-aware token interaction, instance-level contrastive objectives, and lightweight parameter-efficient adaptation. Such designs could enable foundation models to better distinguish visually similar individuals while maintaining robust cross-modal generalization.

Acknowledgements. This work is partially supported by National Natural Science Foundation of China under Grants (62302351,62501428,62376201,62306215), Postdoctoral Fellowship Program of CPSF under Grant Number (GZC20241268, 2024M762479), Hubei Provincial Natural Science Foundation of China (2025AFB219).

Conflict of interest statement. The authors declare that there are no conflict of interests, we do not have any possible conflicts of interest.

Data Availability. All datasets used in this paper are available. Market-1501 (Unimodal ReID) is available at https://www.kaggle.com/datasets/pengcw1/market-1501. MSMT17 is available at https://www.kaggle.com/datasets/achrafabid7/msmt17. SYSU-MM01 (VI-Reid) is available at https://github.com/wuancong/SYSU-MM01. RegDB (VI-Reid) is available at https://arxiv.org/abs/2001.04193. LLCM (VI-Reid) is available at https://github.com/ZYK100/LLCM. CUHK-PEDES (TI-Reid) is available at https://github.com/ShuangLI59/Person-Search-with-Natural-Language-Description. ICFG-PEDES (TI-Reid) is available at https://github.com/zifyloo/SSAN. RSTPReid (TI-Reid) is available at https://github.com/NjtechCVLab/RSTPReid-Dataset. PKU-Sketch (Sketch ReID) is available at https://www.pkuml.org/resources/pkusketchreid-dataset.html. CUFSF (Sketch ReID) is available at http://mmlab.ie.cuhk.edu.hk/archive/cufsf/. LReID and LReID-sync (NLOS ReID) are available at https://github.com/GWxuan/ReID3D. Wi-PER81 (NLOS ReID) is available at https://www.nature.com/articles/s41597-025-05804-0. Tri-CUHK-PEDES, Tri-ICFG-PEDES, and Tri-RSTPReid (Multimodal ReID) are available at https://github.com/ccq195/UNIReID. ORBench (Multimodal ReID) is available at https://github.com/zplusdragon/reid5o_orbench.

References↩︎