1 Introduction↩︎

Composed image retrieval (CIR) [1]–[7] offers a practical visual search interface: rather than describing a target image from scratch, a user provides a reference image and a relative instruction, and the system retrieves images that match the intended edit (see Fig. 1 (a)). Previous approaches [3], [8] are supervised and rely on human-annotated triplets whose large-scale collection is laborious and time-consuming. In addition, models trained on a particular dataset or domain often generalize poorly beyond their training distribution. These limitations motivate the zero-shot setting [9], [10], which studies CIR without domain-specific annotations and typically relies on unsupervised objectives or synthetic supervision to learn more transferable representations.

Recent zero-shot CIR methods [9]–[12] largely build on textual inversion over well-aligned vision–language models clip?, blip?, effectively reducing cross-modal composition to a text-only manipulation problem. Concretely, they learn a projector to transfer the reference image \(I_r\) into a small set of pseudo-text tokens and compose them with the relative instruction \(T_r\) in the text space to form a retrieval query, which is then matched against gallery image embeddings as shown in Fig. 1 (b). This paradigm rests on the key assumption that the semantics of an image can be faithfully captured by a few textual tokens and manipulated entirely within a textual space. However, this assumption is often impractical: compressing a rich reference image into only a handful of tokens inevitably loses fine-grained visual cues, so the composed representation may miss essential semantics needed for accurate retrieval. Moreover, learning an effective inversion projector often requires substantial computation, typically involving hours of large-batch contrastive training on multiple GPUs with web-scale image–text data [10], [11], [13]. These limitations motivate a more reliable and training-efficient mechanism for cross-modal composition beyond textual inversion.

In this paper, we reframe zero-shot CIR as a semantic transport problem between the reference and target distributions in the embedding space of vision–language models, and accordingly employ flow matching to bridge the two distributions. Building on this view, we introduce FlowCIR. Formally, FlowCIR learns a conditional transport that maps the relative-instruction text embedding to a target-oriented text embedding, conditioned on the reference image embedding. Different from prior methods that form retrieval queries via static token operations in text space, we perform composition as a continuous semantic transport through conditional flow matching, which better preserves semantics and enables a progressive, reference-guided composition process. This formulation provides a principled mechanism for cross-modal composition and is also training-efficient: we only learn a lightweight conditional velocity field, enabling training within about one hour on a single GPU. As illustrated in Fig. 2, our method achieves strong performance on both CIRR and CIRCO while incurring substantially lower training cost than prior baselines.

Meanwhile, as our transport is initialized from the instruction embedding, it may be influenced by a known weakness of CLIP-style VLMs: negation is often geometrically ambiguous in the text space [14], [15]. For example, expressions such as “a dog” and “remove/no dog” may lie undesirably close, providing an unreliable starting signal and biasing the transfer toward the negated concept. Therefore, inspired by [15], we propose an inference-only multi-negative steering strategy that explicitly pushes the composed instruction representation away from negative semantic directions, alleviating affirmative/negated collapse in CLIP space and improving retrieval on negation-heavy queries.

In summary, our contributions are threefold. (i) We introduce FlowCIR, a new paradigm for zero-shot composed image retrieval by conditional semantic transport in embedding space. (ii) We propose a multi-negative steering strategy that substantially improves robustness to negation and removal instructions, a prominent failure mode of CLIP-based composition. (iii) We achieve consistent improvements on standard CIR benchmarks, while remaining training-efficient.

2 Related Work↩︎

Zero-Shot Composed Image Retrieval. Existing ZS-CIR methods can be broadly grouped into three lines. (1) Textual-inversion based methods [9]–[12], [16] convert image information into learnable pseudo tokens, and then perform composition in the text embedding space for retrieval; representative works include Pic2Word [10] and SEARLE [9] and subsequent variants that design context-dependent mappings [12] or multiple pseudo words [17] to enhance controllability. (2) Generative-based methods introduce generative models (e.g, diffusion models) to synthesize target-oriented features [13] or leverage large-scale pretrained image generators (e.g, Stable Diffusion [18]) to produce pseudo target images as auxiliary visual evidence [19]. (3) MLLM-based training-free methods [20]–[22] leverage large vision–language models to obtain richer textual signals without task-specific training: they first caption the reference image with a VLM and then use an LLM/MLLM to parse, rewrite, or decompose the modification instruction and fuse it with the caption to form a composed query for retrieval [22]–[24], at the cost of extra inference-time computation.

In contrast to the above paradigms, we formulate composed image retrieval as conditional semantic transport and instantiate composition with conditional flow matching, which directly fuses the reference image evidence and relative instruction via a learned velocity field rather than token inversion, image synthesis, or inference-time prompting. Moreover, unlike DualCIR [20] and CoTMR [21] that leverage positive/negative semantics mainly by constructing separate textual queries and fusing image-text similarity scores, the Multi-Negative Steering aims to reshape the instruction embedding by steering it away from negative semantics.

Flow Matching (FM). FM learns a time-conditioned velocity field (a neural ODE) that transports samples along a predefined probability path from a simple prior to the data distribution by directly regressing the corresponding velocities [25]–[27]. Early studies [28], [29] instantiate this transport from noise to images, achieving high-fidelity synthesis on par with diffusion models [30] while retaining simpler training and sampling dynamics. More recent text-to-image FM variants [31], [32] further learn cross-modal transports from text embeddings to image embeddings, enabling direct language-to-vision generation through learned velocity fields. Beyond text-conditioned synthesis, FM has also been adapted to visual domain transfer problems such as monocular depth estimation [33] and semantic segmentation [34], [35], by formulating them as transport between distributions defined over different visual representations.

Generative Models for Perception. Beyond synthesis, modern generative models have become increasingly useful for visual recognition and perception tasks [36]–[43]. One prominent direction leverages generators as data engines, producing realistic and diverse samples to mitigate limited supervision and improve performance in fine-grained recognition [44], [45], few-shot learning [36], long-tailed classification [46], and category discovery [38]. In parallel, a complementary line uses the generative process for recognition itself: text-to-image diffusion models can be repurposed as zero-shot or few-shot classifiers by evaluating class-conditioned denoising likelihoods [39]–[41], with efficiency improved via hierarchical prompting [47] and auxiliary image encoders [42]. More recent studies further explore parameterizing few-shot adaptation through diffusion time steps to capture subtle attributes [48], and selecting or optimizing the injected noise to stabilize and strengthen the recognition pipeline [37]. FMA [49] and HFM [50] employ unconditional flow matching to bridge image and text within VLM embedding space for few-shot learning, and FlowComposer [51] employs flow matching for compositional zero-shot learning.

In contrast to the above methods, we are the first to explore conditional flow matching for composed image retrieval.

3 Method↩︎

3.1 Preliminaries↩︎

Flow Matching. Flow Matching [25], [26] aims to learn a continuous-time transport between two distributions (\(X_0 \rightarrow X_1\)) by regressing a time-dependent velocity field. Under the rectified flow formulation [27], a linear path is defined between paired samples \((\boldsymbol{x}_0\sim X_0,\boldsymbol{x}_1\sim X_1)\) as \(\boldsymbol{x}_t=(1-t)\boldsymbol{x}_0+t\boldsymbol{x}_1, t\in[0,1]\), whose ground-truth velocity is constant: \(\boldsymbol{v}^{\star}(\boldsymbol{x}_t,t)=\boldsymbol{x}_1-\boldsymbol{x}_0\). A neural velocity field \(\boldsymbol{v}_{\theta}(\boldsymbol{x}_t,t)\) is then trained by minimizing \[\mathcal{L}_{\mathrm{FM}}(\theta) = \mathbb{E}_{\boldsymbol{x}_0,\boldsymbol{x}_1,t} \Bigl[ \bigl\| \boldsymbol{v}_{\theta}(\boldsymbol{x}_t,t) - (\boldsymbol{x}_1-\boldsymbol{x}_0) \bigr\|_2^2 \Bigr],\] which is equivalent to matching the marginal velocity field in the original flow-matching formulation [25], [27]. Once trained, transport is performed by solving the ODE \(\frac{d\boldsymbol{x}_t}{dt}=\boldsymbol{v}_\theta(\boldsymbol{x}_t,t)\) from \(t=0\) to \(t=1\). Conditional flow matching extends this formulation by conditioning the velocity field on the condition \(\boldsymbol{c}\), yielding \[\mathcal{L}^{(\mathrm{FM})}_{\mathrm{cond}}(\theta) = \mathbb{E} \Bigl[ \bigl\| \boldsymbol{v}_\theta(\boldsymbol{x}_t,t,\boldsymbol{c}) - (\boldsymbol{x}_1-\boldsymbol{x}_0) \bigr\|_2^2 \Bigr].\] In our setting, we adopt this conditional formulation to transport the relative instruction embedding toward the target text embedding, conditioned on the reference image embedding.

Problem Setup and Notation. Given a gallery of candidate images \(\mathcal{D}=\{I_i\}_{i=1}^{N}\), zero-shot composed image retrieval (ZS-CIR) considers a query triplet \(\langle I_r, T_r, I_t\rangle\), where \(I_r\) is a reference image, \(T_r\) is a relative text instruction describing the desired modification to \(I_r\), and \(I_t\) is the target image satisfying the instruction. The goal is to compose the visual content of \(I_r\) with the semantics of \(T_r\) to retrieve \(I_t\) from \(\mathcal{D}\). In the zero-shot setting, training does not use CIR-specific triplets from the target benchmarks, but instead relies on external supervision such as large-scale image–text data or synthesized tuples.

In the following, given \(\langle I_r, T_r, I_t\rangle\), we denote a target-side text description associated with \(I_t\) as \(T_t\). We encode both images and texts into a shared VLM embedding space, and denote all embeddings by \(\boldsymbol{x}\). Using a pretrained image encoder \(E_I(\cdot)\) and text encoder \(E_T(\cdot)\), we obtain \(\ell_2\)-normalized features \(\boldsymbol{x}_{I_r}=E_I(I_r)/\|E_I(I_r)\|\), \(\boldsymbol{x}_{T_r}=E_T(T_r)/\|E_T(T_r)\|\), \(\boldsymbol{x}_{T_t}=E_T(T_t)/\|E_T(T_t)\|\), and \(\boldsymbol{x}_{I_t}=E_I(I_t)/\|E_I(I_t)\|\). Here, \(\boldsymbol{x}_{I_r}\), \(\boldsymbol{x}_{T_r}\), \(\boldsymbol{x}_{T_t}\), and \(\boldsymbol{x}_{I_t}\) denote the embeddings of the reference image, relative instruction, target-side text, and target image.

3.2 Conditional Flow Matching for Semantic Transport↩︎

We train in a zero-shot manner using synthesized compositions of the form \(\langle I_r, T_r, T_t, I_t\rangle\), where \(I_r\) is a reference image, \(T_r\) is a relative instruction, and \((T_t, I_t)\) describes the target side. Our model then learns a conditional semantic transport mapping \((\boldsymbol{x}_{T_r}, \boldsymbol{x}_{I_r}) \mapsto \boldsymbol{x}_{T_t}\) within the embedding space. We utilize conditional flow matching to model composition as semantic transport. Given a tuple and the extracted features, we sample a time scalar \(t\sim\mathcal{U}(0,1)\) and construct the interpolated state along a linear path as \(\boldsymbol{x}_t = (1-t)\boldsymbol{x}_{T_r} + t\,\boldsymbol{x}_{T_t}\). Our flow matching network \(f_\theta\) predicts a time-dependent velocity conditioned on the reference image \(\hat{\boldsymbol{v}}_\theta = f_\theta(\boldsymbol{x}_t,t,\boldsymbol{x}_{I_r})\), and is trained to match the ground-truth constant velocity \(\boldsymbol{v}^\star=\boldsymbol{x}_{T_t}-\boldsymbol{x}_{T_r}\) via the flow-matching regression objective: \[\mathcal{L}_{\mathrm{FM}} = \mathbb{E}\!\left[\left\|\hat{\boldsymbol{v}}_\theta(\boldsymbol{x}_t,t,\boldsymbol{x}_{I_r})-(\boldsymbol{x}_{T_t}-\boldsymbol{x}_{T_r})\right\|_2^2\right].\] Besides regressing the velocity, we adopt a contrastive retrieval objective where the predicted text embedding and the ground-truth image embedding \((\hat{\boldsymbol{x}}_{T_t}, \boldsymbol{x}_{I_t})\) form a positive pair, and we mine top-\(K\) hard negatives from the remaining target-image embeddings in the minibatch to sharpen discrimination against challenging distractors. We first form a predicted target text embedding \(\hat{\boldsymbol{x}}_{T_t}\) by a one-step transport from time \(t\) to \(1\): \(\hat{\boldsymbol{x}}_{T_t}=\boldsymbol{x}_t + (1-t)\,\hat{\boldsymbol{v}}_\theta(\boldsymbol{x}_t,t,\boldsymbol{x}_{I_r})\), and compute a sampled InfoNCE loss over the positive and the selected hard negatives. Specifically, we define the hard-negative index set as the top-\(K\) most similar candidates to the query (excluding the positive): \[\mathcal{H}_K(\hat{\boldsymbol{x}}_{T_t}) =\operatorname{TopK}\!\left( \left\{\, \Big\langle \tfrac{\hat{\boldsymbol{x}}_{T_t}}{\|\hat{\boldsymbol{x}}_{T_t}\|},\, \boldsymbol{x}^{\,j}_{I_t}\Big\rangle \,\right\}_{j\neq i} \right),\] where \(\operatorname{TopK}(\cdot)\) returns the indices of the \(K\) largest similarities in the minibatch. The resulting top-\(K\) InfoNCE objective is \[\mathcal{L}_{\mathrm{RET}} = -\log \frac{\exp\!\left(\big\langle \tfrac{\hat{\boldsymbol{x}}_{T_t}}{\|\hat{\boldsymbol{x}}_{T_t}\|},\, \boldsymbol{x}_{I_t}\big\rangle/\tau\right)}{\exp\!\left(\big\langle \tfrac{\hat{\boldsymbol{x}}_{T_t}}{\|\hat{\boldsymbol{x}}_{T_t}\|},\, \boldsymbol{x}_{I_t}\big\rangle/\tau\right) +\sum_{j\in\mathcal{H}_K(\hat{\boldsymbol{x}}_{T_t})}\exp\!\left(\big\langle \tfrac{\hat{\boldsymbol{x}}_{T_t}}{\|\hat{\boldsymbol{x}}_{T_t}\|},\, \boldsymbol{x}^{\,j}_{I_t}\big\rangle/\tau\right)},\] where \(\boldsymbol{x}_{I_t}^{\,j}\) denotes the target-image embedding of the \(j\)-th sample in the minibatch, \(\tau\) is a temperature scalar, and \(\mathcal{H}_K(\hat{\boldsymbol{x}}_{T_t})\) indexes the \(K\) hardest negatives. The overall training objective is \(\mathcal{L}=\mathcal{L}_{\mathrm{FM}}+\lambda\,\mathcal{L}_{\mathrm{RET}}\), where \(\lambda\) balances the learning of FM and retrieval. During inference, we adopt the same one-step transport scheme for efficiency: predicting the velocity \(f_\theta(\boldsymbol{x}_{T_r},\boldsymbol{x}_{I_r},0)\) yields direct one-step transport to \(\hat{\boldsymbol{x}}_{T_t}\), which is then used to retrieve images by nearest-neighbor search in the gallery embedding space.

3.3 Multi-Negative Steering Strategy↩︎

Negation-heavy instructions (e.g., “no dog”, “remove stripes”) often induce ambiguous geometry in contrastive text spaces, where affirmative and negated descriptions can collapse to nearby directions [14]. To address this issue without additional training, we propose an inference-only multi-negative steering strategy. During inference, given a relative instruction \(T_r\), we decompose it into an affirmative intent \(T_r^{a}\) and a set of negated concepts \(\{T_{r,k}^{n}\}_{k=1}^{K}\) using a lightweight parser (e.g, a small LLM with simple rule-based heuristics), and encode them as normalized embeddings \(\boldsymbol{x}_{T_r^{a}}=E_T(T_r^{a})/\|E_T(T_r^{a})\|\) and \(\boldsymbol{x}_{T_{r,k}^{n}}=E_T(T_{r,k}^{n})/\|E_T(T_{r,k}^{n})\|\). Our goal is to construct a steered instruction embedding \(\hat{\boldsymbol{x}}_{T_r}\) that remains aligned with \(\boldsymbol{x}_{T_r^{a}}\) while being repelled from all \(\{\boldsymbol{x}_{T_{r,k}^{n}}\}_{k=1}^{K}\).

Following SpaceVLM [15], we model each concept as a hyper-spherical cap in the embedding space: for a normalized point \(\boldsymbol{x}\) and a cosine threshold \(\delta\in[-1,1]\), the cap region is defined as \(\mathcal{R}(\boldsymbol{x})=\{\boldsymbol{z}\in\mathbb{R}^d \mid \boldsymbol{x}^\top \boldsymbol{z}\ge \delta\}.\) Accordingly, a feasible negation-aware representation should lie close to the affirmative intent but outside the neighborhoods induced by negatives, yielding \[\mathcal{R}(T_r)=\mathcal{R}(\boldsymbol{x}_{T_r^{a}})\cap\bigcap_{k=1}^{K}\mathcal{R}^c(\boldsymbol{x}_{T_{r,k}^{n}}),\] where \(\mathcal{R}^c(\cdot)\) denotes the complement. To obtain a single embedding compatible with our transport model, we extend this construction by computing, for each negative \(\boldsymbol{x}_{T_{r,k}^{n}}\), a closed-form “center” direction: \[\tilde{\boldsymbol{d}}_k = \boldsymbol{x}_{T_r^{a}}\frac{\sin\!\left(\alpha+\tfrac{\theta_k}{2}\right)}{\sin\theta_k} - \boldsymbol{x}_{T_{r,k}^{n}}\frac{\sin\!\left(\alpha-\tfrac{\theta_k}{2}\right)}{\sin\theta_k}, \qquad \boldsymbol{d}_k=\frac{\tilde{\boldsymbol{d}}_k}{\|\tilde{\boldsymbol{d}}_k\|},\] where \(\alpha=\arccos(\delta)\) is set by a cosine-margin hyperparameter \(\delta\in[-1,1]\) controlling the hyper-spherical cap region size, and \(\theta_k=\arccos(\boldsymbol{x}_{T_r^{a}}^\top \boldsymbol{x}_{T_{r,k}^{n}})\). When \(\theta_k\) is very small, we use \(\tilde{\boldsymbol{d}}_k=\boldsymbol{x}_{T_r^{a}}-\boldsymbol{x}_{T_{r,k}^{n}}\) for numerical stability. We then aggregate and normalize these directions to form the steered instruction embedding: \[\hat{\boldsymbol{x}}_{T_r} = \frac{1}{K}\sum_{k=1}^{K}\boldsymbol{d}_k, \qquad \hat{\boldsymbol{x}}_{T_r}\leftarrow \frac{\hat{\boldsymbol{x}}_{T_r}}{\|\hat{\boldsymbol{x}}_{T_r}\|},\] and replace the original instruction embedding with \(\hat{\boldsymbol{x}}_{T_r}\) in our transport pipeline, such that the composed representation remains aligned with the affirmative semantics while being repelled from all negative directions in the embedding space.

3.4 Theoretical Justification↩︎

We provide a theoretical justification for applying conditional flow matching for zero-shot CIR. Formally, standard flow matching is designed for distribution-level transport, where reaching any valid target mode is sufficient, whereas composed image retrieval requires instance-specific semantic alignment: for a given pair \((x_{I_r},x_{T_r})\), the model should recover its corresponding target \(x_{T_t}\) rather than an arbitrary mode. Below, we analyze (1) why conditioning on the reference image is necessary, and (2) when the learned conditional velocity yields exact point-to-point semantic transport.

Ambiguity of Unconditional Flow. In realistic composed retrieval data, a single relative instruction \(x_{T_r}\) (e.g, Make the dog look up at the sky) can correspond to multiple valid targets \(\{I_t^k\}_{k=1}^K\). As a result, without conditioning on the reference image, the same source instruction embedding \(\boldsymbol{x}_{T_r}\) is associated with a multi-modal conditional target distribution \(p(\boldsymbol{x}_{T_t}\mid \boldsymbol{x}_{T_r})\).

Proposition 1 (Mean Collapse). For an unconditioned flow, the marginal velocity field \({v}^\star(x_t)\) deterministically transports \(x_{T_r}\) to the semantic expectation: \[x_{T_r} + \int_0^1 {v}^\star(x_t) \mathrm{d}t = \mathbb{E}[x_{T_t}|x_{T_r}] \label{eq:mean95collapse}\tag{1}\] Therefore, when \(p(\boldsymbol{x}_{T_t}\mid \boldsymbol{x}_{T_r})\) is multi-modal, \(\mathbb{E}[x_{T_t}|x_{T_r}] \notin \{x_{T_t}^1, \dots, x_{T_t}^K\}\), which induces a mean-collapse tendency and may prevent recovery of the correct target.

Semantic Transport via Conditioning. To disambiguate the transport direction, we condition the transport on the reference image embedding \(\boldsymbol{x}_{I_r}\). We construct the conditional probability path to establish exact point-to-point semantic transport as: \[p_t(x_t|x_1 = x_{T_t}^i, x_{I_r}^i) = \begin{cases} \delta(x_t - x_{T_r}^i), & t = 0 \\ \delta(t x_{T_t}^i + (1 - t)x_{T_r}^i), & t \in (0,1] \end{cases} \label{eq:conditional95path}\tag{2}\] Subsequently, a corresponding velocity \(\boldsymbol{v}^\star(\boldsymbol{x}_t|\boldsymbol{x}_{T_t}^i, \boldsymbol{x}_{I_r}^i)\) is defined to generate this path \(p_t\), and we introduce the following proposition:

Proposition 2 (Semantic Transport). Under above assumption, the marginal velocity field is mathematically identical to the exact conditional velocity field: \[\boldsymbol{v}^{\star}(\boldsymbol{x}_t,t,\boldsymbol{x}_{I_r}) = \boldsymbol{v}^\star(\boldsymbol{x}_t,t|\boldsymbol{x}_{I_r}^i, \boldsymbol{x}_{T_t}^i) \label{eq:marginal95identity}\tag{3}\] Consequently, integrating this velocity along the path yields exact point-to-point semantic transport from \(x_{T_r}^i\) strictly towards its corresponding target \(x_{T_t}^i\): \[\boldsymbol{x}_{T_t} = \boldsymbol{x}_{T_r} + \int_0^1 {\boldsymbol{v}}^{\star}(\boldsymbol{x}_t,t,\boldsymbol{x}_{I_r})\,\mathrm{d}t. \label{eq:exact95semantic95transport}\tag{4}\] We provide the detailed proof in the supplementary material. This suggests that, under the idealized conditional path assumption, reference conditioning can reduce multimodal ambiguity and support point-to-point semantic alignment.

4 Experiments↩︎

4.1 Setups and Implementations↩︎

Datasets. Following prior zero-shot CIR works [9], [10], [52], we evaluate on three standard benchmarks: CIRR [5], CIRCO [9] and Fashion-IQ [53]. CIRR consists of real-world image pairs with relative captions describing the desired transformation from a reference to a target. CIRCO further scales composed retrieval to diverse everyday objects and scenes with more challenging distractors. Fashion-IQ is an e-commerce fashion benchmark, where queries pair a product image with a brief modification text to retrieve the edited item.

Implementation Details. Following prior zero-shot CIR works, we use CLIP clip? as the frozen vision–language backbone and report results with two variants: ViT-B/32 and ViT-L/14. For the flow matching model, we adopt the lightweight network design from MAR [54], implemented as a deep residual MLP with timestep conditioning. We train our model on the HQ-Edit-200k [55] synthetic image-editing dataset, which contains reference images paired with editing instructions, the corresponding edited results, and associated textual descriptions, matching the supervision signals required by our transport-based formulation. For multi-negative steering, we first apply a rule-based strategy to identify negation-related instructions by checking whether the query contains keywords such as no, without, remove, and replace. For the detected cases, we further use a lightweight language model, TinyLlama-1.1B-Chat-v1.0 [56], to decouple the negative component from the original instruction, which is then used to construct the additional negative steering signal at inference time. Unless otherwise specified, all experiments are conducted on a single NVIDIA RTX 3090 GPU with \(24\)GB memory. Further details on datasets, optimization, and hyperparameters are deferred to the supplementary material.

4.2 Main Results↩︎

4pt

4pt

Tab. 1 and Tab. ¿tbl:tab:fashioniq? compare FlowCIR with both textual-inversion-based methods (including Pic2Word [10], SEARLE [9], LinCIR [11] and iSEARLE [52]) and generative-based methods (including Compo-Diff [13] and CIG [19]) on CIRR [5], CIRCO [9], and Fashion-IQ [53] under CLIP ViT-B/32 and ViT-L/14 backbones, together with training cost. Overall, our method achieves the strongest or highly competitive performance across benchmarks while requiring substantially less training than prior methods. For completeness, we also include comparisons with methods relying on substantially stronger external resources, including MagicLens [58], which is trained on 36.7M private triplets with several TPUs, and MCL [24], which utilizes an external LLM.

CIRR & CIRCO. On the more challenging CIRR and CIRCO benchmarks, FlowCIR achieves the strongest or highly competitive performance among prior public-data textual-inversion and generative ZS-CIR baselines, while remaining competitive with methods that rely on substantially stronger external resources. With CLIP ViT-B/32, our method improves over the strongest competing baseline by 1.4% on CIRR R@5 (\(55.7\rightarrow 56.5\)), and 23.6% (\(10.6\rightarrow 13.1\)) on CIRCO mAP@5. The gains become more evident under CLIP ViT-L/14: compared with the previous best results, FlowCIR improves CIRR R@1 by 2.3% (\(25.6\rightarrow 26.2\)) and CIRCO mAP@5 by 14.6% (\(13.0\rightarrow 14.9\)). Notably, these improvements are obtained with only 0.5 hours of training on a single GPU, compared with the reported multi-GPU training costs of prior inversion-based methods, and dramatically higher costs for generative approaches, which are typically reported on substantially larger \(80\)GB GPUs. This confirms that our flow-based conditional semantic transport is not only more accurate than existing textual-inversion and generative baselines, but also markedly more training-efficient.

Fashion-IQ. Tab. ¿tbl:tab:fashioniq? further demonstrates that FlowCIR generalizes to the e-commerce setting of Fashion-IQ [53]. Under CLIP ViT-B/32, FlowCIR improves the best prior method by +2.2% (\(22.9\rightarrow 23.4\)) in average R@10 and +2.9% (\(42.0\rightarrow 43.2\)) in average R@50. Under CLIP ViT-L/14, it yields a +6.8% (\(26.3\rightarrow 29.7\)) gain in average R@10 and matches the best prior R@50. Although the relative gains are smaller than those on CIRR/CIRCO, we attribute this to the domain-specific and fine-grained nature of Fashion-IQ, where many queries focus on subtle local fashion attributes and thus leave less room for semantic transport to yield larger improvements. Overall, FlowCIR remains consistently competitive and attains the best average performance.

4.3 Diagnostic Analysis↩︎

Component Analysis. We conduct ablations to quantify the contributions of the two key components in FlowCIR: Conditional Flow Matching (CFM) and inference-time Multi-Negative Steering (Neg-Steering). As shown in Tab. ¿tbl:tab:ablation?, we report results on CIRR and CIRCO by progressively adding these components on top of a text-only baseline. First, the text-only baseline provides only limited performance, and adding the multi-negative steering alone on top of the textual instruction (Row (1)) does not constitute a reliable solution. Although it brings a slight improvement on CIRR, it degrades performance on CIRCO, suggesting that refining the text-side instruction alone is insufficient for accurate cross-modal retrieval. Second, introducing Conditional Flow Matching (Row (2)) yields a substantial improvement. This clearly demonstrates the advantage of modeling composition as semantic transport: rather than directly manipulating text features, the model learns a reference-conditioned flow that transports the instruction representation toward target semantics, resulting in much stronger cross-modal composition. The particularly large gains on CIRCO, together with the consistent improvements on CIRR, confirm that flow matching is an effective mechanism for composing visual and textual signals in zero-shot retrieval. Third, adding Multi-Negative Steering on top of CFM leads to further and consistent gains, yielding the full FlowCIR model with the best overall performance. These results show that CFM is the main driver of performance, highlighting the effectiveness of flow-based semantic transport for cross-modal composition, while Multi-Negative Steering is complementary and further improves robustness by reducing ambiguity caused by negated semantics.

Hyperparameter analysis. Table ¿tbl:tab:hyperparam? studies two important hyperparameters in FlowCIR, tuned on a held-out validation split: the weight \(\lambda\) of the retrieval objective and the hard-negative ratio \(K\), where \(K\) is defined as the proportion of mined negatives relative to the minibatch size (\(B\)). Overall, the method is stable across a broad range of settings, but moderate values consistently work best. For \(\lambda\), using a very small value (e.g, \(\lambda=0.1\)) leads to weaker CIRCO performance, indicating insufficient retrieval supervision, while overly large values do not bring further gains. The best overall trade-off is achieved around \(\lambda=0.5\), which yields the strongest or near-strongest results on both CIRR and CIRCO. For hard-negative mining, selecting too few negatives (e.g, \(K=\frac{1}{16}B\)) or too many negatives (e.g, \(K=\frac{1}{4}B\)) is suboptimal, whereas a moderate choice \(K=\frac{1}{8}B\) gives the best overall performance. This suggests that an appropriate number of hard negatives is important: too few fail to provide sufficient discriminative pressure, while too many may introduce noisy distractors.

Flows vs. Regressor. We further analyze the rationale of our flow matching design by comparing it with a direct regressor variant that uses the same model architecture and the same reference-image conditioning, but directly regresses the target residual instead of learning a time-dependent transport field. As shown in the bottom part of Table ¿tbl:tab:hyperparam?, replacing flow matching with direct regression leads to a clear performance drop across all metrics, especially on CIRCO, where mAP@5 decreases from \(14.8\) to \(12.3\) on the test set. In contrast, the flow-matching model consistently achieves the best results on both validation and test splits. This comparison shows that learning a time-conditioned transport field is more effective and has stronger generalization ability than directly regressing the final embedding, likely because flow matching provides a more structured optimization target and better captures the semantic transition from instruction to target under the reference condition.

Transport Target Space Analysis. We additionally study a key design choice in FlowCIR: whether the transport process should target the image embedding space or the text embedding space. As shown in Tab. ¿tbl:tab:target95space95analysis?, transporting toward the target text embedding consistently outperforms transporting toward the target image embedding on both CIRR and CIRCO. This comparison is closely related to prior generative-based methods such as Compo-Diff [13], which aim to conditionally generate target image embeddings. In contrast, our goal is semantic transport: given the reference image and relative instruction, we transport the query toward the target text embedding and then use this transported representation for retrieval. We favor the text space for two reasons. First, composed retrieval is inherently one-to-many, so supervising the model with a specific target image embedding may over-emphasize instance-level visual details, whereas target text embeddings provide a more semantics-centered signal. Second, transporting from relative text to target text forms a more homogeneous mapping, which is easier to learn than a cross-space text-to-image transport. These results support our design choice of modeling zero-shot CIR as text-side semantic transport rather than direct prediction in the image space.

4.4 Qualitative Analysis↩︎

Effect of reference-conditioned transport. Fig. 4 illustrates that our method effectively composes cross-modal signals while preserving reference-image semantics, even under the same relative instruction. In the first two examples, the instruction “make the dog look up at the sky” yields different retrieval behaviors depending on the reference: when the reference is a bulldog-like dog, the retrieved results largely preserve its breed and appearance while changing the pose; when the reference is a fluffy white dog, the retrieved images remain visually consistent with that appearance and mainly modify the head orientation. A similar phenomenon appears in the third and fourth examples with the instruction “the outfit of the lady has changed from blue to red,” where the retrieved results preserve the person’s pose and cloth style while modifying the clothing attribute. In the fourth example, although there may be no exact target in the gallery, the retrieved images still remain close in clothing style and visual appearance, further showing that our model effectively combines the reference visual cues with the textual modification. Overall, these examples show that FlowCIR enables effective cross-modal composition by jointly preserving the intended textual modification and the reference-image semantics; even under the same instruction, different reference conditions lead to distinct retrieval results.

Effect of multi-negative steering. The bottom examples in Fig. 4 highlight the benefit of Multi-Negative Steering on negation-heavy instructions. Without steering, retrieval often remains biased toward the negated concepts, reflecting the well-known difficulty of CLIP-style embeddings in separating affirmative and negated semantics. For instance, for “remove the toy and zoom in on the dog’s face while it is looking left with a red collar on”, the unsteered results still often contain distracting toys or irrelevant objects, whereas applying Multi-Negative Steering yields more dog-face-centered results and better respects the intended removal. Likewise, for “remove the snow and add a man holding the dogs with leashes,” the unsteered retrieval remains dominated by snowy scenes, while the steered version more consistently retrieves images with a human, leashes, and non-snowy environments. These examples show that Multi-Negative Steering effectively pushes the composed representation away from undesired semantic directions, alleviating negation collapse and improving retrieval faithfulness.

5 Conclusion↩︎

In this paper, we introduce FlowCIR, a new paradigm for zero-shot composed image retrieval that formulates composition as conditioned semantic transport via flow matching in the embedding space. Instead of reducing cross-modal composition to static token operations in text space, our method learns a lightweight conditional velocity field that transports the instruction representation toward a target-aligned query embedding under the guidance of the reference image. We further propose an inference-only Multi-Negative Steering strategy that steers a negation-containing relative instruction away from its negated semantics, mitigating the limited negation handling of VLMs and improving robustness on negation-heavy queries. Extensive experiments on three common benchmarks demonstrate that FlowCIR achieves highly competitive performance among recent ZS-CIR methods, while offering a lightweight training protocol. We hope this work can inspire future research on flow-based compositional retrieval and open up a new paradigm for cross-modal retrieval beyond static composition.

Acknowledgments. This work was supported by National Natural Science Foundation of China (NSFC) Young Scientists Fund Category C (62402408), National Natural Science Foundation of China (NSFC) Young Scientists Fund Category B (62522216), Hong Kong SAR Research Grants Council (RGC) Early Career Scheme (26208924), and Hong Kong SAR Research Grants Council (RGC) General Research Fund (16219025).

References↩︎