1 Introduction↩︎

Remote sensing change detection (CD) aims to identify surface changes from multi-temporal imagery acquired over the same geographic area, serving as an important technology for urban planning, environmental monitoring, disaster response, and resource management [1], [2]. Over the past decade, the rapid growth of both Earth observation data and deep learning methods has propelled CD into one of the most active research frontiers in remote sensing.

The predominant formulation of CD is binary change detection (BCD), which classifies each pixel as either changed or unchanged. A large number of BCD benchmarks have been established, spanning diverse geographic contexts, spatial resolutions, and change categories (see Table ¿tbl:tab:datasets? for a comprehensive summary) [3]–[19]. While the vast majority of these benchmarks adopt the bi-temporal setting and have driven extensive algorithmic development [10], [11], [20]–[24], the community has also explored single-temporal CD that exploits pretrained semantic representations to infer changes from unpaired imagery [25]–[27], as well as multi-temporal monitoring that tracks continuous change trajectories from dense satellite time series [28]–[31]. Despite these differences, all of the above approaches produce pixel-level masks that encode where change occurred but remain silent on what changed or why.

max width= Abbreviations: Opt.= Optical RGB; MS = Multispectral; HS = Hyperspectral; Cross = Cross-modal; Multi = Multi-modal fusion; GE = Google Earth; GF-2 = Gaofen-2; JL-1 = Jilin-1; S1/S2 = Sentinel-1/2; PV = Photovoltaic; Bi = Bi-temporal; Bldg.= Building(s); Veg.= Vegetation; VHR = Very High Resolution (\(<\)1 m). Dataset names are hyperlinked to download pages where available.

In parallel, the range of input modalities has expanded considerably. While optical RGB imagery remains the dominant data source for CD research [7], [10], [13], [15], [19], multispectral and hyperspectral sensors offer richer spectral discrimination for fine-grained change analysis [5], [8], [9], [32]–[35], and cross-modal fusion of optical and SAR data enables all-weather, day-and-night monitoring [14], [17], [36]–[40]. Although these modalities provide richer visual information, the output of BCD remains a pixel-level binary mask devoid of semantic content.

Semantic change detection (SCD) and building damage assessment (BDA) partially bridge this gap by assigning categorical labels to changed pixels. SCD introduces per-pixel land-cover annotations at both time phases, enabling identification of change types (e.g., farmland\(\to\)buildings) [30], [41]–[50]. BDA further introduces ordinal damage scales (e.g., minor/major/destroyed) for disaster response [51]–[57]. Nevertheless, the semantic content in these datasets is still encoded as discrete numerical class indices drawn from a closed taxonomy. This gap calls for a shift toward natural-language-grounded change understanding.

The advent of vision–language models (VLMs) has begun to close this gap. In the natural image domain, change captioning benchmarks first demonstrated the feasibility of describing visual differences in natural language [58], [59]. The remote sensing community quickly adopted this paradigm, producing a growing family of change captioning (CC) datasets that range from low-resolution multitemporal pairs to very-high-resolution urban and disaster scenes [60]–[66]. In parallel, change detection visual question answering (CDVQA) has emerged as a complementary task, enabling users to query specific aspects of surface changes through natural-language questions [67], [68], while instruction-tuning datasets have further extended the interaction to multi-turn conversational analysis [69]–[71]. Despite this rapid progress, rare benchmarks jointly provide binary change masks, change captions, and change-oriented question–answer pairs on the same set of image pairs, which limits multi-task learning and cross-task evaluation.

The evolution of datasets has been accompanied by corresponding advances in model architectures (see Fig. 1). Methods built on CNN backbones with change decoders [10], [11], [20], [21], [72]–[93] established the dominant Siamese paradigm, while Transformer [22], [23], [33], [94]–[104] introduced global cross-temporal attention, and Mamba-based methods [24], [40], [105]–[111] achieved comparable perception at linear computational cost. Foundation model adaptations have enabled zero-shot and few-shot CD by transferring pretrained CLIP and SAM representations [112]–[120]. Generative approaches based on GANs [37], [121]–[123] and diffusion models [124]–[130] have addressed data scarcity through synthetic bi-temporal image generation. Agent-based systems have integrated LLMs as reasoning engines for interactive change analysis [131]–[134]. A critical observation is that the text modality has been a key enabler at each stage: CLIP-based models use text prompts to guide change semantics, captioning models translate visual change into language, VQA models support interactive querying, and agents conduct multi-step reasoning in natural language. This trajectory underscores the need for language-grounded benchmarks that can support the training and evaluation of these increasingly capable architectures.

Motivated by the above analysis, we present JL1-CC&QA, extending the JL1-CD benchmark [19], which consists of 5,000 bi-temporal image pairs from the Jilin-1 satellite with a resolution of 0.5–0.75 m, with two new annotation layers: (i) JL1-CC, providing 17,021 quality-verified change captions spanning both anthropogenic and natural changes; and (ii) JL1-QA, offering 20,060 question–answer pairs across eight question types (existence, description, location, magnitude, temporal comparison, causation, relative comparison, and visual detail). All annotations are produced via a three-stage pipeline: multi-modal LLM generation, vision-grounded LLM judging, and human expert verification. Our main contributions are summarized as follows:

1. We construct a change understanding benchmark that provides binary change masks, natural-language change descriptions, and diverse QA pairs over 5,000 bi-temporal satellite image pairs, facilitating joint training and evaluation across CD, CC, and QA tasks.

2. We design a scalable, reproducible annotation pipeline that couples multi-modal LLM generation with vision-grounded LLM judging and human verification.

3. We provide comprehensive dataset statistics, and release all data and code to support community research toward unified remote sensing change understanding.

2 Dataset Construction↩︎

This section describes the construction of JL1-CC&QA. We first introduce the source dataset JL1-CD (Section 2.1), then detail the change captioning pipeline (Section 2.2) and the question answering pipeline (Section 2.3).

2.1 JL1-CD↩︎

JL1-CC&QA is built upon JL1-CD [19], which comprises 5,000 bi-temporal image pairs captured by the Jilin-1 high-resolution optical satellite. The imagery was acquired across multiple provinces in China, including Shandong, Ningxia, Anhui, Hebei, and Hunan, between early 2022 and late 2023, and was carefully curated to exclude blur, cloud occlusion, and extreme illumination conditions. Each image pair consists of a pre-event image, a post-event image, and a pixel-level binary change mask annotated by professional interpreters. Key specifications are summarized in Table 1.

Two properties of JL1-CD make it suitable for change captioning and question answering. First, the dataset is all-inclusive in terms of change types: it encompasses both anthropogenic changes (buildings, roads, hardened surfaces, photovoltaic panels, etc.) and natural changes (woodlands, grasslands, croplands, water bodies, etc.). This diversity ensures that the derived CC and QA annotations cover a broad spectrum of real-world surface dynamics. Second, the change area ratio (CAR)—defined as the proportion of changed pixels in each image pair—spans the full range from near-zero to 100%, with a mean of 9.7% and a median of 2.9% (Fig. 2). This long-tailed distribution mirrors the natural imbalance of real-world change scenarios and poses a meaningful challenge for language-grounded change understanding at varying scales.

2.2 JL1-CC↩︎

2.2.1 Task Definition↩︎

Given a bi-temporal image pair \((I_A, I_B)\) and its corresponding binary change mask \(M\), the change captioning task requires generating a set of natural-language sentences \(\{c_1, c_2, \ldots, c_k\}\) that accurately describe the semantic content of the observed surface changes, including the type, location, and extent of change.

2.2.2 Annotation Pipeline↩︎

As illustrated in Fig. 3, the JL1-CC annotation pipeline consists of three stages.

Stage 1: Multi-modal LLM Generation. For each image pair, we prompt a multi-modal LLM (Kimi-K2.6) with three visual inputs: the pre-event image \(I_A\), the post-event image \(I_B\), and the binary change mask \(M\), together with spatial metadata including the CAR and a textual description of the primary change region (e.g., “upper-left area of the image”). The model is instructed to generate five diverse captions, each describing the same change from a different perspective: change type, spatial location, visual appearance, scale, or implication.

Stage 2: Vision-Grounded LLM Judging. A second LLM call evaluates each caption by examining the original image pair alongside the generated text. Each caption is scored on a 1–10 integer scale across five criteria: (1) accuracy: whether the description matches the visible changes, with heavy penalties for hallucination; (2) specificity: whether concrete land-cover terms are used; (3) spatial correctness: whether location references are accurate; (4) naturalness: whether the English is fluent; and (5) informativeness: whether the caption conveys meaningful detail. The top-3 scoring captions are retained, and ties at the cutoff are preserved.

Stage 3: Human Expert Verification. A subset of the generated captions is reviewed by domain experts to verify factual accuracy and identify systematic errors, ensuring that the automated pipeline produces reliable annotations at scale.

2.2.3 Statistics↩︎

Table 2 summarizes the JL1-CC dataset statistics. The pipeline generates 25,000 candidate captions (5 per pair) and retains 17,021 after quality filtering, yielding a pass rate of 68.1%. The judge score distribution (Fig. 4) shows clear discrimination: 39.8% of captions score 9–10, 42.3% score 7–8, and 17.9% score below 7 and are rejected. The selected captions have a mean length of 26.2 words, with a vocabulary of 7,458 unique tokens. Fig. 5 presents a word cloud of the selected captions, where the most prominent terms—upper, lower, bare soil, agricultural, building, road, water—reflect both the spatial referencing style and the diversity of land-cover types in JL1-CD.

2.3 JL1-QA↩︎

2.3.1 Task Definition↩︎

Given a bi-temporal image pair \((I_A, I_B)\) and a natural-language question \(Q\), the change question answering task requires generating a textual answer \(A\) that accurately responds to the question based on the visible surface changes.

2.3.2 Question Taxonomy↩︎

We define eight question types adapted for open-ended answer generation, as summarized in Table 3. Each type targets a distinct aspect of change understanding, ranging from binary existence judgments to causal reasoning, ensuring comprehensive coverage of user information needs.

2.3.3 Annotation Pipeline↩︎

As illustrated in Fig. 6, the JL1-QA annotation pipeline shares the same three-stage architecture as JL1-CC, with two key enhancements in the generation stage.

Stage 1: Context-Enriched QA Generation. Beyond the three visual inputs (\(I_A\), \(I_B\), \(M\)), the LLM additionally receives contextual metadata from JL1-CC: the change area ratio, the spatial change region description, and up to three selected change captions. This context enrichment grounds the QA generation in verified change descriptions, yielding more accurate and diverse question–answer pairs. The model generates five QA pairs per image, each covering a different question type randomly sampled from the taxonomy. Crucially, the prompt explicitly prohibits copying exact numerical metadata (e.g., change percentages) into answers, as such precision cannot be derived from visual inspection alone.

Stage 2: Multi-Criteria Judging. Each QA pair is scored on a 1–10 integer scale across four criteria: (1) answer accuracy: whether the answer is factually correct given the images; (2) question quality: whether the question is clear, natural, and non-trivial; (3) answer completeness: whether the answer adequately addresses the question; and (4) redundancy: whether the QA pair is substantially different from the others for the same image. QA pairs scoring below 7 are discarded. The most common rejection reasons are hallucinated precise percentages (score 1–4), redundancy with other QA pairs (score 5–6), and vague answers (score 5–6).

Stage 3: Human Expert Verification. As with JL1-CC, domain experts review a subset of selected QA pairs to verify factual accuracy and check for systematic errors in question formulation or answer content.

2.3.4 Statistics↩︎

Table 4 summarizes the JL1-QA dataset statistics. From 24,995 generated QA pairs, 20,060 pass the quality threshold (score \(\geq\) 7), yielding a pass rate of 80.3%. The higher pass rate compared to JL1-CC (68.1%) is attributed to the context enrichment from change captions, which reduces hallucination in the generation stage. The judge score distribution (Fig. 7) confirms effective quality discrimination: 51.4% of QA pairs score 9–10, 28.4% score 7–8, and 20.2% are rejected. The question type distribution (Fig. 8) shows broad coverage across all eight categories, with YES/NO (21.2%), WHERE (18.3%), and WHAT (16.9%) being the most frequent. Questions average 11.1 words and answers 19.3 words in length.

3 Conclusion↩︎

In this paper, we presented JL1-CC&QA, a multi-task benchmark that extends the JL1-CD binary change detection dataset with two complementary annotation layers: change captioning (JL1-CC) and change question answering (JL1-QA). Both layers are produced through a three-stage pipeline—multi-modal LLM generation, vision-grounded LLM judging, and human expert verification—balancing scalability with factual reliability. We hope this benchmark serves as a valuable resource for the community to advance multi-task change understanding in remote sensing.

References↩︎