Keywords: automated species recognition; coral reef ecology; cross-habitat generalisation; decision framework; few-shot learning; marine biodiversity monitoring; underwater imaging; vision foundation models

1 Introduction↩︎

Coral reef monitoring programmes along the Great Barrier Reef generate upwards of 100 000 images per annual survey, each requiring expert examination to identify and count the organisms present [1]. A single trained annotator processes roughly 50–100 images per hour for species-level identification, with inter-observer agreement often below 80% even among experts [2]; scaling such effort to continental monitoring programmes demands thousands of expert-hours that most research groups cannot afford [3]. Automated species recognition based on deep learning can close this gap, and classification accuracies exceeding 90% have been reported on individual marine datasets [4], [5]. Yet these systems share a critical limitation: a model trained at one site rarely works at another without extensive relabelling, and ecologists deploying automated recognition at a new site lack evidence-based guidance on a basic operational question: how many labelled images from this new site do we actually need?

To answer this question, we construct a benchmark that spans five marine datasets across three oceans, from tropical reefs to temperate fjords, and systematically evaluate how modern vision foundation models transfer across these habitats (Figure 1).

The Cross-Habitat Transfer Challenge in Marine Systems↩︎

Underwater imagery is collected at scale through diverse platforms: baited remote underwater video stations (BRUVS), diver-operated cameras, towed camera sleds, remotely operated vehicles, and fixed camera arrays [6]. Coral reef benthic surveys typically photograph quadrats at fixed transect points, generating hundreds of images per dive site [7]. The bottleneck is not collection but annotation, and existing automated tools such as CoralNet require site-specific training sets of several thousand labelled points before achieving acceptable accuracy [3].

Cross-habitat transfer compounds the problem. Monitoring programmes span sites that differ in depth, turbidity, substrate, and species composition, and a classifier trained on clear tropical reef imagery performs poorly when deployed in a turbid temperate estuary [8]. The model learns habitat-specific shortcuts (water colour, background texture) rather than species-diagnostic features (body shape, fin configuration, skeletal structure), and these shortcuts fail at new sites. Four properties of underwater imagery make this transfer particularly challenging for computer vision: (i) wavelength-dependent light attenuation distorts colour as a function of depth and water type; (ii) suspended particles reduce contrast and obscure fine detail; (iii) many marine organisms are camouflaged against their substrate; and (iv) species abundance distributions are highly skewed, with a few common species and many rare ones [8]. These properties motivate models that rely on structural and textural features rather than colour, and that can adapt to new visual domains from small numbers of examples.

Foundation Models and Parameter-Efficient Adaptation↩︎

Vision foundation models offer a potential answer. These are large neural networks pretrained on hundreds of millions of images, either from unlabelled images alone (self-supervised learning) or from images paired with text descriptions (language-supervised learning), producing general-purpose visual representations that can be adapted to new tasks with minimal additional data [9], [10]. The analogy to ecological expertise is instructive: an experienced marine biologist arriving at an unfamiliar reef can identify species far more quickly than a novice, because years of field experience have built an internal representation of what distinguishes one species from another, independent of site-specific conditions. Foundation models acquire a similar “visual experience” during pretraining, learning to encode shape, texture, and structure in ways that generalise across visual domains. Domain-specific adaptations have followed for ecology: BioCLIP [11] fine-tunes a CLIP backbone (the underlying pretrained network) on 10 million biodiversity images spanning 450,000+ species, and FathomNet [12] provides over 100,000 annotated underwater images that could serve as a marine pretraining resource, but no marine-specific foundation model has yet been published.

Adapting a pretrained foundation model to a new monitoring site can take several forms. The simplest is linear probing, in which the backbone is frozen and only a single classifier layer is trained on top of the extracted features. Parameter-efficient fine-tuning (PEFT) methods modify a small fraction of the model’s parameters: Low-Rank Adaptation [13] inserts low-rank matrices into the attention layers, and Visual Prompt Tuning [14] prepends learnable tokens at every transformer layer. Full fine-tuning, in which all parameters are updated, remains the conventional approach in marine computer vision [15], [16] but typically requires large labelled datasets and is prone to overfitting when target data are scarce. Despite the ecological appeal of these methods, prior work in marine computer vision has largely treated each dataset as an independent problem [16], [17], and cross-site transfer has received limited systematic attention; studies that do evaluate generalisation typically consider a single model on a single dataset pair [18], [19]. Three gaps remain: (a) no multi-dataset benchmark exists for evaluating cross-habitat transfer in marine systems; (b) modern foundation models and parameter-efficient adaptation methods have not been systematically compared for underwater imagery; and (c) ecologists lack evidence-based guidance on the number of labelled samples required at a new site.

This paper addresses all three gaps through the following contributions:

1. A decision framework for ecological monitoring that quantifies the trade-off between labelling effort and recognition accuracy at new sites, derived from systematic evaluation across five marine ecosystems spanning three oceans and three taxonomic groups (fish, corals, and invertebrates).

2. A cross-ecosystem evaluation methodology based on an explicit difficulty gradient (within-habitat \(\rightarrow\) cross-dataset \(\rightarrow\) cross-taxon), allowing monitoring programmes to anticipate how recognition performance degrades when transferring across geographic and ecological boundaries.

3. Evidence-based labelling guidelines showing that 10–20 labelled images per species suffice for reliable deployment at a new site, reducing annotation effort by roughly an order of magnitude compared to conventional training-from-scratch.

Although we demonstrate the framework on marine datasets, the evaluation protocols and adaptation methodology apply to any ecological monitoring system that must generalise across sites, seasons, or imaging platforms; camera trap networks, acoustic monitoring arrays, and aerial survey programmes face the same cross-site transfer challenge [20], [21].

2 Materials and Methods↩︎

2.1 Study System: A Five-Dataset Marine Benchmark↩︎

We assembled a benchmark of five publicly available marine image datasets, selected to span a gradient of ecological and visual diversity (Table 1; Figure 2). All datasets were collected and released by their respective original authors under the permits, ethical approvals, and open licences described in the corresponding publications cited below; the present study performed no primary field data collection.

DeepFish [17] comprises 39 766 images from 20 distinct reef habitats along the Great Barrier Reef, Australia. Each image is labelled as containing fish (valid) or empty. The 20 habitats vary in depth, substrate, and turbidity, making this dataset uniquely suited to within-habitat transfer experiments where species labels are shared but visual conditions differ.

AQUA20 [22] contains 8 171 images of 20 marine species, drawn from mixed tropical and subtropical sources. Species range from reef fish to cephalopods, providing a multi-class classification benchmark with moderate taxonomic breadth. Class sizes in the training split range from 20 to 1 329 images, with six classes containing fewer than 50 samples.

Moorea Corals [7], [23] includes approximately 2 050 benthic quadrat images from the Moorea Coral Reef Long Term Ecological Research site in French Polynesia, with point annotations for nine substrate classes: crustose coralline algae (CCA), Montipora (MONT), Porites (POR), Pocillopora (POCILL), Montastraea (MCAV), turf algae (TURF), macroalgae (MACRO), sand (SAND), and “other” (OTHER). We converted point annotations to image-level classification labels using the majority class per image. The resulting class distribution is highly skewed: the dominant class (CCA) has 515 training images while three classes have fewer than 20 (POCILL: 14, TURF: 17, MCAV: 18), and one class (POCILL) has zero test samples.

Coralscapes [24] provides approximately 2 075 high-resolution images from Red Sea reefs with pixel-level semantic segmentation masks for 39 benthic classes. We derived image-level labels by assigning each image to its dominant class, yielding 14 classes with sufficient representation. The resulting distribution is severely long-tailed: the largest class has 458 training images while eight classes have fewer than 10 samples, making Coralscapes the most imbalanced target in our benchmark.

Brackish [25] contains approximately 14 500 frames extracted from 89 underwater videos recorded in Limfjorden, a cold temperate fjord in Denmark. Six species are annotated with bounding boxes; we converted detections to image-level presence labels. All six classes are well-represented, with training samples ranging from 161 to 5 948. This dataset represents the most visually distinct domain in our benchmark: murky, greenish water with low contrast, entirely different from tropical reef imagery.

The five datasets define a transfer difficulty gradient anchored by ecological and geographic distance:

• Easiest: Within DeepFish (habitat A to habitat B within the same reef system)

• Easy: DeepFish to AQUA20 (tropical fish to mixed marine species, same taxon)

• Medium: DeepFish to Brackish (tropical reef to cold temperate fjord, same taxon but different biome)

• Hard: DeepFish to Moorea (fish to coral, cross-taxon and cross-ocean)

• Hardest: DeepFish to Coralscapes (fish to coral, cross-taxon with severe class imbalance)

This gradient is a core organising principle of our experimental design: it allows us to measure how gracefully different models degrade as the visual and ecological gap between source and target widens.

2.2 Vision Foundation Models and CNN Baselines↩︎

We evaluate two categories of model: foundation models pretrained on large-scale data with self-supervised or language-supervised objectives, and convolutional neural network (CNN) baselines pretrained with supervised classification on ImageNet [26].

Foundation models. DINOv2-Base [9] is a Vision Transformer (ViT-B/14) trained by self-distillation on 142 million curated images. Self-supervised pretraining works by presenting the model with different augmented views of the same image and training it to produce consistent representations; the model learns to encode visual structure without any labels. The resulting features capture shape, texture, and spatial arrangement in ways that transfer across visual domains, much as an experienced field biologist recognises species by diagnostic morphological features regardless of the site visited. CLIP-Base [10] is a ViT-B/16 trained to match images with their text descriptions across 400 million image-text pairs, learning visual concepts grounded in natural language. CLIP’s language supervision provides a complementary learning signal: the model learns that images described with the same words should have similar representations, enabling zero-shot classification, in which images are assigned to categories from a text description of each category alone, with no labelled training examples.

CNN baselines. ResNet-50 [27] and EfficientNet-B4 [28] are convolutional architectures pretrained on ImageNet-1K (1.28 million images, 1 000 classes) with supervised classification. These baselines represent the standard approach in applied marine computer vision, where models are initialised from ImageNet weights and fine-tuned on task-specific data. We include both architectures to control for the possibility that performance differences stem from the transformer architecture itself rather than the pretraining strategy.

Table 2 summarises the model specifications. We note that EfficientNet-B4 uses a native input resolution of \(380 \times 380\) (2.88\(\times\) more pixels than the \(224 \times 224\) used by the other three models), which may confer an advantage from higher-resolution input rather than architectural or pretraining differences alone.

2.3 Adaptation Strategies↩︎

Rather than updating all parameters in a pretrained model (which risks overfitting when labelled data are scarce), parameter-efficient adaptation methods modify only a small fraction of the model while keeping the pretrained backbone frozen. We employ model-dependent adaptation strategies: transformer-specific methods (LoRA and VPT) for the Vision Transformer foundation models, and full fine-tuning for the CNN baselines. All models are additionally evaluated with a linear probe as the simplest baseline.

Linear Probe (all models). The backbone is entirely frozen and a single linear classification layer is trained on top of the extracted features. This tests the quality of the pretrained representations without any adaptation; analogously, it asks the model questions about what it already knows, without teaching it anything new. The classification head has on the order of thousands of parameters.

Low-Rank Adaptation (LoRA; foundation models only). LoRA [13] inserts small, low-rank matrices into the attention layers of a Vision Transformer, allowing targeted adjustments to how the model attends to visual features. We apply LoRA with rank \(r = 4\), scaling factor \(\alpha = 8\), and dropout 0.1 to the query and value projections. This adds approximately 149K trainable parameters for DINOv2-Base (0.17% of the full model) and 75K for CLIP-Base (0.09%), yet enables meaningful adaptation of the attention mechanism.

Visual Prompt Tuning (VPT-Deep; foundation models only). VPT [14] prepends a set of learnable “prompt” tokens at every transformer layer, providing the model with task-specific context at each processing stage. We use 10 prompt tokens per layer with dropout 0.1, adding approximately 94K trainable parameters for DINOv2-Base (0.11%) and 93K for CLIP-Base (0.06%).

Full Fine-tuning (CNN baselines only). All parameters are unfrozen and trained end-to-end. This is the conventional approach in applied marine computer vision, and it represents the upper bound of what can be achieved by updating the entire model. We apply layer-wise learning rate decay (factor 0.75) and label smoothing (0.1) to mitigate overfitting.

All configurations use the AdamW optimiser [29] with cosine learning rate scheduling, 5% linear warmup, early stopping with patience of 10 epochs, and mixed-precision training. Table 3 summarises the adaptation strategies and their key hyperparameters.

2.4 Experimental Protocols↩︎

We designed three complementary protocols, each answering a distinct question about transferability. Together, they trace a gradient from the easiest transfer scenario (same reef system, different habitat) to the hardest (cross-taxon transfer to severely imbalanced coral datasets with few labelled samples).

2.4.1 Protocol A: Can Foundation Models Recognise Species in a Reef Habitat They Have Never Seen?↩︎

Binary species occupancy detection, distinguishing images that contain target organisms from empty frames, is a foundational task in underwater monitoring. BRUVS programmes routinely process thousands of video frames to identify which contain fish [17], and automated occupancy detection is the first stage in multi-step monitoring pipelines. Using the 20 habitats in DeepFish, we perform leave-one-habitat-out evaluation: train on 19 habitats, evaluate on the held-out habitat, and repeat for all 20. The controlled nature of this binary protocol (shared label space, identical annotation procedure, 20 visually distinct habitats) makes it uniquely suited to isolating the effect of habitat-induced domain shift without confounding factors such as differing label spaces or annotation inconsistencies. Each of the 10 model–adaptation configurations is evaluated across all 20 target habitats with a single seed (seed = 42), yielding 200 runs. To assess sensitivity to random initialisation, we additionally ran DINOv2 LP (the top-performing configuration) with two further seeds (123, 456) across all 20 habitats, yielding 240 total Protocol A runs. We report macro F1, a recognition-accuracy score from 0 (worst) to 1 (best) that weights all classes equally, averaged across the 20 held-out habitats, with standard deviation capturing inter-habitat variability.

2.4.2 Protocol B: How Far Can Representations Transfer Across Marine Ecosystems?↩︎

We evaluate cross-dataset transfer along the difficulty gradient defined in Section 2.1, using DeepFish as the source for all four transfer pairs. Because source and target datasets have non-overlapping label spaces, we use open-set evaluation based on nearest-prototype matching in feature space: each target class is represented by the mean feature vector of its labelled examples, and test images are assigned to the nearest prototype. This approach requires no shared labels and is therefore always applicable. We note that nearest-prototype matching inherently favours models with better-organised feature spaces; we select this method because it is the simplest non-parametric classifier applicable to open-set transfer and because it mirrors the practical scenario where an ecologist has labelled examples of target species but no target-domain training infrastructure. Each of the 10 model–adaptation configurations is evaluated on all four pairs, yielding 40 runs.

2.4.3 Protocol C: How Many Labelled Samples from a New Site Are Actually Needed?↩︎

This is the central experiment. For three transfer pairs spanning the difficulty gradient (DeepFish to AQUA20, DeepFish to Moorea, DeepFish to Brackish), we vary the number of labelled target samples per class (\(k\)) from 0 to 100 (\(k \in \{0, 1, 2, 5, 10, 20, 50, 100\}\)) and measure performance as a function of \(k\). At each \(k\) value, we draw five independent random samples of target data for \(k \leq 20\) and three for \(k \geq 50\), reporting mean \(\pm\) standard deviation across trials (at \(k = 0\) no sampling is required, so the result is deterministic). Training uses a combined strategy: source data and \(k\) target samples per class are mixed and the model is trained on the combined set. For CLIP at \(k = 0\), we additionally evaluate zero-shot classification using text prompts of the form “a photo of a {species} in the ocean”. Six model–adaptation configurations are evaluated: DINOv2-B with linear probe, DINOv2-B with LoRA (\(r=4\)), CLIP-B with linear probe, CLIP-B with LoRA (\(r=4\)), ResNet-50 with linear probe, and EfficientNet-B4 with linear probe. This yields 648 runs across all \(k\) values, pairs, and trials.

An important practical constraint is that target datasets have uneven class distributions. When \(k\) exceeds the number of available training samples for a class, all samples from that class are assigned to the support set, leaving zero query samples for that class during evaluation. AQUA20 is well-balanced (minimum 20 samples per class), so all \(k\) values up to 20 are fully supported. Moorea has three classes with fewer than 20 training samples (POCILL: 14, TURF: 17, MCAV: 18), so at \(k = 20\) these classes are exhausted. Brackish is the most balanced target (minimum 161 samples per class), with no sampling constraints at any \(k\) value. We report results only over classes that retain at least one query sample at each \(k\), and note the effective number of evaluable classes where relevant.

2.4.4 Metrics and Statistical Analysis↩︎

We use macro F1 as the primary metric for Protocols A and B because it handles class imbalance by weighting each class equally, which is essential for datasets with skewed species abundances. For Protocol C, we report balanced accuracy because the few-shot sampling protocol progressively depletes the query set at high \(k\), causing macro F1 to decline artifactually as minority classes are exhausted from evaluation. For cross-protocol comparability, we report both metrics in Appendix S1: Table S2. Pairwise model comparisons across habitats (Protocol A) use the Wilcoxon signed-rank test (Table 4). For all protocols, 10% of the source training data is randomly held out as a validation set for early stopping. All random seeds are fixed and all configurations specified in YAML files to ensure full reproducibility. Code is available at https://github.com/alzayats/cross-habitat-marine.

3 Results↩︎

3.1 Within-Habitat Transfer: Do Foundation Models Generalise Across Reef Sites?↩︎

Protocol A tests whether models learn species-diagnostic features that transfer across the 20 reef habitats in DeepFish, or whether they rely on habitat-specific visual shortcuts. Figure 3 presents the macro F1 scores for each model–adaptation combination, averaged across all 20 leave-one-habitat-out experiments.

DINOv2-Base with a linear probe achieved the highest mean macro F1 (\(0.70 \pm 0.28\) across all 20 habitats; \(0.74 \pm 0.23\) excluding habitat 7268, a degenerate case with zero empty-class test samples), followed by EfficientNet-B4 with full fine-tuning (\(0.67 \pm 0.24\)), DINOv2-Base with VPT (\(0.65 \pm 0.30\)), and DINOv2-Base with LoRA (\(0.64 \pm 0.30\)). CLIP-Base configurations performed below, with its best adaptation (linear probe) reaching a mean F1 of \(0.60 \pm 0.23\). Among CNN baselines, EfficientNet-B4 with full fine-tuning achieved \(0.67 \pm 0.24\) and ResNet-50 with full fine-tuning reached \(0.63 \pm 0.27\).

Wilcoxon signed-rank tests across the 20 habitats (Table 4) revealed that DINOv2 LP was significantly better than all three CLIP configurations (\(P < 0.05\)), both CNN linear probes (\(P < 0.05\)), and ResNet-50 with full fine-tuning (\(P = 0.033\)). However, DINOv2 LP was not significantly different from DINOv2 LoRA (\(P = 0.11\)), DINOv2 VPT (\(P = 0.29\)), or EfficientNet-B4 with full fine-tuning (\(P = 0.41\)). The practical implication is that no statistically significant difference was detected between DINOv2 LP and the best alternatives, yet DINOv2 LP uses only 1,538 trainable parameters, four orders of magnitude fewer than LoRA (149K) or full fine-tuning (17.6M); exact parameter counts for all ten configurations are reported in Appendix S1: Table S1.

That a frozen linear probe matches LoRA and VPT is confirmed by the statistical tests: the pretrained DINOv2 features already contain sufficient species-relevant information that parameter-efficient adaptation of the attention layers provides no measurable benefit for within-habitat transfer. Training dynamics confirm this: DINOv2 linear probe converges in 2–3 epochs, while adapted configurations require substantially more training without improving final performance (Appendix S1: Figure S1). This has practical implications: ecologists need only train a lightweight classifier head, avoiding the complexity and hyperparameter tuning of LoRA or VPT.

Across three random seeds, DINOv2 LP mean F1 varied by only 0.015 (0.70, 0.72, 0.71 for seeds 42, 123, 456 respectively), confirming that inter-habitat variability (SD \(= 0.28\)) dominates over initialisation variance.

Performance varied substantially across target habitats (Figure 4). The hardest habitats to generalise to were 7268 (F1 = 0.00, a degenerate case with zero empty-class test samples), 7482 (F1 = 0.23), and 9862 (F1 = 0.25), which share high class imbalance. The easiest habitats (9892, F1 = 1.00; 9852, F1 = 0.99; 7434, F1 = 0.99) had more balanced class distributions. This variability (standard deviation of 0.28 across habitats for DINOv2 linear probe) underscores that even within a single reef system, some sites pose greater transfer challenges than others, driven primarily by class imbalance rather than visual difficulty (Figure 5).

For monitoring programmes operating across multiple sites within a reef system, these results indicate that a DINOv2-based model trained on a subset of sites will perform reliably at held-out sites without site-specific retraining, using only a frozen feature extractor with a linear classifier.

3.2 Cross-Dataset Transfer: From Tropical Reefs to Temperate Fjords↩︎

Protocol B tests how far pretrained representations can transfer across datasets that differ not only in visual domain but in species composition, geographic origin, and ecological context. Figure 6 presents the macro F1 for all model–adaptation combinations across the four cross-dataset transfer pairs. Performance declined sharply along the difficulty gradient. The easiest transfer pair (DeepFish to AQUA20) yielded the highest F1 for all models, with DINOv2 linear probe reaching 0.84 and DINOv2 LoRA reaching 0.82. The medium-difficulty transfer (DeepFish to Brackish, same taxon but different biome) saw DINOv2 VPT reach 0.44 and DINOv2 LP reach 0.34. The cross-taxon transfers proved substantially harder: DeepFish to Moorea yielded F1 = 0.18 for the best models, while DeepFish to Coralscapes reduced all models to near-chance performance (\(\sim\)​0.04 F1), indicating that open-set nearest-prototype matching cannot bridge the gap between fish and coral class distributions combined with severe class imbalance.

Foundation models degraded more gracefully than CNN baselines across the difficulty gradient (Figure 7). On the easiest pair (DF \(\to\) AQ20), DINOv2 LP achieved 0.84 vs.EffNet LP at 0.60 and RN50 LP at 0.39. All models showed a relative F1 drop of 44–55% from their within-habitat (Protocol A) baselines, with CLIP LP showing the largest relative drop (55%) and DINOv2 VPT the smallest (44%). In practical terms, foundation model features encode species-relevant structure that transfers across geographic regions, whereas CNN features are more tightly coupled to the training domain.

For monitoring programmes planning to deploy recognition at sites with no local training data, these results quantify what to expect: foundation models provide a useful starting point for within-taxon transfer (e.g., fish to fish), but cross-taxon transfer remains challenging and motivates the few-shot adaptation explored in Protocol C.

3.3 How Many Labels Do Ecologists Need? The Few-Shot Answer↩︎

Protocol C directly addresses the title question. Figure 8 shows how recognition performance (balanced accuracy) scales with the number of labelled target samples per class (\(k\)) across three transfer scenarios of increasing difficulty (see Table 5). We report balanced accuracy rather than macro F1 because the few-shot sampling protocol progressively depletes the query set at high \(k\), causing macro F1 to decline artifactually as minority classes are exhausted from evaluation.

Six model–adaptation combinations were evaluated: DINOv2 LP and LoRA, CLIP LP and LoRA, ResNet50 LP, and EfficientNet-B4 LP. On the easiest transfer (DeepFish \(\to\) AQUA20), DINOv2 LP exhibited a characteristic elbow: balanced accuracy rose steeply from \(k = 0\) (0.07) to \(k = 5\) (0.87), reaching \(0.90\) at \(k = 10\) and plateauing thereafter. DINOv2 LoRA followed a similar trajectory (0.86 at \(k = 5\), 0.90 at \(k = 20\)), confirming that the frozen features already capture transferable structure.

CLIP zero-shot (\(k = 0\)) provided a useful baseline that requires no labelled data. Using text prompts alone, CLIP achieved balanced accuracy of 0.58 on AQUA20 (where class names align well with CLIP’s language understanding), but only 0.15 on Moorea and 0.21 on Brackish, where specialised taxonomic labels fall outside CLIP’s pretraining vocabulary. While limited, this zero-shot capability could serve as a preliminary survey tool to prioritise which species or sites warrant further labelling effort.

For Moorea (the hardest Protocol C transfer, cross-taxon fish to coral), all models converged to balanced accuracy \(\sim\)​0.84–0.89 at \(k = 100\), though DINOv2 LP reached 80% of its maximum by \(k = 10\). Brackish showed intermediate difficulty with wider separation between models: DINOv2 LP reached 0.83 at \(k = 100\), while CLIP LP lagged at 0.71 and CLIP LoRA achieved only 0.40, suggesting that DINOv2 features transfer more effectively across biomes.

The practical message is that even a small number of labelled target samples (5–10 per class) can substantially boost performance over zero-shot transfer (Figure 9). DINOv2 LP reached 80% of its maximum balanced accuracy at \(k \approx 11\) (interpolated between tested values \(k = 10\) and \(k = 20\)), the most sample-efficient combination tested. This sample efficiency advantage of DINOv2 features, achieving strong performance with minimal labelling, directly lowers the barrier to deploying automated monitoring at new marine sites.

3.4 What Makes Foundation Models Different? Feature Analysis↩︎

To understand why foundation models generalise across habitats while CNN baselines do not, we analysed the learned feature representations using t-SNE [30], a technique that projects a model’s high-dimensional internal representations onto a two-dimensional plot so that images the model treats as similar appear close together.

3.4.1 Feature Space Structure↩︎

Figure 10 presents t-SNE projections of features extracted from 250 DeepFish test images spanning five habitats, coloured by class label (top row) and by habitat of origin (bottom row). DINOv2 + LoRA features cluster tightly by class (empty vs.fish present), achieving a silhouette score of 0.66, a measure of cluster quality ranging from \(-1\) to \(1\), where higher values indicate tighter, better-separated groups, with samples from different habitats intermixed within each class cluster (habitat silhouette = 0.26). This indicates that the model has learned class-diagnostic representations that are invariant to habitat-specific visual conditions. ResNet-50 features, by contrast, show weak class separation (silhouette = 0.17) but strong habitat clustering (silhouette = 0.62): samples from the same site group together regardless of class identity, suggesting that the model relies on background cues (substrate colour, water clarity) rather than species-diagnostic morphology.

This difference has direct ecological implications. A model that clusters by species will generalise to new habitats because it has learned what a fish looks like regardless of background. A model that clusters by habitat will fail at new sites because it has learned what a particular site looks like, not what the organisms within it look like.

3.4.2 Efficiency Frontier↩︎

Appendix S1: Figure S2 plots the performance–efficiency trade-off for all model–adaptation combinations, with trainable parameters on the horizontal axis (log scale) and macro F1 on the vertical axis. DINOv2-Base linear probe is the sole Pareto-optimal point, achieving the highest F1 (0.70) with only 1,538 trainable parameters, four orders of magnitude fewer than LoRA (149K) or full fine-tuning (17–24M). This efficiency has practical relevance: linear probing of DINOv2-Base requires minimal GPU memory and trains in minutes, making foundation model performance accessible to ecology research groups with limited computational resources.

4 Discussion↩︎

4.1 Practical Recommendations for Ecologists↩︎

The results across all three protocols converge on a set of concrete recommendations for deploying automated species recognition at new marine sites. We present these as a decision framework based on the number of labelled images available from the target site:

• Zero labels: Use CLIP zero-shot classification with species-descriptive text prompts. This requires no labelled data and provides a preliminary estimate of species composition, useful for survey prioritisation. Expected performance: balanced accuracy = 0.15–0.58, depending on how well species names align with CLIP’s vocabulary and target domain similarity.

• 1–5 labels per species: Use DINOv2-Base with a linear probe on the frozen features. With very few labels, the pretrained features are already informative and the risk of overfitting is minimised. Expected performance: balanced accuracy rises steeply from 0.46 at \(k=1\) to 0.68 at \(k=5\) (averaged across transfer pairs).

• 10–20 labels per species (recommended): Use DINOv2-Base with a linear probe. This is the practical sweet spot: 10 labelled images per species (an annotation effort of 1–4 hours depending on species count) achieves near-plateau performance. Expected performance: balanced accuracy \(\approx\) 0.73 at \(k=10\), rising to 0.79 at \(k=20\) (averaged across transfer pairs).

• 50+ labels per species: Marginal gains above \(k=50\) are small (balanced accuracy 0.84 at \(k=50\), 0.85 at \(k=100\)). LoRA adaptation and full fine-tuning of CNNs offer no clear advantage over a frozen linear probe at any \(k\) value, while requiring substantially more computation.

Hardware requirements. Linear probing of DINOv2-Base requires minimal GPU memory and trains in minutes on a single consumer GPU. Even LoRA adaptation runs on a GPU with 16 GB of memory and completes in under one hour. Full fine-tuning of the same backbone requires 40 GB or more. Since the best-performing configuration (frozen linear probe) is also the most efficient, foundation model performance is accessible to research groups without specialised computing infrastructure.

4.2 Why Foundation Models Succeed Underwater↩︎

The feature analysis (Section 3.4) reveals the mechanism behind foundation model transferability: self-supervised pretraining produces representations that encode structural and textural properties of objects, suppressing domain-specific confounds such as water colour and substrate type.

The self-distillation objective used in DINOv2 training is instructive. By learning to produce consistent representations across different augmented views of the same image (which include colour jitter, random cropping, and geometric distortion), the model is explicitly trained to ignore surface-level visual variation and encode deeper structural invariances. This is precisely what is needed for cross-habitat transfer in marine systems, where the same species appears under dramatically different colour profiles, lighting conditions, and backgrounds.

The attention mechanism in Vision Transformers plays a complementary role. Unlike CNNs, which process images through fixed local receptive fields, transformers can attend selectively to any spatial location. Prior work has shown that self-supervised ViTs learn to attend to salient foreground objects [31], suggesting that this flexibility enables the model to focus on organisms and suppress background variation, much as an expert taxonomist examines diagnostic features (body shape, fin morphology, skeletal structure) rather than the surrounding environment. CNNs, constrained to local feature extraction, are more susceptible to encoding spatially co-occurring habitat features alongside target features.

4.3 Limitations, Uncertainty, and Transferability↩︎

Several limitations bound the scope of our conclusions.

Benchmark composition. All five datasets are publicly available benchmarks collected under specific conditions. Real-world deployment may encounter challenges not represented here, including novel species absent from training data, extreme turbidity or biofouling on camera housings, and camera artefacts specific to particular platforms. Our difficulty gradient is observational rather than experimentally controlled. Geographic distance, taxonomic distance, number of target classes (2 to 14), and class imbalance severity are correlated across our benchmark and cannot be cleanly disentangled. The near-chance performance on the Coralscapes transfer (F1 \(\approx\) 0.04) reflects not only the ecological distance between fish and coral but also the challenge of nearest-prototype matching into 14 severely imbalanced classes. We note, however, that these confounds reflect ecological reality: harder real-world transfers typically involve more classes, greater imbalance, and larger visual domain gaps simultaneously. The gradient should therefore be interpreted as an ordinal ranking of practical transfer difficulty rather than a controlled measurement of any single factor.

Task simplification. DeepFish and Brackish use binary or coarse species labels. Fine-grained species identification (distinguishing closely related congeners) is a harder task that may not benefit as strongly from pretrained features. The conversion from detection and segmentation annotations to image-level classification labels, while enabling cross-task comparison, discards spatial information that may be ecologically relevant.

Temporal considerations. DeepFish and Brackish contain frames extracted from video. Although we split by video to mitigate temporal data leakage, consecutive frames within a video share visual context, and our within-video independence assumption may be imperfect. Video-specific temporal models are not evaluated.

Transfer directionality. All cross-dataset transfers in Protocols B and C use DeepFish as the sole source dataset. We did not evaluate other transfer directions (e.g., coral-to-coral or coral-to-fish), and transferability from non-fish sources remains untested.

Ecosystem coverage. Our benchmark spans tropical to temperate marine systems but does not include freshwater, deep-sea, pelagic, or polar environments. Transferability to these domains remains an open question, though the underlying methodology (foundation model plus parameter-efficient adaptation) is domain-agnostic.

Computational costs. While LoRA adaptation is efficient (16 GB GPU memory, under one hour training), the DINOv2-Base backbone itself contains 86 million parameters and requires GPU inference. For CLIP configurations, the text encoder (63.4M parameters) is also loaded in memory even though it is not trained, increasing the total memory footprint to approximately 149M parameters. For resource-constrained settings (e.g., on-board processing on autonomous underwater vehicles), model distillation or quantisation may be necessary, which we do not evaluate.

4.4 Future Directions↩︎

Several extensions would strengthen the practical value of this framework. Active learning methods that use model uncertainty to prioritise which images an expert should label could further reduce annotation requirements by selecting the most informative samples [20]. Video-aware models that leverage temporal continuity (tracking individuals across frames) could improve both accuracy and ecological utility by linking detections to abundance estimates. Foundation models pretrained specifically on underwater imagery, as underwater image repositories grow, may provide even stronger starting representations for marine tasks. Multi-task learning that jointly addresses classification, detection, and segmentation could unify the currently fragmented landscape of marine computer vision tools. Finally, integration with ecological modelling (species distribution models, community composition analysis) would close the loop from raw imagery to ecological inference.

5 Conclusions↩︎

We present a decision framework, validated across five marine ecosystems spanning three oceans and three taxonomic groups, that allows ecological monitoring programmes to budget labelling effort against expected recognition accuracy when deploying automated species recognition at new sites. The framework rests on three findings, established through 928 systematic experimental runs across within-habitat, cross-dataset, and few-shot adaptation protocols. First, as few as 10–20 labelled images per species suffice to deploy reliable recognition at a new site (an annotation effort of 1–4 hours), reducing the labelling bottleneck that currently constrains the geographic scope of automated monitoring by roughly an order of magnitude. Second, frozen self-supervised foundation model features (DINOv2 with a linear classifier) match or exceed the performance of fully fine-tuned convolutional baselines across every transfer scenario tested, while requiring four orders of magnitude fewer trainable parameters and no specialised computing infrastructure. Third, foundation models generalise where convolutional baselines fail because they encode species-diagnostic representations that are habitat-invariant rather than habitat-specific visual shortcuts.

For monitoring programmes, the practical implication is that expanding to a new reef, estuary, or kelp forest no longer requires restarting the labelling cycle from scratch. A modest annotation effort, guided by the framework presented here, is sufficient to achieve reliable performance. Although demonstrated on marine imagery, the evaluation methodology and decision rules apply to any ecological monitoring system that must generalise across sites with limited labelled data, from camera trap networks to acoustic surveys to aerial monitoring.

Data Availability Statement↩︎

All code, configurations, and analysis scripts for reproducing the experiments are publicly available at https://github.com/alzayats/cross-habitat-marine. Upon acceptance, a versioned snapshot of the code together with all derived analysis outputs (per-run metrics, feature embeddings, and figure-generation scripts) will be archived in the Dryad Digital Repository with a citable DOI. The five datasets used in this study are publicly available and were obtained from their original sources: DeepFish [17], AQUA20 [22], Moorea Labeled Corals [7], [23], Coralscapes [24], and Brackish [25].

CO\(_2\) Emission Related to Experiments↩︎

All experiments were conducted on a single NVIDIA GeForce RTX 4090 GPU (450 W TDP) at James Cook University’s High Performance Computing facilities. The experiments comprised 240 training runs (Protocol A), 40 evaluation runs (Protocol B), and 648 training and evaluation runs (Protocol C), for a total of approximately 730 GPU hours. Assuming 80% average GPU utilisation, a power usage effectiveness (PUE) of 1.1, and the Australian average grid carbon intensity of 0.65 kgCO\(_2\)/kWh (2023 estimate), we estimate total energy consumption at approximately 289 kWh and associated CO\(_2\) emissions at approximately 188 kg. For context, this is comparable to approximately 1,200 km of driving in an average passenger car. We note that the most parameter-efficient configuration (DINOv2-Base linear probe, 1,538 trainable parameters) trains in under 5 minutes per habitat, suggesting that the majority of compute was spent on the full factorial comparison rather than on the recommended deployment configuration.

Acknowledgements↩︎

We thank the providers of the five public datasets used in this study (DeepFish, AQUA20, Moorea Labeled Corals, Coralscapes, and Brackish), whose openly released imagery made this benchmark possible; full citations and access details are given in the Data Availability Statement. Computational resources were provided by James Cook University’s High Performance Computing facilities (Townsville, QLD, Australia).

Use of generative AI. Generative AI tools (Anthropic Claude) were used to assist with manuscript drafting (text refinement and clarity edits) and with code documentation. The AI tools did not contribute to research design, dataset curation, model implementation, statistical analyses, or scientific interpretation of results. All AI-assisted text was reviewed and revised by the authors, and all AI-assisted code was inspected, tested, and integrated into the publicly archived repository by the authors. The corresponding author takes full responsibility for the accuracy of all content.

Funding↩︎

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author Contributions↩︎

Alzayat Saleh: Conceptualization, methodology, software, data curation, formal analysis, investigation, writing (original draft), visualization, and project administration. Mostafa Rahimi Azghadi: Conceptualization and review & editing of the manuscript.

Conflict of Interest Statement↩︎

The authors declare no conflicts of interest. No financial, professional, or personal relationships have influenced the design, conduct, analysis, or reporting of this work.

6 Supplementary Material↩︎

This appendix provides supporting tables and figures referenced from the main manuscript. Table S1 reports exact trainable parameter counts for all ten model–adaptation configurations evaluated in Protocol A. Table S2 reports Protocol A results with both macro F1 and balanced accuracy for cross-protocol comparability. Figure S1 shows training dynamics for the top six model–adaptation combinations. Figure S2 shows the efficiency–performance trade-off across all combinations.

References↩︎