WIO-ReefFish: A High-Resolution Dataset for Taxon-Aware Coral Reef Fish Detection in the Western Indian Ocean

Combining Deep Learning and Ecological Monitoring for BRUV Coral Reef Megafauna Assessment

Coral reef ecosystems are increasingly threatened by climate change, pollution, and overfishing, causing major declines in marine megafauna and high-trophic-level fishes. Monitoring these species is vital for conservation, yet traditional survey methods are slow and resource intensive. This study presents a semi-automated monitoring pipeline that integrates deep learning (DL) with human-in-the-loop validation to streamline Baited Remote Underwater Video (BRUV) analyses in the Gita Nada Marine Protected Area (MPA), Indonesia. A total of 244 BRUV deployments from SORCE’s long-term monitoring program in the Gita Nada MPA, comprising 328 h of footage, collected 2023–2025 under Indonesian research oversight through Yayasan SORCE Konservasi Indonesia, were processed using a DL workflow. To address long-tailed species distributions, focal taxa were grouped into six Morphological Groups and detected using a YOLOv12x model trained via transfer learning from the Community Fish Detector. A custom temporal-tracking framework extracted ecological metrics including N, Time to First Visit (T1st), and Visit Duration (Tvisit). The pipeline achieved moderate to high detection and tracking performance for several Morphological Groups, achieving object detection F1-scores of up to 0.873 and an overall tracker recall and precision of 0.80 and 0.76, respectively, although performance varied substantially among groups and was substantially limited for data-deficient taxa. As a proof-of-concept, we applied the framework to assess ecological shifts in Cheloniidae and Carangidae across coral-cover gradients. Overall, this semi-automated approach reduces BRUV processing effort and provides a scalable foundation for generating the large datasets needed to detect subtle ecological change.

PICOGRAM - informing coral reef resilience-based management through prediction of individual coral organismal growth, recruitment, and mortality

Coral reef science and management are increasingly relying on imagery-based methods to expand monitoring capacity and improve the assessment of reef condition. However, scalable monitoring requires coral colony-scale segmentation from high-resolution underwater imagery and orthomosaic-derived survey products, while dense manual pixel-wise annotation remains costly, time-consuming, and inconsistent across sites and imaging conditions. Operational methods therefore need to generalize across sites, optics, and platforms while providing calibrated confidence with minimal quality-assurance effort. We introduce NASA/NOAA PICOGRAM, an open-source extension inspired by the NASA NeMO-Net ecosystem, as an image-only, user-promptable framework for automated coral colony detection, segmentation, and percent-cover estimation. The current study evaluates segmentation and cover estimation from underwater still images and orthomosaic-derived image tiles; repeated site imagery can subsequently support longitudinal analyses of colony change. PICOGRAM adapts a Segment Anything Model (SAM)-style encoder–decoder to underwater imagery while freezing the visual backbone and training only low-rank adapters, a lightweight prompt encoder for points and boxes, and a transformer mask decoder. PICOGRAM is supervised using Simple Linear Iterative Clustering (SLIC)-derived pseudolabels. Enhanced images are over-segmented into superpixels, scored with underwater-aware color, texture, and edge features, smoothed on a superpixel graph, and converted to masks using a Potts conditional random field (CRF). Mask-level non-maximum suppression and a two-round curriculum further tighten pseudo-label selection. A quality head predicts mask Intersection-overUnion (IoU) and is calibrated on validation data to support site-specific operating thresholds. Using NOAA National Coral Reef Monitoring Program benthic survey imagery, PICOGRAM is evaluated on a site-disjoint expert-labeled hold-out ( n = 137). The method attains an IoU of 87.5% in-domain and 82.5% cross-site, with strong boundary accuracy (bIoU 82.1%) and favorable precision–recall behavior (93.0% precision; PR–AUC 94.5%). Quality scores are well calibrated, with an expected calibration error of 0.028, and ambiguous masks can be refined efficiently with an average of 1.8 clicks to reach 0.90 IoU. Percent-cover estimates closely match expert annotations (Pearson r = 0.98, mean absolute error of 1.2 percentage points; bias −0.3 ± 2.1 pp). Across matched operating points, PICOGRAM modestly but consistently outperforms strong baselines based on SAM 1, SAM 2, YOLOv11, and CoralScop. By reducing the need for manual pixel-wise training masks and enabling calibrated, low-click refinement, PICOGRAM provides deployable coral colony-scale segmentation and cover estimation from NOAA underwater imagery and orthomosaic-derived tiles. This capability can support higher spatial coverage in coral monitoring programs, improve quality control in large image collections, and facilitate rapid post disturbance assessment, while future longitudinal validation will extend the framework toward direct estimation of colony growth, recruitment, and mortality.

A large-scale crowd-sourced annotated acoustic dataset of Indian fauna

A Dataset for Fish Segmentation and Tracking in Underwater Videos

The automatic monitoring of fish in underwater imagery plays a key role in marine ecology, fisheries management, and environmental monitoring, yet progress is limited by the lack of large, high-quality fish-focused datasets. We present a new dataset of underwater videos of fish in natural habitats, annotated for pixel-level segmentation and multi-object tracking. The data was collected in the Balearic Sea, the western Mediterranean region surrounding the island of Mallorca (Spain), across diverse marine environments to capture variations in species, lighting, turbidity, and background complexity. Each video frame has been carefully annotated to ensure spatial and temporal consistency, yielding a challenging and comprehensive resource for developing and benchmarking underwater vision algorithms. To illustrate the dataset’s utility, we provide baseline tracking results obtained with Deep OC-SORT, which highlight both the dataset’s challenging nature and its potential for future method evaluation. In addition, we release an open-source, browser-based annotation tool integrating the Segment Anything Model (SAM2) and CUTIE for efficient semi-automatic segmentation and tracking. This tool facilitates high-quality annotations without specialized hardware, improving accessibility and reproducibility within the marine imaging community.

WildFin: An In-the-Wild Dataset for Fish Behavioral Recognition

Recent advances in field technology have led to a massive influx of in-the-wild video data for ecological science. The primary bottleneck in leveraging this data is the high cost of expert annotation. While computer vision offers a potential solution, current models frequently fail when deployed in complex marine environments. To characterize these failures, we introduce WildFin, a novel benchmark for fish behavior recognition collected and annotated by ecologists.WildFin spans two critical real-world paradigms: stationary cameras monitoring groups of fish and dynamic divers following individual subjects. The dataset represents a massive curation effort, involving 1,350 hours of fieldwork and 600 hours of expert annotation to produce 9 hours of behavioral data with over 2 million frame-by-frame labels. We benchmark modern vision foundation models and quantify tradeoffs between static and spatiotemporal architectures, revealing the substantial gap that remains between current model capabilities and the demands of real-world underwater behavioral analysis. Project website: https://team-wildfin.github.io/.

HiFi-Det: Collaborative Multi-Scale Frequency-Domain Feature Optimization for Crown-of-Thorns Starfish Detection in Complex Underwater Environments

Outbreaks of the Crown-of-Thorns Starfish (COTS, Acanthaster spp.) are a leading biological driver of coral cover loss, making timely and accurate population monitoring essential for reef management. Conventional diver-based surveys are labor-intensive and prone to missed detections, motivating automated detection from underwater imagery. However, COTS detection in complex underwater scenes still faces three major challenges. First, COTS individuals are often very small and carry limited discriminative information, making them inherently difficult to detect. Second, low underwater contrast and complex coral textures blur target boundaries and cause targets to be easily confused with the background. Third, ecological monitoring values recall more highly than precision—missing a COTS individual is far more costly than a false alarm—yet the recall of existing detectors remains insufficient. To address these challenges, we propose HiFi-Det (High-resolution Frequency-integration Detector), a collaborative multi-scale frequency-domain feature optimization method built on YOLO11. HiFi-Det integrates three complementary enhancements: a high-resolution detection branch that strengthens feature representation for small targets; wavelet transform convolution (WTConv) modules in the backbone and neck that apply band-separated processing in the wavelet domain to improve discrimination of COTS targets from low-contrast, textured coral backgrounds; and a WIoUv3 bounding box regression loss that dynamically focuses on ordinary-quality samples to improve recall while maintaining precision. On the public Great Barrier Reef dataset, HiFi-Det attains 81.02% F2 and 87.54% mAP@50, surpassing the YOLO11 baseline by 3.00% and 2.57%, respectively, while keeping the parameter count essentially unchanged relative to the YOLO11s baseline (within 3%), so that the accuracy gains are obtained without inflating model size. Ablation studies confirm the synergy of the three components: the high-resolution branch preserves spatial details, WTConv suppresses background textures, and WIoUv3 further curbs false positives while sustaining high recall. Applying the same recipe to a larger YOLO11m backbone yields HiFi-Det-m, which likewise improves over that backbone in both F2 and recall, indicating that the approach is a transferable recipe rather than a single fixed architecture. These results show that task-specific architectural and training designs can effectively adapt generic detectors to the demands of underwater ecological monitoring.