FedSAM-3D: Adapter-Constrained Federated Adaptation for Transferable Medical Segmentation Foundation Models.

Objective

Transferring large-scale medical foundation models to specific clinical tasks remains challenging, particularly in multi-center scenarios with heterogeneous data distributions and privacy constraints. Existing adaptation strategies provide limited solutions for collaboratively adapting foundation models across institutions while preserving their transferable representations.

Methods

We propose FedSAM-3D, a foundation model adaptation framework for multi-center medical image segmentation. Built upon the SAM-Med3D backbone, FedSAM-3D defines the collaborative optimization space within adapter parameters while keeping the pretrained backbone unchanged. Through federated optimization within this constrained adaptation space, our framework enables efficient cross-center knowledge aggregation without exchanging full model parameters, while allowing each client to adapt the foundation model to local medical data distributions.

Results

FedSAM-3D was evaluated on multi-center abdominal organ and brain tumor segmentation datasets under federated adaptation and zero-shot evaluation settings. Across both tasks and multiple clinical datasets, FedSAM-3D generally outperformed ablation variants and existing segmentation methods, demonstrating improved adaptation performance and robustness across heterogeneous medical data distributions. Moreover, FedSAM-3D achieved improved generalization on unseen external datasets, including cross-modality evaluation, highlighting its ability to enhance the transferability of medical foundation models.

Conclusion

FedSAM-3D provides an effective paradigm for federated transfer of medical foundation models, achieving improved adaptation performance and generalization while avoiding direct sharing of raw medical data across institutions.

Significance

FedSAM-3D provides a parameter-efficient approach for transferring medical foundation models across institutions without directly sharing raw data, facilitating their potential deployment in diverse clinical environments. Our code is available at https://github.com/huavhuahua/FedSAM-3D.