MOTIVATION
The reproducibility crisis in bioinformatics and computational biology can be mitigated by using Continuous Integration (CI), a proven software engineering practice, to automate the testing of analysis pipelines and models. However, the adoption of CI in bioinformatics is limited as existing solutions are typically tightly linked to specific CI platforms, specific HPC job schedulers, or require actions from HPC administrators.
RESULTS
We developed hpci, a platform-agnostic, open-source utility that integrates CI platforms (e.g., GitHub Actions, GitLab CI) with common HPC job schedulers (Slurm, PBS). hpci operates as a standard unprivileged user via SSH, and requires no cluster-side installation. It terminates on the CI runner using the exact same exit code generated by the HPC job, providing feedback to researchers and preventing downstream CI steps if there are errors. We demonstrate hpci's utility across five diverse computational biology workflows on HPC, highlighting the value of automated testing and continuous analysis for verifying reproducibility in bioinformatics workflows.
AVAILABILITY
hpci is implemented in Haskell and is available under the GPL-3.0 licence. Source code and binaries can be downloaded from https://github.com/Garvan-Data-Science-Platform/hpci.Source code is archived in Software Heritage via Zenodo (https://zenodo.org/records/19587650).
This work presents a software-engineering perspective for developing reproducible bioinformatics pipelines, with emphasis on pipeline-specific code, and reinterpreted the SOLID principles in this context at two levels: workflow management systems and pipeline implementation.
D. Pérez-Rodríguez, Alba Nogueira-Rodríguez, Jorge Vieira et al.· Frontiers in Bioinformatics· 0 citations
Cloud computing and high-performance computing (HPC) typically follow different paradigms: cloud services are often orchestrated using Kubernetes, whereas HPC workloads are managed through batch schedulers such as Slurm. Growing demand for shared computational resources increases the need for interoperability between t...
(English) High-performance computing (HPC) platforms are evolving towards increasingly complex architectures: many-core CPUs with multi-level NUMA hierarchies, heterogeneity with multiple classes of accelerators and higher-capacity interconnects. The increasing complexity and variety of resources in these machines make...
The core design goal is to let users decide the application’s hardware mapping and orchestration by editing only the high-level recipe—without modifying the maintained source code or binding the application to a particular runtime system.
Youngjun Lee, Klaus Weide, Wesley Kwiecinski et al.· The international journal of...· 1 citation
LineShine is presented, a newly deployed exascale supercomputer whose online acceleration architecture provides a new design point for converged HPC and AI and demonstrates that its balanced architecture and software stack can translate system capability into high throughput, robust scalability, and sustained scientifi...
During the School of Humanities, Arts, and Social Sciences’ 75th Anniversary Symposium, President Sally Kornbluth said “MIT has never needed your particular insight and guidance more than we do now.”