Guix-HPC is an effort to optimize GNU Guix for reproducible scientific workflows in high-performance computing (HPC). Our introductory article explains how we got started and gives an overview of what we want to achieve. We regularly publish articles on this Web site highlighting specific features or achievements. Stay tuned!
Guix-HPC is a joint software development project currently involving three research institutes: Inria, the Max Delbrück Center for Molecular Medicine (MDC), University of Tennessee Health Science Center (UTHSC) and the Utrecht Bioinformatics Center (UBC).
Articles
The material below covers our work and motivation for Guix-HPC:
- Toward practical transparent verifiable and long-term reproducible research using Guix, Nature Scientific Data (volume 9, Oct. 2022)
- Reproducibility and Performance: Why Choose?, CiSE May–June 2022 issue (volume 24, issue 3)
- [Re] Storage Tradeoffs in a Collaborative Backup Service for Mobile Devices, ReScience C, June 2020
- PiGx: Reproducible Genomics Analysis Pipelines with GNU Guix, GigaScience ICG-13 (video), Dec. 2018
- Reproducible genomics analysis pipelines with GNU Guix, Apr. 2018
- Reproducible and User-Controlled Software Environments in HPC with Guix (slides), paper presented at the 2nd International Workshop on Reproducibility in Parallel Computing (RepPar), Aug. 2015
Check out the Guix web site for more publications.
Talks
The following talks present our work and vision for reproducible software deployment:
- Reproducibility and performance: why choose? CPU tuning in GNU Guix, FOSDEM, Feb. 2023
- Guix, toward practical transparent, verifiable and long-term reproducible research, FOSDEM, Feb. 2023
- Towards reproducible Jupyter notebooks, FOSDEM, Feb. 2020
- Beyond Bundles—Reproducible Software Environments with GNU Guix, CERN Computing Seminars, May 2018
- Tying software deployment to scientific workflows, FOSDEM, Feb. 2018
- Reproducible and user-controlled software management in HPC with GNU Guix (PDF), BOSC, July 2017
- Optimized and Reproducible HPC Deployment, FOSDEM, Feb. 2017
- Workflow Management with GNU Guix, FOSDEM, Feb. 2017
Check out the Guix web site for more talks.
Code
Most of the code developed for Guix-HPC aims to consolidate the code base and package collection of Guix proper, and thus be pushed upstream. This has already given rise to a large collection of bioinformatics, algebra, and R packages, as well as features to simplify Guix deployment on clusters.
Some auxiliary tools and package sets are maintained elsewhere, or kept in a staging area until they are mature enough to be submitted for inclusion in Guix:
- The Guix Workflow Language, a lightweight framework implementing reproducible computational pipelines.
- hpcguix-web is a web interface that allows users to search for packages and guides them the installation and the job submission process.
- Check out the third-party channels providing scientific and HPC packages!
All this is free software that you are welcome to use and contribute to!
Cluster Deployments
Here are known deployments of Guix on clusters and contact information:
Max Delbrück Center for Molecular Medicine (Germany)
- 250-node cluster + workstations
- contact: Ricardo Wurmus
Utrecht Bioinformatics Center (The Netherlands)
- 68-node cluster (1,000+ cores)
- contact: Roel Janssen
Australian Centre for Ecogenomics (Australia)
- 21-node cluster (1,000 cores)
- contact: Ben Woodcroft
PlaFRIM (France)
- 120-node heterogeneous cluster (3,000+ cores)
- contact: Ludovic Courtès
GriCAD (France)
- 72-node “Dahu” cluster (1,000+ cores)
- contact: Violaine Louvet, Pierre-Antoine Bouttier
GliCID (France)
- 230-node cluster (4,000+ cores)
- contact: Yann Dupont
UTHSC (USA)
- 11-node HPC Octopus cluster 264 cores). Pangenome and genetics research cluster
- contact: Pjotr Prins
Grid’5000 (France)
31 clusters, 828 nodes (12,000+ cores)
contact: Dimitri Delabroye, Lucas Nussbaum
If you would like to be listed here, please email us at guix-hpc@gnu.org
.
Contact
There are several ways we can get in touch:
- Subscribe to the
guix-science
mailing list. - Email us at
guix-hpc@gnu.org
. - Come chat on the
#guix-hpc
IRC channel of the Libera Chat network… - … or on the French-speaking Café Guix Mattermost channel
Logotype
The Guix-HPC logo was designed by Roel Janssen as a derivative of the Guix logo. It is distributed under the following terms:
Copyright © 2015 Luis Felipe López Acevedo
Copyright © 2017 Roel Janssen
Permission is granted to copy, distribute and/or modify this work under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License (CC-BY-SA 4.0).
Join us!
If you are using Guix in an HPC context at your workplace, or if you would like to discuss ways to address your own HPC use cases, or if you are an HPC vendor interested in improving the software deployment experience for your users, please consider joining us!