HPC & reproducible research in Guix 1.2.0
Version 1.2.0 of GNU Guix was announced yesterday. Some 200 people contributed more than 10,000 commits since the previous release. This post focuses on important changes for HPC users, admins, and practitioners made since version 1.1.0 was released in April 2020.
Package customization and faster “relocatable packs” for HPC
We’re giving users more flexibility on the command line, with the
addition of three package transformation
options:
--with-debug-info
(always debug in good
conditions!),
--with-c-toolchain
, and --without-tests
. Consider this example:
guix build octave-cli \
--with-c-toolchain=fftw=gcc-toolchain@10 \
--with-c-toolchain=fftwf=gcc-toolchain@10
The command above builds a variant of the fftw
and fftwf
packages using
version 10 of gcc-toolchain
instead of the default tool chain, and then builds
a variant of the GNU Octave command-line interface using them. GNU Octave
itself is also built with gcc-toolchain@10
.
This other example builds the Hardware Locality (hwloc) library and its dependents up to intel-mpi-benchmarks with the Clang C compiler:
guix build --with-c-toolchain=hwloc=clang-toolchain \
intel-mpi-benchmarks
--with-c-toolchain
can come in handy if you want to check the impact
of the compiler toolchain on the performance of your software stack.
The guix pack
command creates “application bundles” that can be used to deploy
software on machines that do not run Guix (yet!), such as HPC clusters.
Since its inception in
2017,
it has seen a number of improvements. Code in relocatable packs can now
run more efficiently thanks to the addition of the fakechroot
engine,
which we covered back in
May.
Reproducible research workflows
The new option --path
of guix graph
,
shows the shortest path between two nodes. The example below shows the
shortest path between the packages gmsh and cunit:
$ guix graph --path gmsh cunit
gmsh@4.6.0
metis@5.1.0
openblas@0.3.9
cunit@2.1-3
Moreover, the guix repl
command can now be passed a script, which eases package exploration in
Guile,
notably when dealing with the new Scheme (guix transformation)
module for
package transformations.
On the side of long-term archival of all the software Guix packages refer to,
Guix now serves the file sources.json
that is ingested by Software Heritage via the
nixguix
loader.
In addition to the “archival” check of guix lint
which sends a “save”
request to Software Heritage for the specified packages. More packages are
continuously archived.
Packages
Here are highlights among the 2,179 packages added and 4,487 packages upgraded since the previous release:
- Gmsh has been upgraded to 4.6.0.
- MPICH was added, version 3.3.2.
- Open MPI is now at 4.0.5.
- GCC 10.2.0 is available and LLVM, Clang 11.0.0 as well.
- Julia has been upgraded to 1.5.2.
- MPI4PY is at 3.0.3.
- For statisticians, there’s now a total of 1,488 R packages, many of which come from Bioconductor 3.11.
The manual has been improved in various ways. In particular, the “Programming interface” section has been expounded and should be helpful for those writing package definitions. It is fully translated into French, German, and Spanish, with partial translations in Russian and Chinese.
Do not miss the release notes for more.
Try it!
Check out the download options. If
you’re on Debian or a derivative distribution, you will soon be able to
apt install guix
.
If you’re a first-time user, take a look at the new “Getting
Started”
section and try out guix help
to get an overview of the available
commands.
Enjoy the new release!
Unless otherwise stated, blog posts on this site are copyrighted by their respective authors and published under the terms of the CC-BY-SA 4.0 license and those of the GNU Free Documentation License (version 1.3 or later, with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts).