Tutorial setup

If you have not done the prior sections, you’ll need to start the docker image:

docker run -it ghcr.io/spack/tutorial:hpcic26

and then set Spack up like this:

git clone --depth=2 --branch=releases/v1.2 https://github.com/spack/spack
. spack/share/spack/setup-env.sh
spack mirror add tutorial /mirror
spack buildcache keys --install --trust --yes-to-all
spack bootstrap now
spack compiler find

See the Basic Installation Tutorial for full details on setup. For more help, join us in the #tutorial channel on Slack – get an invitation at slack.spack.io

Stacks Tutorial

Spack environments are a powerful tool not just for managing a single user’s project, but for deploying entire software stacks. In this tutorial we will see how to use them to manage large deployments, a common pattern in HPC centers.

What usually characterizes these deployments, compared to a typical single-user environment, is the need to provide the same set of packages built against a variety of configurations: different MPI libraries, LAPACK implementations, or compilers.

Below, we’ll build a representative example of such a deployment. Our goal is to install netlib-scalapack compiled with gcc@16, which is newer than the system-provided gcc@15, and linked against:

  • two MPI libraries (openmpi and mpich)

  • two LAPACK providers (openblas and netlib-lapack)

We’ll also install py-scipy linked against openblas.

We’ll first focus on how to configure and install the software correctly. Then we’ll discuss how to make it accessible to users through filesystem views and module files.

Installing a Software Stack

Let’s start by creating an environment in a directory of our choice:

$ spack env activate --create ~/stacks
$ spack env status
==> In environment /home/spack/stacks

The first step is to install the compiler we want to use. Let’s write a minimal spack.yaml that contains just the compiler and lmod, which we’ll need later to generate module files:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  specs:
  - gcc@16 %gcc@15
  - lmod@8.7.67 %gcc@15
  view: false

Concretize and install:

$ spack concretize
==> Concretized 2 specs:
[b]  hkkcdmj  gcc@16.1.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs+profiled+strip build_system=autotools build_type=Release languages:='c,c++,fortran' platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  6ene6zr	  ^binutils@2.46.1~debuginfod+gas~gprofng~headers~interwork+ld~libiberty~lto~nls~pgo+plugins build_system=autotools compress_debug_sections=zlib libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  mmywg7x	  ^compiler-wrapper@1.1.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  ao2onuz	  ^diffutils@3.12 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  vbwvgwx	      ^libiconv@1.18 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  z7wqeps	  ^gawk@5.4.0~nls build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  k25xiih	      ^readline@8.3 build_system=autotools patches:=21f0a03,72dee13,e273643 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[e]  yjlog5x	  ^gcc@15.2.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs~profiled~strip build_system=autotools build_type=RelWithDebInfo languages:='c,c++,fortran' platform=linux os=ubuntu26.04 target=x86_64_v3
[+]  xm76mt3	  ^gcc-runtime@15.2.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[e]  yc4n2pp	  ^glibc@2.43 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  r4lhaok	  ^gmake@4.4.1~guile build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  xpmsy5x	  ^gmp@6.3.0+cxx build_system=autotools libs:=shared,static patches:=28a6673 platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  jo3eg4r	      ^autoconf@2.72 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  yhkgfai	      ^automake@1.18.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  y6uqrto	      ^m4@1.4.21+sigsegv build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  phcmfqk		  ^libsigsegv@2.15 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  3gdq456	  ^libtool@2.5.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  mguwetc	      ^file@5.46+static build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  hhjyyqy		  ^xz@5.8.3~pic build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  punbqrx	      ^findutils@4.10.0 build_system=autotools patches:=440b954 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  gxmoxea	  ^mpc@1.4.1 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  melg7ga	  ^mpfr@4.2.2 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  sf5iutv	      ^autoconf-archive@2024.10.16 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[+]  bvphs3b	  ^perl@5.42.0+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  27d4iyp	      ^berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  cbtgjrh	      ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  cq4dkuc	      ^gdbm@1.26 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  ohmdb2l	      ^less@692 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  jvlaabk	  ^texinfo@7.2~xs build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  sle3ix4	      ^gettext@1.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  ujlg2ua		  ^libxml2@2.15.3+pic~python+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  f5xe4px		  ^tar@1.35 build_system=autotools zip=pigz platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  afklka7		      ^pigz@2.8 build_system=makefile platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  ekvivpv	      ^ncurses@6.6~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  g72d7i3	  ^zlib-ng@2.3.3+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  63aruxk	  ^zstd@1.5.7+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  wf22jsg  lmod@8.7.67+auto_swap~redirect build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  ywyq4f5	  ^bc@1.08.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  litd2mh	      ^ed@1.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  q66mudl	  ^lua@5.4.8+shared build_system=makefile fetcher=curl platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  j2fa7xl	      ^curl@8.20.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2+nghttp2 build_system=autotools libs:=shared,static tls:=openssl platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  4lyasaz		  ^nghttp2@1.67.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  33yozp5		  ^openssl@3.6.1~docs+shared build_system=generic certs=mozilla platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  axeuqg3		      ^ca-certificates-mozilla@2026-03-19 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  4qgksuy	      ^unzip@6.0 build_system=makefile patches:=179330d,24582ff,251d575,3371314,44599c8,47e9def,4e5a081,59c0983,64f6498,74bc961,7d8e5c7,81ca46c,881d2ed,aced0f2,b6f64d7,b7a14c3,c9a863e,ee9e260,f6f6236,f88b9d4,fde8f9d platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  su5b2hs	  ^lua-luafilesystem@1.9.0 build_system=lua platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  egzwtik	  ^lua-luaposix@36.3 build_system=lua platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  bkzcu2s	      ^libxcrypt@4.5.2~obsolete_api build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  yvl6jpi	  ^pkgconf@2.5.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  ci53fkm	  ^procps@4.0.6+nls build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  a27iqiv	  ^sed@4.9 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  d4ibosb	  ^tcl@8.6.17 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0

$ spack install
[ ] 4qgksuy unzip@6.0 fetching from build cache (0s)
[ ] hkkcdmj gcc@16.1.0 fetching from build cache (0s)
[ ] 4qgksuy unzip@6.0 relocating (0s)
[+] 4qgksuy unzip@6.0 /home/spack/spack/opt/spack/linux-x86_64_v3/unzip-6.0-4qgksuyohalb7cfo76mqqjlmndeh3tyg (0s)
[ ] q66mudl lua@5.4.8 fetching from build cache (0s)
[ ] hkkcdmj gcc@16.1.0 relocating (0s)
[ ] q66mudl lua@5.4.8 relocating (0s)
[+] q66mudl lua@5.4.8 /home/spack/spack/opt/spack/linux-x86_64_v3/lua-5.4.8-q66mudlsappt33u2frzcijegjay54ep3 (0s)
[ ] su5b2hs lua-luafilesystem@1.9.0 fetching from build cache (0s)
[ ] egzwtik lua-luaposix@36.3 fetching from build cache (0s)
[ ] su5b2hs lua-luafilesystem@1.9.0 relocating (1s)
[ ] egzwtik lua-luaposix@36.3 relocating (1s)
[+] su5b2hs lua-luafilesystem@1.9.0 /home/spack/spack/opt/spack/linux-x86_64_v3/lua-luafilesystem-1.9.0-su5b2hsgwgk2a4b7hkys2r23kf3me6lh (1s)
[+] egzwtik lua-luaposix@36.3 /home/spack/spack/opt/spack/linux-x86_64_v3/lua-luaposix-36.3-egzwtikjsfubsjiqpwdjemys3xqv2ytf (1s)
[ ] wf22jsg lmod@8.7.67 fetching from build cache (0s)
[ ] wf22jsg lmod@8.7.67 relocating (0s)
[+] wf22jsg lmod@8.7.67 /home/spack/spack/opt/spack/linux-x86_64_v3/lmod-8.7.67-wf22jsggn5vfsyoyuqm6ban76y7333we (0s)
[+] hkkcdmj gcc@16.1.0 /home/spack/spack/opt/spack/linux-x86_64_v3/gcc-16.1.0-hkkcdmj2guegqqkt2w3e3owuklnyvgek (2s)

Spec groups

We are now ready to build the software stack on top of this compiler. To do that cleanly, we’ll use spec groups, which allow you to organize specs into named collections, each with its own configuration and concretization order.

Let’s start with a single spec and a two-group spack.yaml:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  specs:
  - group: compiler
    specs:
    - gcc@16 %gcc@15
    - lmod@8.7.67 %gcc@15
  - group: stack
    needs: [compiler]
    specs:
    - netlib-scalapack %gcc@16 ^openblas ^openmpi
  view: false

The environment declares two groups:

  1. The compiler group contains GCC 16 and lmod (both already installed).

  2. The stack group lists the software we want to deploy.

The needs: [compiler] declaration ensures that the compiler group is always concretized before the stack group, making its specs available for reuse.

Let’s concretize to see this in action:

$ spack concretize
==> Concretizing the 'compiler' group of specs
==> Concretizing the 'stack' group of specs
==> Concretized 3 specs:
[+]  hkkcdmj  gcc@16.1.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs+profiled+strip build_system=autotools build_type=Release languages:='c,c++,fortran' platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  6ene6zr	  ^binutils@2.46.1~debuginfod+gas~gprofng~headers~interwork+ld~libiberty~lto~nls~pgo+plugins build_system=autotools compress_debug_sections=zlib libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  mmywg7x	  ^compiler-wrapper@1.1.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  ao2onuz	  ^diffutils@3.12 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  vbwvgwx	      ^libiconv@1.18 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  z7wqeps	  ^gawk@5.4.0~nls build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  k25xiih	      ^readline@8.3 build_system=autotools patches:=21f0a03,72dee13,e273643 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[e]  yjlog5x	  ^gcc@15.2.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs~profiled~strip build_system=autotools build_type=RelWithDebInfo languages:='c,c++,fortran' platform=linux os=ubuntu26.04 target=x86_64_v3
[+]  xm76mt3	  ^gcc-runtime@15.2.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[e]  yc4n2pp	  ^glibc@2.43 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  r4lhaok	  ^gmake@4.4.1~guile build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  xpmsy5x	  ^gmp@6.3.0+cxx build_system=autotools libs:=shared,static patches:=28a6673 platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  jo3eg4r	      ^autoconf@2.72 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  yhkgfai	      ^automake@1.18.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  y6uqrto	      ^m4@1.4.21+sigsegv build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  phcmfqk		  ^libsigsegv@2.15 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  3gdq456	  ^libtool@2.5.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  mguwetc	      ^file@5.46+static build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  hhjyyqy		  ^xz@5.8.3~pic build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  punbqrx	      ^findutils@4.10.0 build_system=autotools patches:=440b954 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  gxmoxea	  ^mpc@1.4.1 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  melg7ga	  ^mpfr@4.2.2 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  sf5iutv	      ^autoconf-archive@2024.10.16 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[+]  bvphs3b	  ^perl@5.42.0+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  27d4iyp	      ^berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  cbtgjrh	      ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  cq4dkuc	      ^gdbm@1.26 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  ohmdb2l	      ^less@692 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  jvlaabk	  ^texinfo@7.2~xs build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  sle3ix4	      ^gettext@1.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  ujlg2ua		  ^libxml2@2.15.3+pic~python+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  f5xe4px		  ^tar@1.35 build_system=autotools zip=pigz platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  afklka7		      ^pigz@2.8 build_system=makefile platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  ekvivpv	      ^ncurses@6.6~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  g72d7i3	  ^zlib-ng@2.3.3+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  63aruxk	  ^zstd@1.5.7+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  wf22jsg  lmod@8.7.67+auto_swap~redirect build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  ywyq4f5	  ^bc@1.08.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  litd2mh	      ^ed@1.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  q66mudl	  ^lua@5.4.8+shared build_system=makefile fetcher=curl platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  j2fa7xl	      ^curl@8.20.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2+nghttp2 build_system=autotools libs:=shared,static tls:=openssl platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  4lyasaz		  ^nghttp2@1.67.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  33yozp5		  ^openssl@3.6.1~docs+shared build_system=generic certs=mozilla platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  axeuqg3		      ^ca-certificates-mozilla@2026-03-19 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[+]  4qgksuy	      ^unzip@6.0 build_system=makefile patches:=179330d,24582ff,251d575,3371314,44599c8,47e9def,4e5a081,59c0983,64f6498,74bc961,7d8e5c7,81ca46c,881d2ed,aced0f2,b6f64d7,b7a14c3,c9a863e,ee9e260,f6f6236,f88b9d4,fde8f9d platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  su5b2hs	  ^lua-luafilesystem@1.9.0 build_system=lua platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  egzwtik	  ^lua-luaposix@36.3 build_system=lua platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  bkzcu2s	      ^libxcrypt@4.5.2~obsolete_api build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  yvl6jpi	  ^pkgconf@2.5.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  ci53fkm	  ^procps@4.0.6+nls build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  a27iqiv	  ^sed@4.9 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  d4ibosb	  ^tcl@8.6.17 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  faqiqu4  netlib-scalapack@2.2.3~ipo~pic+shared build_system=cmake build_type=Release generator=make platform=linux os=ubuntu26.04 target=x86_64_v3 %c,fortran=gcc@16.1.0
[b]  kw6i4f4	  ^cmake@3.31.11~doc+ncurses+ownlibs~qtgui build_system=generic build_type=Release platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  i3sc4sa	      ^curl@8.20.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2+nghttp2 build_system=autotools libs:=shared,static tls:=openssl platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  qjecybs		  ^nghttp2@1.67.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  rzn5joe		      ^diffutils@3.12 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  mladgc5		  ^openssl@3.6.1~docs+shared build_system=generic certs=mozilla platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  romubi2	      ^ncurses@6.6~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  gc236kh	      ^zlib-ng@2.3.3+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  nn3xbub	  ^gcc-runtime@16.1.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  lwhtcck	  ^gmake@4.4.1~guile build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  uh2kn3c	  ^openblas@0.3.33~bignuma~consistent_fpcsr+dynamic_dispatch+fortran~ilp64+locking+pic+shared~static build_system=makefile patches:=723ddc1 symbol_suffix=none threads=none platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx,fortran=gcc@16.1.0
[b]  s6hcsd5	  ^openmpi@5.0.10+atomics~cuda~debug+fortran~gpfs~internal-hwloc~internal-libevent~internal-pmix~ipv6~java~lustre~memchecker~openshmem~rocm~romio+rsh~static~two_level_namespace+vt+wrapper-rpath build_system=autotools fabrics:=none patches:=646eb1a romio-filesystem:=none schedulers:=none platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx,fortran=gcc@16.1.0
[b]  uesc5yd	      ^autoconf@2.72 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  hkvksc3		  ^m4@1.4.21+sigsegv build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  4v4bncv		      ^libsigsegv@2.15 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  qhb7ywb	      ^automake@1.18.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  fkki5gm	      ^hwloc@2.13.0~cairo~cuda~gl~level_zero~libudev+libxml2~nvml~opencl+pci~rocm build_system=autotools libs:=shared,static patches:=b4db98b platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  t5frspj		  ^libpciaccess@0.17 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  nc2zhu5		      ^util-macros@1.20.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  3rlsf7j		  ^libxml2@2.15.3+pic~python+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  xrsocgc		      ^libiconv@1.18 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  rrvhely		      ^xz@5.8.3~pic build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  ul3skja	      ^libevent@2.1.12+openssl build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  wiglecf	      ^libtool@2.5.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  ra2jarr		  ^file@5.46+static build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  mzuiikv		      ^zstd@1.5.7+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  fep3wol		  ^findutils@4.10.0 build_system=autotools patches:=440b954 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  qdxpkom		      ^gettext@1.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  u62ty4d			  ^tar@1.35 build_system=autotools zip=pigz platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  7ppidie			      ^pigz@2.8 build_system=makefile platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  vn7wvoe	      ^numactl@2.0.19 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  plgdjpa	      ^openssh@10.3p1+gssapi build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  7a4l5v5		  ^krb5@1.22.2+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  qu5uhax		      ^bison@3.8.2~color build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  4binz2p		  ^libedit@3.1-20251016 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  aca7p7l		  ^libxcrypt@4.5.2~obsolete_api build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  c6ebuxy	      ^perl@5.42.0+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  zsx5k5v		  ^berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  5kv66tq		  ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  2qw4v66		  ^gdbm@1.26 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  teh56fo		      ^readline@8.3 build_system=autotools patches:=21f0a03,72dee13,e273643 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  d2d5je7		  ^less@692 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  rws4suo	      ^pkgconf@2.5.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  2vgljh5	      ^pmix@6.1.0~munge~python build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  wwm2ix6	      ^prrte@4.1.0 build_system=autotools patches:=64faa1a,91b28f5 schedulers:=none platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  4rup75b		  ^flex@2.6.3+lex~nls build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0

Notice that the compiler group is processed first, and gcc@16 is immediately reused by the stack group.

We just used %gcc@16 to pin the compiler, but it’s a coarse constraint that must be repeated on every spec and does not distinguish between C, C++, and Fortran compilers. Spec groups offer a better tool: the override block, which scopes any Spack configuration to a single group.

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  specs:
  - group: compiler
    specs:
    - gcc@16
    - lmod@8.7.67
    override:
      packages:
        c:
          prefer: [gcc@15]
        cxx:
          prefer: [gcc@15]
        fortran:
          prefer: [gcc@15]
  - group: stack
    needs: [compiler]
    specs:
    - netlib-scalapack ^openblas ^openmpi
    override:
      packages:
        c:
          prefer: [gcc@16]
        cxx:
          prefer: [gcc@16]
        fortran:
          prefer: [gcc@16]
  view: false

Here we used the override block to set compiler preferences at the language level, once for each group:

  1. The compiler group prefers gcc@15

  2. The stack group prefers gcc@16

These language preferences are expressed once and apply automatically to every spec in the group.

Tuning concretizer options

Now we add a second MPI variant to the stack, listing both ^openmpi and ^mpich variants of netlib-scalapack:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  specs:
  - group: compiler
    specs:
    - gcc@16
    - lmod@8.7.67
    override:
      packages:
        c:
          prefer: [gcc@15]
        cxx:
          prefer: [gcc@15]
        fortran:
          prefer: [gcc@15]
  - group: stack
    needs: [compiler]
    specs:
    - netlib-scalapack ^openblas ^openmpi
    - netlib-scalapack ^openblas ^mpich
    override:
      packages:
        c:
          prefer: [gcc@16]
        cxx:
          prefer: [gcc@16]
        fortran:
          prefer: [gcc@16]
  view: false

The stack group now has two conflicting configurations, so, before concretizing, let’s check the concretizer:unify option:

$ spack config get concretizer | grep unify
  unify: true

With unify: true, the concretizer restricts the environment to a single configuration of each package within its root unification set: the nodes reachable from the root specs via link or run edges. Pure build dependencies, which fall outside the set, are not affected:

_images/stacks-unify.svg

This is the right default for a single-user environment, since it ensures a filesystem view can be built without conflicts. A software stack, however, requires multiple configurations of the same package by design, so we need to relax this constraint.

Since only the stack group has conflicting configurations, we add unify: false to its override block and concretize:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  specs:
  - group: compiler
    specs:
    - gcc@16
    - lmod@8.7.67
    override:
      packages:
        c:
          prefer: [gcc@15]
        cxx:
          prefer: [gcc@15]
        fortran:
          prefer: [gcc@15]
  - group: stack
    needs: [compiler]
    specs:
    - netlib-scalapack ^openblas ^openmpi
    - netlib-scalapack ^openblas ^mpich
    override:
      packages:
        c:
          prefer: [gcc@16]
        cxx:
          prefer: [gcc@16]
        fortran:
          prefer: [gcc@16]
      concretizer:
        unify: false
  view: false
$ spack concretize
==> Concretizing the 'compiler' group of specs
==> Concretizing the 'stack' group of specs
==> Starting concretization pool with 2 processes
==> Concretized 4 specs:
[+]  hkkcdmj  gcc@16.1.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs+profiled+strip build_system=autotools build_type=Release languages:='c,c++,fortran' platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  6ene6zr	  ^binutils@2.46.1~debuginfod+gas~gprofng~headers~interwork+ld~libiberty~lto~nls~pgo+plugins build_system=autotools compress_debug_sections=zlib libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  mmywg7x	  ^compiler-wrapper@1.1.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  ao2onuz	  ^diffutils@3.12 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  vbwvgwx	      ^libiconv@1.18 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  z7wqeps	  ^gawk@5.4.0~nls build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  k25xiih	      ^readline@8.3 build_system=autotools patches:=21f0a03,72dee13,e273643 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[e]  yjlog5x	  ^gcc@15.2.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs~profiled~strip build_system=autotools build_type=RelWithDebInfo languages:='c,c++,fortran' platform=linux os=ubuntu26.04 target=x86_64_v3
[+]  xm76mt3	  ^gcc-runtime@15.2.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[e]  yc4n2pp	  ^glibc@2.43 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  r4lhaok	  ^gmake@4.4.1~guile build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  xpmsy5x	  ^gmp@6.3.0+cxx build_system=autotools libs:=shared,static patches:=28a6673 platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  jo3eg4r	      ^autoconf@2.72 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  yhkgfai	      ^automake@1.18.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  y6uqrto	      ^m4@1.4.21+sigsegv build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  phcmfqk		  ^libsigsegv@2.15 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  3gdq456	  ^libtool@2.5.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  mguwetc	      ^file@5.46+static build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  hhjyyqy		  ^xz@5.8.3~pic build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  punbqrx	      ^findutils@4.10.0 build_system=autotools patches:=440b954 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  gxmoxea	  ^mpc@1.4.1 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  melg7ga	  ^mpfr@4.2.2 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  sf5iutv	      ^autoconf-archive@2024.10.16 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[+]  bvphs3b	  ^perl@5.42.0+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  27d4iyp	      ^berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  cbtgjrh	      ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  cq4dkuc	      ^gdbm@1.26 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  ohmdb2l	      ^less@692 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  jvlaabk	  ^texinfo@7.2~xs build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  sle3ix4	      ^gettext@1.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  ujlg2ua		  ^libxml2@2.15.3+pic~python+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  f5xe4px		  ^tar@1.35 build_system=autotools zip=pigz platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  afklka7		      ^pigz@2.8 build_system=makefile platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  ekvivpv	      ^ncurses@6.6~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  g72d7i3	  ^zlib-ng@2.3.3+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  63aruxk	  ^zstd@1.5.7+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  wf22jsg  lmod@8.7.67+auto_swap~redirect build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  ywyq4f5	  ^bc@1.08.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  litd2mh	      ^ed@1.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  q66mudl	  ^lua@5.4.8+shared build_system=makefile fetcher=curl platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  j2fa7xl	      ^curl@8.20.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2+nghttp2 build_system=autotools libs:=shared,static tls:=openssl platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  4lyasaz		  ^nghttp2@1.67.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  33yozp5		  ^openssl@3.6.1~docs+shared build_system=generic certs=mozilla platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  axeuqg3		      ^ca-certificates-mozilla@2026-03-19 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[+]  4qgksuy	      ^unzip@6.0 build_system=makefile patches:=179330d,24582ff,251d575,3371314,44599c8,47e9def,4e5a081,59c0983,64f6498,74bc961,7d8e5c7,81ca46c,881d2ed,aced0f2,b6f64d7,b7a14c3,c9a863e,ee9e260,f6f6236,f88b9d4,fde8f9d platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  su5b2hs	  ^lua-luafilesystem@1.9.0 build_system=lua platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  egzwtik	  ^lua-luaposix@36.3 build_system=lua platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  bkzcu2s	      ^libxcrypt@4.5.2~obsolete_api build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  yvl6jpi	  ^pkgconf@2.5.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  ci53fkm	  ^procps@4.0.6+nls build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  a27iqiv	  ^sed@4.9 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  d4ibosb	  ^tcl@8.6.17 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  y2obsoc  netlib-scalapack@2.2.3~ipo~pic+shared build_system=cmake build_type=Release generator=make platform=linux os=ubuntu26.04 target=x86_64_v3 %c,fortran=gcc@16.1.0
[b]  kw6i4f4	  ^cmake@3.31.11~doc+ncurses+ownlibs~qtgui build_system=generic build_type=Release platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  i3sc4sa	      ^curl@8.20.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2+nghttp2 build_system=autotools libs:=shared,static tls:=openssl platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  qjecybs		  ^nghttp2@1.67.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  rzn5joe		      ^diffutils@3.12 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  mladgc5		  ^openssl@3.6.1~docs+shared build_system=generic certs=mozilla platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  romubi2	      ^ncurses@6.6~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  gc236kh	      ^zlib-ng@2.3.3+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  nn3xbub	  ^gcc-runtime@16.1.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  lwhtcck	  ^gmake@4.4.1~guile build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  fxetsgw	  ^mpich@5.0.1~argobots~cuda+fortran+hwloc+hydra~level_zero+libxml2+pci~rocm+romio~slurm~vci~verbs+wrapperrpath~xpmem build_system=autotools datatype-engine=auto device=ch4 netmod=ofi pmi=default platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx,fortran=gcc@16.1.0
[b]  fep3wol	      ^findutils@4.10.0 build_system=autotools patches:=440b954 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  qdxpkom		  ^gettext@1.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  u62ty4d		      ^tar@1.35 build_system=autotools zip=pigz platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  7ppidie			  ^pigz@2.8 build_system=makefile platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  fkki5gm	      ^hwloc@2.13.0~cairo~cuda~gl~level_zero~libudev+libxml2~nvml~opencl+pci~rocm build_system=autotools libs:=shared,static patches:=b4db98b platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  o3dcvlb	      ^libfabric@2.5.1~asan~cuda~debug~kdreg~level_zero~lsan~rocm~tsan~ubsan~uring build_system=autotools fabrics:=sockets,tcp,udp platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  t5frspj	      ^libpciaccess@0.17 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  nc2zhu5		  ^util-macros@1.20.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  3rlsf7j	      ^libxml2@2.15.3+pic~python+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  xrsocgc		  ^libiconv@1.18 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  rrvhely		  ^xz@5.8.3~pic build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  rws4suo	      ^pkgconf@2.5.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  vuoqpmo	      ^python@3.14.5+bz2+ctypes+dbm~debug~freethreading+libxml2+lzma~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~static~tests~tkinter+uuid+zlib+zstd build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  5kv66tq		  ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  kqm2d3g		  ^expat@2.8.1+libbsd build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  2zdnupj		      ^libbsd@0.12.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  ypuui7v			  ^libmd@1.1.0 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  2qw4v66		  ^gdbm@1.26 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  pkl6qup		  ^libffi@3.5.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  teh56fo		  ^readline@8.3 build_system=autotools patches:=21f0a03,72dee13,e273643 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  qy73hym		  ^sqlite@3.53.1+column_metadata+fts+rtree build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  2wcnb6n		  ^util-linux-uuid@2.41 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  mzuiikv		  ^zstd@1.5.7+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  uh2kn3c	  ^openblas@0.3.33~bignuma~consistent_fpcsr+dynamic_dispatch+fortran~ilp64+locking+pic+shared~static build_system=makefile patches:=723ddc1 symbol_suffix=none threads=none platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx,fortran=gcc@16.1.0
[b]  faqiqu4  netlib-scalapack@2.2.3~ipo~pic+shared build_system=cmake build_type=Release generator=make platform=linux os=ubuntu26.04 target=x86_64_v3 %c,fortran=gcc@16.1.0
[b]  s6hcsd5	  ^openmpi@5.0.10+atomics~cuda~debug+fortran~gpfs~internal-hwloc~internal-libevent~internal-pmix~ipv6~java~lustre~memchecker~openshmem~rocm~romio+rsh~static~two_level_namespace+vt+wrapper-rpath build_system=autotools fabrics:=none patches:=646eb1a romio-filesystem:=none schedulers:=none platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx,fortran=gcc@16.1.0
[b]  uesc5yd	      ^autoconf@2.72 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  hkvksc3		  ^m4@1.4.21+sigsegv build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  4v4bncv		      ^libsigsegv@2.15 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  qhb7ywb	      ^automake@1.18.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  ul3skja	      ^libevent@2.1.12+openssl build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  wiglecf	      ^libtool@2.5.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  ra2jarr		  ^file@5.46+static build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  vn7wvoe	      ^numactl@2.0.19 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  plgdjpa	      ^openssh@10.3p1+gssapi build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  7a4l5v5		  ^krb5@1.22.2+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  qu5uhax		      ^bison@3.8.2~color build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  4binz2p		  ^libedit@3.1-20251016 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  aca7p7l		  ^libxcrypt@4.5.2~obsolete_api build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  c6ebuxy	      ^perl@5.42.0+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  zsx5k5v		  ^berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  d2d5je7		  ^less@692 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  2vgljh5	      ^pmix@6.1.0~munge~python build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  wwm2ix6	      ^prrte@4.1.0 build_system=autotools patches:=64faa1a,91b28f5 schedulers:=none platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  4rup75b		  ^flex@2.6.3+lex~nls build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0

With unify: false, Spack concretizes each root spec independently and merges the results, which is exactly what a software stack needs.

Note

If the stack is expected to have only a few duplicate nodes, when_possible is a less aggressive alternative:

override:
  concretizer:
    unify: when_possible

With this option Spack tries to unify the group in an eager, multi-round process. The concretization at round n contains all the specs that could not be unified at round n-1, and considers all the specs from previous rounds for reuse.

Spec matrices

Let’s further expand our stack to link against different LAPACK providers as well. Adding them explicitly would be tedious, so we use a matrix to express the cross-product compactly:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  specs:
  - group: compiler
    specs:
    - gcc@16
    - lmod@8.7.67
    override:
      packages:
        c:
          prefer: [gcc@15]
        cxx:
          prefer: [gcc@15]
        fortran:
          prefer: [gcc@15]
  - group: stack
    needs: [compiler]
    specs:
    - matrix:
      - [netlib-scalapack]
      - [^openmpi, ^mpich]
      - [^openblas, ^netlib-lapack]
    override:
      packages:
        c:
          prefer: [gcc@16]
        cxx:
          prefer: [gcc@16]
        fortran:
          prefer: [gcc@16]
      concretizer:
        unify: false
  view: false

A matrix expands to the cross-product of its rows. This matrix:

- matrix:
  - [netlib-scalapack]
  - [^openmpi, ^mpich]
  - [^openblas, ^netlib-lapack]

is equivalent to this explicit list of specs:

- "netlib-scalapack ^openblas ^openmpi"
- "netlib-scalapack ^openblas ^mpich"
- "netlib-scalapack ^netlib-lapack ^openmpi"
- "netlib-scalapack ^netlib-lapack ^mpich"

No compiler annotation appears in the matrix rows: the override block in the stack group handles that. Let’s concretize to verify our four variants:

$ spack concretize
==> Concretizing the 'stack' group of specs
==> Starting concretization pool with 2 processes
==> Concretized 2 specs:
[b]  ugsrem2  netlib-scalapack@2.2.3~ipo~pic+shared build_system=cmake build_type=Release generator=make platform=linux os=ubuntu26.04 target=x86_64_v3 %c,fortran=gcc@16.1.0
[b]  kw6i4f4	  ^cmake@3.31.11~doc+ncurses+ownlibs~qtgui build_system=generic build_type=Release platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  i3sc4sa	      ^curl@8.20.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2+nghttp2 build_system=autotools libs:=shared,static tls:=openssl platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  qjecybs		  ^nghttp2@1.67.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  rzn5joe		      ^diffutils@3.12 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  mladgc5		  ^openssl@3.6.1~docs+shared build_system=generic certs=mozilla platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  axeuqg3		      ^ca-certificates-mozilla@2026-03-19 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  romubi2	      ^ncurses@6.6~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  gc236kh	      ^zlib-ng@2.3.3+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[+]  mmywg7x	  ^compiler-wrapper@1.1.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[+]  hkkcdmj	  ^gcc@16.1.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs+profiled+strip build_system=autotools build_type=Release languages:='c,c++,fortran' platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  6ene6zr	      ^binutils@2.46.1~debuginfod+gas~gprofng~headers~interwork+ld~libiberty~lto~nls~pgo+plugins build_system=autotools compress_debug_sections=zlib libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  yvl6jpi		  ^pkgconf@2.5.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  ao2onuz	      ^diffutils@3.12 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  vbwvgwx		  ^libiconv@1.18 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  z7wqeps	      ^gawk@5.4.0~nls build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  k25xiih		  ^readline@8.3 build_system=autotools patches:=21f0a03,72dee13,e273643 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[e]  yjlog5x	      ^gcc@15.2.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs~profiled~strip build_system=autotools build_type=RelWithDebInfo languages:='c,c++,fortran' platform=linux os=ubuntu26.04 target=x86_64_v3
[+]  xm76mt3	      ^gcc-runtime@15.2.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  r4lhaok	      ^gmake@4.4.1~guile build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  xpmsy5x	      ^gmp@6.3.0+cxx build_system=autotools libs:=shared,static patches:=28a6673 platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  jo3eg4r		  ^autoconf@2.72 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  yhkgfai		  ^automake@1.18.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  y6uqrto		  ^m4@1.4.21+sigsegv build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  phcmfqk		      ^libsigsegv@2.15 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  3gdq456	      ^libtool@2.5.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  mguwetc		  ^file@5.46+static build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  hhjyyqy		      ^xz@5.8.3~pic build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  punbqrx		  ^findutils@4.10.0 build_system=autotools patches:=440b954 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  gxmoxea	      ^mpc@1.4.1 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  melg7ga	      ^mpfr@4.2.2 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  sf5iutv		  ^autoconf-archive@2024.10.16 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[+]  bvphs3b	      ^perl@5.42.0+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  27d4iyp		  ^berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  cbtgjrh		  ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  cq4dkuc		  ^gdbm@1.26 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  ohmdb2l		  ^less@692 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  jvlaabk	      ^texinfo@7.2~xs build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  sle3ix4		  ^gettext@1.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  ujlg2ua		      ^libxml2@2.15.3+pic~python+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  f5xe4px		      ^tar@1.35 build_system=autotools zip=pigz platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  afklka7			  ^pigz@2.8 build_system=makefile platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  ekvivpv		  ^ncurses@6.6~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  g72d7i3	      ^zlib-ng@2.3.3+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  63aruxk	      ^zstd@1.5.7+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  nn3xbub	  ^gcc-runtime@16.1.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[e]  yc4n2pp	  ^glibc@2.43 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  lwhtcck	  ^gmake@4.4.1~guile build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  fxetsgw	  ^mpich@5.0.1~argobots~cuda+fortran+hwloc+hydra~level_zero+libxml2+pci~rocm+romio~slurm~vci~verbs+wrapperrpath~xpmem build_system=autotools datatype-engine=auto device=ch4 netmod=ofi pmi=default platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx,fortran=gcc@16.1.0
[b]  fep3wol	      ^findutils@4.10.0 build_system=autotools patches:=440b954 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  qdxpkom		  ^gettext@1.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  u62ty4d		      ^tar@1.35 build_system=autotools zip=pigz platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  7ppidie			  ^pigz@2.8 build_system=makefile platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  fkki5gm	      ^hwloc@2.13.0~cairo~cuda~gl~level_zero~libudev+libxml2~nvml~opencl+pci~rocm build_system=autotools libs:=shared,static patches:=b4db98b platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  o3dcvlb	      ^libfabric@2.5.1~asan~cuda~debug~kdreg~level_zero~lsan~rocm~tsan~ubsan~uring build_system=autotools fabrics:=sockets,tcp,udp platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  t5frspj	      ^libpciaccess@0.17 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  nc2zhu5		  ^util-macros@1.20.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  3rlsf7j	      ^libxml2@2.15.3+pic~python+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  xrsocgc		  ^libiconv@1.18 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  rrvhely		  ^xz@5.8.3~pic build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  rws4suo	      ^pkgconf@2.5.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  vuoqpmo	      ^python@3.14.5+bz2+ctypes+dbm~debug~freethreading+libxml2+lzma~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~static~tests~tkinter+uuid+zlib+zstd build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  5kv66tq		  ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  kqm2d3g		  ^expat@2.8.1+libbsd build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  2zdnupj		      ^libbsd@0.12.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  ypuui7v			  ^libmd@1.1.0 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  2qw4v66		  ^gdbm@1.26 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  pkl6qup		  ^libffi@3.5.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  teh56fo		  ^readline@8.3 build_system=autotools patches:=21f0a03,72dee13,e273643 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  qy73hym		  ^sqlite@3.53.1+column_metadata+fts+rtree build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  2wcnb6n		  ^util-linux-uuid@2.41 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  mzuiikv		  ^zstd@1.5.7+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  77mpzq5	  ^netlib-lapack@3.12.1~external-blas~ipo+lapacke+pic+shared~xblas build_system=cmake build_type=Release generator=make patches:=3059ebf,b1af8b6,e318340 platform=linux os=ubuntu26.04 target=x86_64_v3 %c,fortran=gcc@16.1.0
[b]  6twt5yp  netlib-scalapack@2.2.3~ipo~pic+shared build_system=cmake build_type=Release generator=make platform=linux os=ubuntu26.04 target=x86_64_v3 %c,fortran=gcc@16.1.0
[b]  s6hcsd5	  ^openmpi@5.0.10+atomics~cuda~debug+fortran~gpfs~internal-hwloc~internal-libevent~internal-pmix~ipv6~java~lustre~memchecker~openshmem~rocm~romio+rsh~static~two_level_namespace+vt+wrapper-rpath build_system=autotools fabrics:=none patches:=646eb1a romio-filesystem:=none schedulers:=none platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx,fortran=gcc@16.1.0
[b]  uesc5yd	      ^autoconf@2.72 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  hkvksc3		  ^m4@1.4.21+sigsegv build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  4v4bncv		      ^libsigsegv@2.15 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  qhb7ywb	      ^automake@1.18.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  ul3skja	      ^libevent@2.1.12+openssl build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  wiglecf	      ^libtool@2.5.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  ra2jarr		  ^file@5.46+static build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  vn7wvoe	      ^numactl@2.0.19 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  plgdjpa	      ^openssh@10.3p1+gssapi build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  7a4l5v5		  ^krb5@1.22.2+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  qu5uhax		      ^bison@3.8.2~color build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  4binz2p		  ^libedit@3.1-20251016 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  aca7p7l		  ^libxcrypt@4.5.2~obsolete_api build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  c6ebuxy	      ^perl@5.42.0+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  zsx5k5v		  ^berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  d2d5je7		  ^less@692 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  2vgljh5	      ^pmix@6.1.0~munge~python build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  wwm2ix6	      ^prrte@4.1.0 build_system=autotools patches:=64faa1a,91b28f5 schedulers:=none platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  4rup75b		  ^flex@2.6.3+lex~nls build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0

We now have four variations of netlib-scalapack ready to install.

Reusable definitions

So far each matrix row has been written out inline. When multiple matrices share rows, duplicating them becomes a maintenance burden. Spack allows defining lists of constraints under the definitions attribute and reusing them across matrices. Let’s rewrite our manifest using definitions:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  definitions:
  - mpis: [openmpi, mpich]
  - lapacks: [openblas, netlib-lapack]
  - mpi_packages: [netlib-scalapack]

  specs:
  - group: compiler
    specs:
    - gcc@16
    - lmod@8.7.67
    override:
      packages:
        c:
          prefer: [gcc@15]
        cxx:
          prefer: [gcc@15]
        fortran:
          prefer: [gcc@15]
  - group: stack
    needs: [compiler]
    specs:
    - matrix:
      - [$mpi_packages]
      - [$^mpis]
      - [$^lapacks]
    override:
      packages:
        c:
          prefer: [gcc@16]
        cxx:
          prefer: [gcc@16]
        fortran:
          prefer: [gcc@16]
      concretizer:
        unify: false
  view: false

Concretizing confirms the result is identical:

$ spack concretize
==> No new specs to concretize.

Now that we have definitions, we can add a second matrix for serial packages that link against LAPACK but not MPI. In a real deployment this list might include many packages, but here we use just py-scipy as an example. Not every combination is meaningful though: py-scipy ^netlib-lapack is not a useful build, so we use exclude to drop that specific entry from the cross-product:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  definitions:
  - mpis: [openmpi, mpich]
  - lapacks: [openblas, netlib-lapack]
  - mpi_packages: [netlib-scalapack]
  - serial_packages: [py-scipy]

  specs:
  - group: compiler
    specs:
    - gcc@16
    - lmod@8.7.67
    override:
      packages:
        c:
          prefer: [gcc@15]
        cxx:
          prefer: [gcc@15]
        fortran:
          prefer: [gcc@15]
  - group: stack
    needs: [compiler]
    specs:
    - matrix:
      - [$mpi_packages]
      - [$^mpis]
      - [$^lapacks]
    - matrix:
      - [$serial_packages]
      - [$^lapacks]
      exclude:
      - "py-scipy ^netlib-lapack"
    override:
      packages:
        c:
          prefer: [gcc@16]
        cxx:
          prefer: [gcc@16]
        fortran:
          prefer: [gcc@16]
      concretizer:
        unify: false
  view: false

Concretize the environment again:

$ spack concretize
==> Concretizing the 'stack' group of specs
==> Starting concretization
==> Concretized 1 spec:
[b]  s2p5dpl  py-scipy@1.17.1 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx,fortran=gcc@16.1.0
[+]  mmywg7x	  ^compiler-wrapper@1.1.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[+]  hkkcdmj	  ^gcc@16.1.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs+profiled+strip build_system=autotools build_type=Release languages:='c,c++,fortran' platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  6ene6zr	      ^binutils@2.46.1~debuginfod+gas~gprofng~headers~interwork+ld~libiberty~lto~nls~pgo+plugins build_system=autotools compress_debug_sections=zlib libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  yvl6jpi		  ^pkgconf@2.5.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  ao2onuz	      ^diffutils@3.12 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  vbwvgwx		  ^libiconv@1.18 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  z7wqeps	      ^gawk@5.4.0~nls build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  k25xiih		  ^readline@8.3 build_system=autotools patches:=21f0a03,72dee13,e273643 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[e]  yjlog5x	      ^gcc@15.2.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs~profiled~strip build_system=autotools build_type=RelWithDebInfo languages:='c,c++,fortran' platform=linux os=ubuntu26.04 target=x86_64_v3
[+]  xm76mt3	      ^gcc-runtime@15.2.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  r4lhaok	      ^gmake@4.4.1~guile build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  xpmsy5x	      ^gmp@6.3.0+cxx build_system=autotools libs:=shared,static patches:=28a6673 platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  jo3eg4r		  ^autoconf@2.72 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  yhkgfai		  ^automake@1.18.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  y6uqrto		  ^m4@1.4.21+sigsegv build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  phcmfqk		      ^libsigsegv@2.15 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  3gdq456	      ^libtool@2.5.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  mguwetc		  ^file@5.46+static build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  hhjyyqy		      ^xz@5.8.3~pic build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  punbqrx		  ^findutils@4.10.0 build_system=autotools patches:=440b954 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  gxmoxea	      ^mpc@1.4.1 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  melg7ga	      ^mpfr@4.2.2 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  sf5iutv		  ^autoconf-archive@2024.10.16 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[+]  bvphs3b	      ^perl@5.42.0+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  27d4iyp		  ^berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  cbtgjrh		  ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  cq4dkuc		  ^gdbm@1.26 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  ohmdb2l		  ^less@692 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  jvlaabk	      ^texinfo@7.2~xs build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  sle3ix4		  ^gettext@1.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  ujlg2ua		      ^libxml2@2.15.3+pic~python+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  f5xe4px		      ^tar@1.35 build_system=autotools zip=pigz platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  afklka7			  ^pigz@2.8 build_system=makefile platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  ekvivpv		  ^ncurses@6.6~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  g72d7i3	      ^zlib-ng@2.3.3+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  63aruxk	      ^zstd@1.5.7+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  nn3xbub	  ^gcc-runtime@16.1.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[e]  yc4n2pp	  ^glibc@2.43 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  wg3oazy	  ^meson@1.11.1 build_system=python_pip patches:=0f0b1bd platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  6zizggt	      ^ninja@1.13.2+re2c build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  cowskr3		  ^re2c@4.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  hkjqn4j	      ^py-setuptools@82.0.1 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  uh2kn3c	  ^openblas@0.3.33~bignuma~consistent_fpcsr+dynamic_dispatch+fortran~ilp64+locking+pic+shared~static build_system=makefile patches:=723ddc1 symbol_suffix=none threads=none platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx,fortran=gcc@16.1.0
[b]  lwhtcck	      ^gmake@4.4.1~guile build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  rws4suo	  ^pkgconf@2.5.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  hvfq5pe	  ^py-cython@3.2.4 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  4udrsr2	  ^py-meson-python@0.19.0 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  hqlnorc	      ^py-packaging@26.2 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  kmkn7kg		  ^py-flit-core@3.12.0 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  ibs4tad	      ^py-pyproject-metadata@0.11.0 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  gvvfhjh	  ^py-numpy@2.4.6 build_system=python_pip patches:=873745d platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  kdvgwt6	  ^py-pip@26.1.2 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  jv2364q	  ^py-pybind11@3.0.2+ipo build_system=cmake build_type=Release generator=ninja platform=linux os=ubuntu26.04 target=x86_64_v3 %cxx=gcc@16.1.0
[b]  kw6i4f4	      ^cmake@3.31.11~doc+ncurses+ownlibs~qtgui build_system=generic build_type=Release platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  i3sc4sa		  ^curl@8.20.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2+nghttp2 build_system=autotools libs:=shared,static tls:=openssl platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  qjecybs		      ^nghttp2@1.67.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  mwv6jzs	      ^py-scikit-build-core@0.12.2~pyproject build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx,fortran=gcc@16.1.0
[b]  fvkaky5		  ^py-hatch-vcs@0.5.0 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  jphx547		      ^py-setuptools-scm@9.2.2+toml build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  bpfangz			  ^git@2.53.0+man+nls+perl+subtree~tcltk build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  uesc5yd			      ^autoconf@2.72 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  qhb7ywb			      ^automake@1.18.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  6ajjy27			      ^libidn2@2.3.8 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  ob7hefb				  ^libunistring@1.4.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  wiglecf			      ^libtool@2.5.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  ra2jarr				  ^file@5.46+static build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  fep3wol				  ^findutils@4.10.0 build_system=autotools patches:=440b954 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  hkvksc3			      ^m4@1.4.21+sigsegv build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  4v4bncv				  ^libsigsegv@2.15 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  plgdjpa			      ^openssh@10.3p1+gssapi build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  7a4l5v5				  ^krb5@1.22.2+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  qu5uhax				      ^bison@3.8.2~color build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  4binz2p				  ^libedit@3.1-20251016 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  aca7p7l				  ^libxcrypt@4.5.2~obsolete_api build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  pjhxp26			      ^pcre2@10.44~jit+multibyte+pic build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  hqcsjrm		  ^py-hatchling@1.29.0 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  kyxtnxs		      ^py-pluggy@1.6.0 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  3spx2cq		      ^py-trove-classifiers@2026.6.1.19 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  3urylsu			  ^py-calver@2025.10.20 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  aydqe7l		  ^py-pathspec@1.1.1 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  l6vbm7d	  ^py-pythran@0.18.1 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3 %cxx=gcc@16.1.0
[b]  scwmymw	      ^py-beniget@0.4.2.post1 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  rzdh2tt	      ^py-gast@0.6.0 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  4auh5k2	      ^py-ply@3.11 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  j6saqla	  ^py-wheel@0.45.1 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  vuoqpmo	  ^python@3.14.5+bz2+ctypes+dbm~debug~freethreading+libxml2+lzma~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~static~tests~tkinter+uuid+zlib+zstd build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  5kv66tq	      ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  rzn5joe		  ^diffutils@3.12 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  kqm2d3g	      ^expat@2.8.1+libbsd build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  2zdnupj		  ^libbsd@0.12.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  ypuui7v		      ^libmd@1.1.0 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  2qw4v66	      ^gdbm@1.26 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  qdxpkom	      ^gettext@1.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  xrsocgc		  ^libiconv@1.18 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  3rlsf7j		  ^libxml2@2.15.3+pic~python+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  u62ty4d		  ^tar@1.35 build_system=autotools zip=pigz platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  7ppidie		      ^pigz@2.8 build_system=makefile platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  pkl6qup	      ^libffi@3.5.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  romubi2	      ^ncurses@6.6~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  mladgc5	      ^openssl@3.6.1~docs+shared build_system=generic certs=mozilla platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  axeuqg3		  ^ca-certificates-mozilla@2026-03-19 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  c6ebuxy		  ^perl@5.42.0+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  zsx5k5v		      ^berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  d2d5je7		      ^less@692 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  teh56fo	      ^readline@8.3 build_system=autotools patches:=21f0a03,72dee13,e273643 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  qy73hym	      ^sqlite@3.53.1+column_metadata+fts+rtree build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  2wcnb6n	      ^util-linux-uuid@2.41 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  rrvhely	      ^xz@5.8.3~pic build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  gc236kh	      ^zlib-ng@2.3.3+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  mzuiikv	      ^zstd@1.5.7+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  iyp4doo	  ^python-venv@1.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3

The exclusion worked: the environment contains only py-scipy ^openblas.

Conditional definitions

A common need in multi-site or multi-cluster deployments is enabling different software sets based on environment variables or host properties. Definitions support a when clause for exactly this. when is a Python expression evaluated at concretization time, with access to the following variables:

variable name

value

platform

The spack platform name for this machine

os

The default spack os name and version string for this machine

target

The default spack target string for this machine

architecture

The default spack architecture string platform-os-target for this machine

arch

Alias for architecture

env

A dictionary representing the user’s environment variables

re

The python re module for regex

hostname

The hostname of this node

Suppose we want to build with mpich by default, and add openmpi when a site-specific environment variable is set. We can express this with two definitions of mpis, the second guarded by a when clause:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  definitions:
  - mpis: [mpich]
  - mpis: [openmpi]
    when: 'env.get("SPACK_STACK_USE_OPENMPI", "") == "1"'
  - lapacks: [openblas, netlib-lapack]
  - mpi_packages: [netlib-scalapack]
  - serial_packages: [py-scipy]

  specs:
  - group: compiler
    specs:
    - gcc@16
    - lmod@8.7.67
    override:
      packages:
        c:
          prefer: [gcc@15]
        cxx:
          prefer: [gcc@15]
        fortran:
          prefer: [gcc@15]
  - group: stack
    needs: [compiler]
    specs:
    - matrix:
      - [$mpi_packages]
      - [$^mpis]
      - [$^lapacks]
    - matrix:
      - [$serial_packages]
      - [$^lapacks]
      exclude:
      - "py-scipy ^netlib-lapack"
    override:
      packages:
        c:
          prefer: [gcc@16]
        cxx:
          prefer: [gcc@16]
        fortran:
          prefer: [gcc@16]
      concretizer:
        unify: false
  view: false

When multiple definitions share the same name, they are concatenated, so the conditional entry simply appends openmpi to the list when the variable is set.

First, concretize without the variable:

$ spack concretize
==> No new specs to concretize.
$ spack find -cl netlib-scalapack
==> In environment /home/spack/stacks (5 root specs)
-- root specs / compiler ----------------------------------------
[+] hkkcdmj gcc@16  [+] wf22jsg lmod@8.7.67

-- root specs / stack -------------------------------------------
[b] ugsrem2 netlib-scalapack  [b] y2obsoc netlib-scalapack  [b] s2p5dpl py-scipy

-- linux-ubuntu26.04-x86_64_v3 / %c,fortran=gcc@16.1.0 ----------
[b]  ugsrem2 netlib-scalapack@2.2.3  [b]  y2obsoc netlib-scalapack@2.2.3
==> 0 installed packages
==> 2 concretized packages to be installed

Only mpich is used. Now set the variable and re-concretize:

$ export SPACK_STACK_USE_OPENMPI=1
$ spack concretize
$ spack find -cl netlib-scalapack
==> In environment /home/spack/stacks (7 root specs)
-- root specs / compiler ----------------------------------------
[+] hkkcdmj gcc@16  [+] wf22jsg lmod@8.7.67

-- root specs / stack -------------------------------------------
[b] ugsrem2 netlib-scalapack  [b] 6twt5yp netlib-scalapack  [b] s2p5dpl py-scipy
[b] y2obsoc netlib-scalapack  [b] faqiqu4 netlib-scalapack

-- linux-ubuntu26.04-x86_64_v3 / %c,fortran=gcc@16.1.0 ----------
[b]  ugsrem2 netlib-scalapack@2.2.3  [b]  6twt5yp netlib-scalapack@2.2.3
[b]  y2obsoc netlib-scalapack@2.2.3  [b]  faqiqu4 netlib-scalapack@2.2.3
==> 0 installed packages
==> 4 concretized packages to be installed

Both MPI providers are now active.

We now have a complete, well-structured environment. It’s time to install it:

$ spack install
[ ] nn3xbub gcc-runtime@16.1.0 fetching from build cache (0s)
[ ] nn3xbub gcc-runtime@16.1.0 relocating (0s)
[+] nn3xbub gcc-runtime@16.1.0 /home/spack/spack/opt/spack/linux-x86_64_v3/gcc-runtime-16.1.0-nn3xbub7zhmkubwxasksced5zbbb7xwr (1s)
[ ] romubi2 ncurses@6.6 fetching from build cache (0s)
[ ] aca7p7l libxcrypt@4.5.2 fetching from build cache (0s)
[ ] uh2kn3c openblas@0.3.33 fetching from build cache (0s)
[ ] xrsocgc libiconv@1.18 fetching from build cache (0s)
[ ] romubi2 ncurses@6.6 relocating (0s)
[ ] aca7p7l libxcrypt@4.5.2 relocating (0s)
[ ] xrsocgc libiconv@1.18 relocating (0s)
[ ] uh2kn3c openblas@0.3.33 relocating (0s)
[+] aca7p7l libxcrypt@4.5.2 /home/spack/spack/opt/spack/linux-x86_64_v3/libxcrypt-4.5.2-aca7p7lojtt3defjbdogqmktovdxa5qx (0s)
[ ] t5frspj libpciaccess@0.17 fetching from build cache (0s)
[+] xrsocgc libiconv@1.18 /home/spack/spack/opt/spack/linux-x86_64_v3/libiconv-1.18-xrsocgcig7f5o4fujtkqfiidnnkfnf5g (0s)
[ ] 77mpzq5 netlib-lapack@3.12.1 fetching from build cache (0s)
[ ] t5frspj libpciaccess@0.17 relocating (1s)
[+] uh2kn3c openblas@0.3.33 /home/spack/spack/opt/spack/linux-x86_64_v3/openblas-0.3.33-uh2kn3ctzgxuixv23pw3vqa5nk2xfxll (1s)
[ ] 77mpzq5 netlib-lapack@3.12.1 relocating (1s)
[ ] o3dcvlb libfabric@2.5.1 fetching from build cache (0s)
[+] t5frspj libpciaccess@0.17 /home/spack/spack/opt/spack/linux-x86_64_v3/libpciaccess-0.17-t5frspjun7nqk5pp5mjictlokfcwmmpc (1s)
[ ] 2wcnb6n util-linux-uuid@2.41 fetching from build cache (0s)
[ ] o3dcvlb libfabric@2.5.1 relocating (0s)
[+] 77mpzq5 netlib-lapack@3.12.1 /home/spack/spack/opt/spack/linux-x86_64_v3/netlib-lapack-3.12.1-77mpzq52bkdgbldogwa5snquulfrw7vs (1s)
[ ] 2wcnb6n util-linux-uuid@2.41 relocating (0s)
[ ] pkl6qup libffi@3.5.2 fetching from build cache (0s)
[+] o3dcvlb libfabric@2.5.1 /home/spack/spack/opt/spack/linux-x86_64_v3/libfabric-2.5.1-o3dcvlbnlyjp4ffwrsnuq4h2626gm5sm (1s)
[ ] mzuiikv zstd@1.5.7 fetching from build cache (0s)
[ ] pkl6qup libffi@3.5.2 relocating (1s)
[+] 2wcnb6n util-linux-uuid@2.41 /home/spack/spack/opt/spack/linux-x86_64_v3/util-linux-uuid-2.41-2wcnb6nky3pis26yp54z2yztpi2ggdx4 (1s)
[ ] 5kv66tq bzip2@1.0.8 fetching from build cache (0s)
[+] romubi2 ncurses@6.6 /home/spack/spack/opt/spack/linux-x86_64_v3/ncurses-6.6-romubi25ey4yh64hcscvlcdnwb7lxni3 (2s)
[+] pkl6qup libffi@3.5.2 /home/spack/spack/opt/spack/linux-x86_64_v3/libffi-3.5.2-pkl6qupanohdjgbi2yisoob4gqpdczww (1s)
[ ] mzuiikv zstd@1.5.7 relocating (0s)
[ ] ypuui7v libmd@1.1.0 fetching from build cache (0s)
[ ] rrvhely xz@5.8.3 fetching from build cache (0s)
[ ] 5kv66tq bzip2@1.0.8 relocating (0s)
[+] mzuiikv zstd@1.5.7 /home/spack/spack/opt/spack/linux-x86_64_v3/zstd-1.5.7-mzuiikvvruxmy5bmi63iwbpziaqyaxsp (0s)
[ ] ypuui7v libmd@1.1.0 relocating (0s)
[ ] gc236kh zlib-ng@2.3.3 fetching from build cache (0s)
[ ] rrvhely xz@5.8.3 relocating (0s)
[+] 5kv66tq bzip2@1.0.8 /home/spack/spack/opt/spack/linux-x86_64_v3/bzip2-1.0.8-5kv66tqcv6pqaezk75uxidkelfa2hjx2 (0s)
[ ] vn7wvoe numactl@2.0.19 fetching from build cache (0s)
[+] ypuui7v libmd@1.1.0 /home/spack/spack/opt/spack/linux-x86_64_v3/libmd-1.1.0-ypuui7vsqibbdjx25nya7qsxvjyycve3 (0s)
[ ] gc236kh zlib-ng@2.3.3 relocating (1s)
[ ] 4binz2p libedit@3.1-20251016 fetching from build cache (0s)
[ ] vn7wvoe numactl@2.0.19 relocating (1s)
[+] rrvhely xz@5.8.3 /home/spack/spack/opt/spack/linux-x86_64_v3/xz-5.8.3-rrvhelyvarwsqfyxopmtd5gqt5id4fed (1s)
[+] gc236kh zlib-ng@2.3.3 /home/spack/spack/opt/spack/linux-x86_64_v3/zlib-ng-2.3.3-gc236khsrbih4o6nnaeqea7yvhm4d5e7 (1s)
[ ] teh56fo readline@8.3 fetching from build cache (0s)
[ ] 4binz2p libedit@3.1-20251016 relocating (0s)
[ ] 2zdnupj libbsd@0.12.2 fetching from build cache (0s)
[+] vn7wvoe numactl@2.0.19 /home/spack/spack/opt/spack/linux-x86_64_v3/numactl-2.0.19-vn7wvoehm2bv3im5gxkhb7twi5l3uhmu (1s)
[ ] mladgc5 openssl@3.6.1 fetching from build cache (0s)
[ ] teh56fo readline@8.3 relocating (0s)
[+] 4binz2p libedit@3.1-20251016 /home/spack/spack/opt/spack/linux-x86_64_v3/libedit-3.1-20251016-4binz2pyummtqw6nrr2m3nhygqkf2kjr (0s)
[ ] 2zdnupj libbsd@0.12.2 relocating (0s)
[ ] 7ppidie pigz@2.8 fetching from build cache (0s)
[+] teh56fo readline@8.3 /home/spack/spack/opt/spack/linux-x86_64_v3/readline-8.3-teh56foom4eqf4r22risawici3kacfxn (0s)
[ ] mladgc5 openssl@3.6.1 relocating (0s)
[ ] 3rlsf7j libxml2@2.15.3 fetching from build cache (0s)
[ ] 7ppidie pigz@2.8 relocating (1s)
[+] 2zdnupj libbsd@0.12.2 /home/spack/spack/opt/spack/linux-x86_64_v3/libbsd-0.12.2-2zdnupjtphl5alwmkywobr55gqctkjgy (1s)
[ ] 2qw4v66 gdbm@1.26 fetching from build cache (0s)
[+] 7ppidie pigz@2.8 /home/spack/spack/opt/spack/linux-x86_64_v3/pigz-2.8-7ppidiedssazkk5snujwlaatkjzl4xlz (1s)
[ ] 3rlsf7j libxml2@2.15.3 relocating (1s)
[ ] qy73hym sqlite@3.53.1 fetching from build cache (0s)
[+] mladgc5 openssl@3.6.1 /home/spack/spack/opt/spack/linux-x86_64_v3/openssl-3.6.1-mladgc5eepc2mvvfnzkjarftwmzdae5j (1s)
[ ] 2qw4v66 gdbm@1.26 relocating (0s)
[ ] kqm2d3g expat@2.8.1 fetching from build cache (0s)
[+] 3rlsf7j libxml2@2.15.3 /home/spack/spack/opt/spack/linux-x86_64_v3/libxml2-2.15.3-3rlsf7j7jgr5drqaqujr5uw3qrlguy44 (1s)
[ ] qy73hym sqlite@3.53.1 relocating (0s)
[ ] u62ty4d tar@1.35 fetching from build cache (0s)
[+] 2qw4v66 gdbm@1.26 /home/spack/spack/opt/spack/linux-x86_64_v3/gdbm-1.26-2qw4v6642brrlwkqgbnqdwbovof66nfk (0s)
[ ] kqm2d3g expat@2.8.1 relocating (0s)
[ ] ul3skja libevent@2.1.12 fetching from build cache (0s)
[ ] u62ty4d tar@1.35 relocating (0s)
[+] kqm2d3g expat@2.8.1 /home/spack/spack/opt/spack/linux-x86_64_v3/expat-2.8.1-kqm2d3gvvu4xj42amenolxqhoni66bpo (0s)
[+] qy73hym sqlite@3.53.1 /home/spack/spack/opt/spack/linux-x86_64_v3/sqlite-3.53.1-qy73hymb6hze5xx4azkaytbcl7wykouv (1s)
[ ] fkki5gm hwloc@2.13.0 fetching from build cache (1s)
[ ] ul3skja libevent@2.1.12 relocating (1s)
[+] u62ty4d tar@1.35 /home/spack/spack/opt/spack/linux-x86_64_v3/tar-1.35-u62ty4dsid2jy3ys3wbwoy37nax4y5i7 (1s)
[ ] fkki5gm hwloc@2.13.0 relocating (1s)
[ ] qdxpkom gettext@1.0 fetching from build cache (0s)
[+] ul3skja libevent@2.1.12 /home/spack/spack/opt/spack/linux-x86_64_v3/libevent-2.1.12-ul3skjatebbqwbcuq7rnwdesomenh56w (1s)
[ ] qdxpkom gettext@1.0 relocating (0s)
[+] fkki5gm hwloc@2.13.0 /home/spack/spack/opt/spack/linux-x86_64_v3/hwloc-2.13.0-fkki5gmowqxg3bskipcnmndcgxwiwzuw (1s)
[ ] 2vgljh5 pmix@6.1.0 fetching from build cache (0s)
[ ] fxetsgw mpich@5.0.1 fetching from build cache (0s)
[ ] 2vgljh5 pmix@6.1.0 relocating (0s)
[ ] fxetsgw mpich@5.0.1 relocating (1s)
[+] 2vgljh5 pmix@6.1.0 /home/spack/spack/opt/spack/linux-x86_64_v3/pmix-6.1.0-2vgljh5yyopajc52dgj2bdob5cqszu7y (1s)
[ ] wwm2ix6 prrte@4.1.0 fetching from build cache (0s)
[ ] wwm2ix6 prrte@4.1.0 relocating (0s)
[+] wwm2ix6 prrte@4.1.0 /home/spack/spack/opt/spack/linux-x86_64_v3/prrte-4.1.0-wwm2ix6ijbqqfjaeslnngjq5zyeufgz7 (0s)
[+] qdxpkom gettext@1.0 /home/spack/spack/opt/spack/linux-x86_64_v3/gettext-1.0-qdxpkom3wtc55cnrabhk2mq34kifliiv (2s)
[ ] 7a4l5v5 krb5@1.22.2 fetching from build cache (0s)
[ ] vuoqpmo python@3.14.5 fetching from build cache (0s)
[ ] 7a4l5v5 krb5@1.22.2 relocating (0s)
[ ] vuoqpmo python@3.14.5 relocating (0s)
[+] 7a4l5v5 krb5@1.22.2 /home/spack/spack/opt/spack/linux-x86_64_v3/krb5-1.22.2-7a4l5v5natxi3lmapzc4ea4nzvd5k5ms (0s)
[ ] plgdjpa openssh@10.3p1 fetching from build cache (0s)
[ ] plgdjpa openssh@10.3p1 relocating (1s)
[+] fxetsgw mpich@5.0.1 /home/spack/spack/opt/spack/linux-x86_64_v3/mpich-5.0.1-fxetsgw62gcxvnisz6dwz3um5hjtjtpf (3s)
[ ] ugsrem2 netlib-scalapack@2.2.3 fetching from build cache (0s)
[ ] y2obsoc netlib-scalapack@2.2.3 fetching from build cache (0s)
[+] plgdjpa openssh@10.3p1 /home/spack/spack/opt/spack/linux-x86_64_v3/openssh-10.3p1-plgdjpahipc4dsfxec7kr3eyu4axw4gc (1s)
[ ] s6hcsd5 openmpi@5.0.10 fetching from build cache (0s)
[ ] ugsrem2 netlib-scalapack@2.2.3 relocating (0s)
[ ] y2obsoc netlib-scalapack@2.2.3 relocating (0s)
[ ] s6hcsd5 openmpi@5.0.10 relocating (0s)
[+] ugsrem2 netlib-scalapack@2.2.3 /home/spack/spack/opt/spack/linux-x86_64_v3/netlib-scalapack-2.2.3-ugsrem2k5ctcsem62rh5qj5ji6hpoif4 (1s)
[+] y2obsoc netlib-scalapack@2.2.3 /home/spack/spack/opt/spack/linux-x86_64_v3/netlib-scalapack-2.2.3-y2obsoclpudkxt2mbrc66elctp5tvicy (1s)
[+] vuoqpmo python@3.14.5 /home/spack/spack/opt/spack/linux-x86_64_v3/python-3.14.5-vuoqpmohlxduiwga4ni3jfsf6lbd562b (2s)
[ ] iyp4doo python-venv@1.0 fetching from build cache (0s)
[ ] iyp4doo python-venv@1.0 relocating (1s)
[+] s6hcsd5 openmpi@5.0.10 /home/spack/spack/opt/spack/linux-x86_64_v3/openmpi-5.0.10-s6hcsd5arflozzu5ho32ew2ohqlglpyf (2s)
[+] iyp4doo python-venv@1.0 /home/spack/spack/opt/spack/linux-x86_64_v3/python-venv-1.0-iyp4dooj6gwksm2k25zavxbz2o7cvekx (1s)
[ ] faqiqu4 netlib-scalapack@2.2.3 fetching from build cache (0s)
[ ] 6twt5yp netlib-scalapack@2.2.3 fetching from build cache (0s)
[ ] gvvfhjh py-numpy@2.4.6 fetching from build cache (0s)
[ ] jv2364q py-pybind11@3.0.2 fetching from build cache (0s)
[ ] faqiqu4 netlib-scalapack@2.2.3 relocating (0s)
[ ] 6twt5yp netlib-scalapack@2.2.3 relocating (0s)
[ ] gvvfhjh py-numpy@2.4.6 relocating (0s)
[ ] jv2364q py-pybind11@3.0.2 relocating (0s)
[+] faqiqu4 netlib-scalapack@2.2.3 /home/spack/spack/opt/spack/linux-x86_64_v3/netlib-scalapack-2.2.3-faqiqu4wbwlhklx6q2mjqvrv2veclueb (0s)
[+] 6twt5yp netlib-scalapack@2.2.3 /home/spack/spack/opt/spack/linux-x86_64_v3/netlib-scalapack-2.2.3-6twt5ypw7pqo64et2pu5ol6bolggnkks (0s)
[+] jv2364q py-pybind11@3.0.2 /home/spack/spack/opt/spack/linux-x86_64_v3/py-pybind11-3.0.2-jv2364qenpa3evz5xt73u6b4epqgyqhr (0s)
[+] gvvfhjh py-numpy@2.4.6 /home/spack/spack/opt/spack/linux-x86_64_v3/py-numpy-2.4.6-gvvfhjhkhi7yrqjpq5jnjihpgmhbmxjt (1s)
[ ] s2p5dpl py-scipy@1.17.1 fetching from build cache (0s)
[ ] s2p5dpl py-scipy@1.17.1 relocating (0s)
[+] s2p5dpl py-scipy@1.17.1 /home/spack/spack/opt/spack/linux-x86_64_v3/py-scipy-1.17.1-s2p5dpl2srxtk4cvkdprpqogagdjrvzb (2s)

Other useful features

Before moving to views and modules, two other features are worth knowing about.

The first is source mirrors. When the environment is active, creating one is as simple as:

$ spack mirror create --all -d ./stacks-mirror

This fetches all the tarballs listed in spack.lock into the given directory. You can then move the mirror to an air-gapped machine and register it with:

$ spack mirror add <name> <stacks-mirror>

This lets you rebuild the specs from source on the target machine. Alternatively, to create a buildcache you can:

$ spack gpg create <name> <e-mail>
$ spack buildcache push ./mirror

Don’t forget to set an appropriate value for the padding of the install tree; see how to set up relocation in our documentation.

Multiple Filesystem Views

Views give users access to the installed software through standard filesystem paths, without requiring them to interact with Spack directly. A stack, however, contains multiple configurations of the same package, which would conflict in a single merged directory tree. Instead, we define multiple views, each covering a curated subset of the stack, with optional projections that control the directory layout.

Let’s add view descriptors to our spack.yaml:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  definitions:
  - mpis: [mpich]
  - mpis: [openmpi]
    when: 'env.get("SPACK_STACK_USE_OPENMPI", "") == "1"'
  - lapacks: [openblas, netlib-lapack]
  - mpi_packages: [netlib-scalapack]
  - serial_packages: [py-scipy]

  specs:
  - group: compiler
    specs:
    - gcc@16
    - lmod@8.7.67
    override:
      packages:
        c:
          prefer: [gcc@15]
        cxx:
          prefer: [gcc@15]
        fortran:
          prefer: [gcc@15]
  - group: stack
    needs: [compiler]
    specs:
    - matrix:
      - [$mpi_packages]
      - [$^mpis]
      - [$^lapacks]
    - matrix:
      - [$serial_packages]
      - [$^lapacks]
      exclude:
      - "py-scipy ^netlib-lapack"
    override:
      packages:
        c:
          prefer: [gcc@16]
        cxx:
          prefer: [gcc@16]
        fortran:
          prefer: [gcc@16]
      concretizer:
        unify: false
  view:
    default:
      root: views/default
      select: ['%gcc@16']
      exclude: [^mpich, ^netlib-lapack]
    full:
      root: views/full
      projections:
        ^mpi^lapack: '{name}/{name}-{version}-{^c.name}-{^c.version}-{^mpi.name}-{^mpi.version}-{^lapack.name}-{^lapack.version}'
        ^lapack: '{name}/{name}-{version}-{^c.name}-{^c.version}-{^lapack.name}-{^lapack.version}'
        '%c': '{name}/{name}-{version}-{^c.name}-{^c.version}'
        '%cxx': '{name}/{name}-{version}-{^cxx.name}-{^cxx.version}'
        '%fortran': '{name}/{name}-{version}-{^fortran.name}-{^fortran.version}'
        all: '{name}/{name}-{version}'

We defined two views, default and full. The default view consists of all the packages compiled with gcc@16, excluding those that depend on mpich or netlib-lapack. View descriptors accept both select and exclude constraints.

The full view uses a more complex projection to place each spec into an appropriate subdirectory, according to the first constraint that matches. all is the default projection and always has the lowest priority, regardless of the order in which it appears. To avoid confusion, we advise always keeping it last in projections.

Concretize to regenerate the views and check their structure:

$ spack concretize
==> No new specs to concretize.
==> Updating view at /home/spack/stacks/views/default
==> Updating view at /home/spack/stacks/views/full
$ ls ~/stacks/views/default
bin  etc  include  lib	lib64  libexec	man  pyvenv.cfg	 sbin  share  var
$ ls ~/stacks/views/default/lib
cmake				libmenu.so
gettext				libmenu.so.6
krb5				libmenu.so.6.6
libasan.so.8			libmenu_g.a
libasprintf.a			libmenuw.a
libasprintf.so			libmenuw.so
libasprintf.so.0		libmenuw.so.6
libasprintf.so.0.0.0		libmenuw.so.6.6
libatomic.so.1			libmenuw_g.a
libblas.so			libmpi.a
libblas.so.3			libmpi.so
libblas.so.3.12.0		libmpi.so.12
libbsd-ctor.a			libmpi.so.12.6.1
libbsd.a			libmpi.so.40
libbsd.so			libmpi.so.40.40.7
libbsd.so.0			libmpi_mpifh.so
libbsd.so.0.12.2		libmpi_mpifh.so.40
libbz2.a			libmpi_mpifh.so.40.40.1
libbz2.so			libmpi_usempi_ignore_tkr.so
libbz2.so.1			libmpi_usempi_ignore_tkr.so.40
libbz2.so.1.0			libmpi_usempi_ignore_tkr.so.40.40.1
libbz2.so.1.0.8			libmpi_usempif08.so
libcblas.so			libmpi_usempif08.so.40
libcblas.so.3			libmpi_usempif08.so.40.40.3
libcblas.so.3.12.0		libmpich.so
libcharset.a			libmpichcxx.so
libcharset.so			libmpichf90.so
libcharset.so.1			libmpicxx.a
libcharset.so.1.0.0		libmpicxx.so
libcom_err.so			libmpicxx.so.12
libcom_err.so.3			libmpicxx.so.12.6.1
libcom_err.so.3.0		libmpifort.a
libcrypt.a			libmpifort.so
libcrypt.so			libmpifort.so.12
libcrypt.so.2			libmpifort.so.12.6.1
libcrypt.so.2.0.0		libmpl.so
libcurses.so			libncurses++.a
libedit.a			libncurses++.so
libedit.so			libncurses++.so.6
libedit.so.0			libncurses++.so.6.6
libedit.so.0.0.76		libncurses++_g.a
libevent-2.1.so.7		libncurses++w.a
libevent-2.1.so.7.0.1		libncurses++w.so
libevent.a			libncurses++w.so.6
libevent.so			libncurses++w.so.6.6
libevent_core-2.1.so.7		libncurses++w_g.a
libevent_core-2.1.so.7.0.1	libncurses.a
libevent_core.a			libncurses.so
libevent_core.so		libncurses.so.6
libevent_extra-2.1.so.7		libncurses.so.6.6
libevent_extra-2.1.so.7.0.1	libncurses_g.a
libevent_extra.a		libncursesw.a
libevent_extra.so		libncursesw.so
libevent_openssl-2.1.so.7	libncursesw.so.6
libevent_openssl-2.1.so.7.0.1	libncursesw.so.6.6
libevent_openssl.a		libncursesw_g.a
libevent_openssl.so		libnuma.a
libevent_pthreads-2.1.so.7	libnuma.so
libevent_pthreads-2.1.so.7.0.1	libnuma.so.1
libevent_pthreads.a		libnuma.so.1.0.0
libevent_pthreads.so		libopa.so
libexpat.a			libopen-pal.so
libexpat.so			libopen-pal.so.80
libexpat.so.1			libopen-pal.so.80.0.5
libexpat.so.1.12.1		libopenblas-r0.3.33.a
libfabric.a			libopenblas-r0.3.33.so
libfabric.so			libopenblas.a
libfabric.so.1			libopenblas.so
libfabric.so.1.31.1		libopenblas.so.0
libfmpich.so			libpanel.a
libform.a			libpanel.so
libform.so			libpanel.so.6
libform.so.6			libpanel.so.6.6
libform.so.6.6			libpanel_g.a
libform_g.a			libpanelw.a
libformw.a			libpanelw.so
libformw.so			libpanelw.so.6
libformw.so.6			libpanelw.so.6.6
libformw.so.6.6			libpanelw_g.a
libformw_g.a			libpciaccess.a
libgcc_s.so.1			libpciaccess.so
libgdbm.a			libpciaccess.so.0
libgdbm.so			libpciaccess.so.0.11.1
libgdbm.so.6			libpmix.a
libgdbm.so.6.0.0		libpmix.so
libgdbm_compat.a		libpmix.so.2
libgdbm_compat.so		libpmix.so.2.23.0
libgdbm_compat.so.4		libprrte.a
libgdbm_compat.so.4.0.0		libprrte.so
libgettextlib-1.0.so		libprrte.so.3
libgettextlib.a			libprrte.so.3.1.0
libgettextlib.so		libpython3.14.so
libgettextpo.a			libpython3.14.so.1.0
libgettextpo.so			libpython3.so
libgettextpo.so.0		libquadmath.so.0
libgettextpo.so.0.6.0		libreadline.a
libgettextsrc-1.0.so		libreadline.so
libgettextsrc.a			libreadline.so.8
libgettextsrc.so		libreadline.so.8.3
libgfortran.so.5		libscalapack.so
libgomp.so.1			libscalapack.so.2.2
libgssapi_krb5.so		libscalapack.so.2.2.3
libgssapi_krb5.so.2		libsqlite3.a
libgssapi_krb5.so.2.2		libsqlite3.so
libgssrpc.so			libsqlite3.so.0
libgssrpc.so.4			libsqlite3.so.3.53.1
libgssrpc.so.4.2		libssp.so.0
libhistory.a			libstdc++.so.6
libhistory.so			libtextstyle.a
libhistory.so.8			libtextstyle.so
libhistory.so.8.3		libtextstyle.so.0
libhwasan.so.0			libtextstyle.so.0.2.7
libhwloc.a			libtinfo.a
libhwloc.so			libtinfo.so
libhwloc.so.15			libtinfo.so.6
libhwloc.so.15.10.2		libtinfo.so.6.6
libiconv.a			libtinfo_g.a
libiconv.so			libtinfow.a
libiconv.so.2			libtinfow.so
libiconv.so.2.7.0		libtinfow.so.6
libitm.so.1			libtinfow.so.6.6
libk5crypto.so			libtinfow_g.a
libk5crypto.so.3		libtmglib.so
libk5crypto.so.3.1		libtmglib.so.3
libkadm5clnt.so			libtmglib.so.3.12.0
libkadm5clnt_mit.so		libtsan.so.2
libkadm5clnt_mit.so.12		libubsan.so.1
libkadm5clnt_mit.so.12.0	libuuid.a
libkadm5srv.so			libuuid.so
libkadm5srv_mit.so		libuuid.so.1
libkadm5srv_mit.so.12		libuuid.so.1.3.0
libkadm5srv_mit.so.12.0		libverto.so
libkdb5.so			libverto.so.0
libkdb5.so.10			libverto.so.0.0
libkdb5.so.10.0			libxml2.so
libkrad.so			libxml2.so.16
libkrad.so.0			libxml2.so.16.1.3
libkrad.so.0.0			libz.a
libkrb5.so			libz.so
libkrb5.so.3			libz.so.1
libkrb5.so.3.3			libz.so.1.3.1.zlib-ng
libkrb5support.so		libzstd.a
libkrb5support.so.0		libzstd.so
libkrb5support.so.0.1		libzstd.so.1
liblapack.so			libzstd.so.1.5.7
liblapack.so.3			mpi.mod
liblapack.so.3.12.0		mpi_ext.mod
liblapacke.so			mpi_f08.mod
liblapacke.so.3			mpi_f08_callbacks.mod
liblapacke.so.3.12.0		mpi_f08_ext.mod
liblsan.so.0			mpi_f08_interfaces.mod
liblzma.a			mpi_f08_interfaces_callbacks.mod
liblzma.so			mpi_f08_types.mod
liblzma.so.5			mpi_types.mod
liblzma.so.5.8.3		openmpi
libmd.a				pkgconfig
libmd.so			pmpi_f08_interfaces.mod
libmd.so.0			preloadable_libintl.so
libmd.so.0.1.0			python3.14
libmenu.a			terminfo
$ ls ~/stacks/views/full
binutils     gmp	libiconv      mpfr		openmpi	  sqlite
bzip2	     hwloc	libmd	      mpich		openssh	  tar
curl	     krb5	libpciaccess  ncurses		openssl	  tcl
expat	     libbsd	libxcrypt     netlib-lapack	pigz	  unzip
gcc	     libedit	libxml2	      netlib-scalapack	pmix	  util-linux-uuid
gcc-runtime  libevent	lmod	      nghttp2		prrte	  xz
gdbm	     libfabric	lua	      numactl		python	  zlib-ng
gettext	     libffi	mpc	      openblas		readline  zstd
$ ls ~/stacks/views/full/gcc/
gcc-16.1.0-gcc-15.2.0
$ ls ~/stacks/views/full/gcc/gcc-16.1.0-gcc-15.2.0
bin  include  lib  lib64  libexec  share

By default, a view includes all packages, link and run dependencies included. Setting link: roots restricts it to root specs only:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  definitions:
  - mpis: [mpich]
  - mpis: [openmpi]
    when: 'env.get("SPACK_STACK_USE_OPENMPI", "") == "1"'
  - lapacks: [openblas, netlib-lapack]
  - mpi_packages: [netlib-scalapack]
  - serial_packages: [py-scipy]

  specs:
  - group: compiler
    specs:
    - gcc@16
    - lmod@8.7.67
    override:
      packages:
        c:
          prefer: [gcc@15]
        cxx:
          prefer: [gcc@15]
        fortran:
          prefer: [gcc@15]
  - group: stack
    needs: [compiler]
    specs:
    - matrix:
      - [$mpi_packages]
      - [$^mpis]
      - [$^lapacks]
    - matrix:
      - [$serial_packages]
      - [$^lapacks]
      exclude:
      - "py-scipy ^netlib-lapack"
    override:
      packages:
        c:
          prefer: [gcc@16]
        cxx:
          prefer: [gcc@16]
        fortran:
          prefer: [gcc@16]
      concretizer:
        unify: false
  view:
    default:
      root: views/default
      select: ['%gcc@16']
      exclude: [^mpich, ^netlib-lapack]
      link: roots
    full:
      root: views/full
      projections:
        ^mpi^lapack: '{name}/{name}-{version}-{^c.name}-{^c.version}-{^mpi.name}-{^mpi.version}-{^lapack.name}-{^lapack.version}'
        ^lapack: '{name}/{name}-{version}-{^c.name}-{^c.version}-{^lapack.name}-{^lapack.version}'
        '%c': '{name}/{name}-{version}-{^c.name}-{^c.version}'
        '%cxx': '{name}/{name}-{version}-{^cxx.name}-{^cxx.version}'
        '%fortran': '{name}/{name}-{version}-{^fortran.name}-{^fortran.version}'
        all: '{name}/{name}-{version}'
$ spack concretize
==> No new specs to concretize.
==> Updating view at /home/spack/stacks/views/default
$ ls ~/stacks/views/default
lib
$ ls ~/stacks/views/default/lib
cmake  libscalapack.so	libscalapack.so.2.2  libscalapack.so.2.2.3  pkgconfig  python3.14
$ ls ~/stacks/views/full
binutils     gmp	libiconv      mpfr		openmpi	  sqlite
bzip2	     hwloc	libmd	      mpich		openssh	  tar
curl	     krb5	libpciaccess  ncurses		openssl	  tcl
expat	     libbsd	libxcrypt     netlib-lapack	pigz	  unzip
gcc	     libedit	libxml2	      netlib-scalapack	pmix	  util-linux-uuid
gcc-runtime  libevent	lmod	      nghttp2		prrte	  xz
gdbm	     libfabric	lua	      numactl		python	  zlib-ng
gettext	     libffi	mpc	      openblas		readline  zstd

Now we see only the root packages in the default view. The rest are hidden, but are still available in the full view. Full documentation on views is available in the Spack environments guide.

Module Files

Module files are the standard mechanism for managing multiple software versions on HPC systems. In this section we’ll show how to configure and generate a hierarchical module structure with lmod.

Note

A more in-depth tutorial, focused only on module files, can be found at Module Files Tutorial. It covers the general architecture and differences between environment-modules and lmod that are out of scope here.

Since lmod is already part of our compiler group and has been installed, we just need to add the module command to our shell:

$ . $(spack location -i lmod)/lmod/lmod/init/bash

You should now have the module command available:

$ module --version

Modules based on Lua: Version 8.7.67 2025-10-04 20:16 -05:00
    by Robert McLay mclay@tacc.utexas.edu

The next step is to add some basic configuration to our spack.yaml to generate module files:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  definitions:
  - mpis: [mpich]
  - mpis: [openmpi]
    when: 'env.get("SPACK_STACK_USE_OPENMPI", "") == "1"'
  - lapacks: [openblas, netlib-lapack]
  - mpi_packages: [netlib-scalapack]
  - serial_packages: [py-scipy]

  specs:
  - group: compiler
    specs:
    - gcc@16
    - lmod@8.7.67
    override:
      packages:
        c:
          prefer: [gcc@15]
        cxx:
          prefer: [gcc@15]
        fortran:
          prefer: [gcc@15]
  - group: stack
    needs: [compiler]
    specs:
    - matrix:
      - [$mpi_packages]
      - [$^mpis]
      - [$^lapacks]
    - matrix:
      - [$serial_packages]
      - [$^lapacks]
      exclude:
      - "py-scipy ^netlib-lapack"
    override:
      packages:
        c:
          prefer: [gcc@16]
        cxx:
          prefer: [gcc@16]
        fortran:
          prefer: [gcc@16]
      concretizer:
        unify: false
  view: false
  modules:
    default:
      enable:
      - lmod
      roots:
        lmod: modules
      lmod:
        hierarchy:
        - mpi
        - lapack
        core_specs:
        - '%gcc@15'

This configuration tells Spack to:

  • generate lmod module files under modules/, with a hierarchy based on mpi and lapack

  • place all specs built with the system compiler (%gcc@15) into the Core designation

We can generate the module files and use them with the following commands:

$ spack module lmod refresh -y
$ module use $PWD/stacks/modules/linux-ubuntu26.04-x86_64/Core

Let’s check the generated module files:

$ module av

---- /home/spack/stacks/modules/linux-ubuntu26.04-x86_64/Core ----
   berkeley-db/18.1.40-27d4iyp
   binutils/2.46.1-6ene6zr
   bzip2/1.0.8-cbtgjrh
   compiler-wrapper/1.1.0-mmywg7x
   curl/8.20.0-j2fa7xl
   gcc-runtime/15.2.0-xm76mt3
   gcc-runtime/16.1.0-nn3xbub	   (D)
   gcc/15.2.0-yjlog5x
   gcc/16.1.0-hkkcdmj		   (D)
   gdbm/1.26-cq4dkuc
   gettext/1.0-sle3ix4
   glibc/2.43-yc4n2pp
   gmp/6.3.0-xpmsy5x
   less/692-ohmdb2l
   libiconv/1.18-vbwvgwx
   libxcrypt/4.5.2-bkzcu2s
   libxml2/2.15.3-ujlg2ua
   lmod/8.7.67-wf22jsg
   lua-luafilesystem/1.9.0-su5b2hs
   lua-luaposix/36.3-egzwtik
   lua/5.4.8-q66mudl
   mpc/1.4.1-gxmoxea
   mpfr/4.2.2-melg7ga
   ncurses/6.6-ekvivpv
   nghttp2/1.67.1-4lyasaz
   openssl/3.6.1-33yozp5
   perl/5.42.0-bvphs3b
   pigz/2.8-afklka7
   pkgconf/2.5.1-yvl6jpi
   readline/8.3-k25xiih
   tar/1.35-f5xe4px
   tcl/8.6.17-d4ibosb
   unzip/6.0-4qgksuy
   xz/5.8.3-hhjyyqy
   zlib-ng/2.3.3-g72d7i3
   zstd/1.5.7-63aruxk

  Where:
   D:  Default Module

If the avail list is too long
consider trying:

"module --default avail" or
"ml -d av" to just list the
default modules.
"module overview" or "ml ov"
to display the number of
modules for each name.

Use "module spider" to find
all possible modules and
extensions.
Use "module keyword key1 key2
..." to search for all
possible modules matching any
of the "keys".

The set of modules is already usable, and the hierarchy already works. For instance, we can load the gcc compiler and check that gcc is in our path and that many modules are available, all compiled with gcc@16:

$ module load gcc
$ which gcc
/home/spack/spack/opt/spack/linux-x86_64_v3/gcc-16.1.0-hkkcdmj2guegqqkt2w3e3owuklnyvgek/bin/gcc
$ gcc --version
gcc (Spack GCC) 16.1.0
Copyright (C) 2026 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ module av

---- /home/spack/stacks/modules/linux-ubuntu26.04-x86_64/gcc/16.1.0 ----
   bzip2/1.0.8-5kv66tq		(D)
   expat/2.8.1-kqm2d3g
   gdbm/1.26-2qw4v66		(D)
   gettext/1.0-qdxpkom
   hwloc/2.13.0-fkki5gm
   krb5/1.22.2-7a4l5v5
   libbsd/0.12.2-2zdnupj
   libedit/3.1-20251016-4binz2p
   libevent/2.1.12-ul3skja
   libfabric/2.5.1-o3dcvlb
   libffi/3.5.2-pkl6qup
   libiconv/1.18-xrsocgc	(D)
   libmd/1.1.0-ypuui7v
   libpciaccess/0.17-t5frspj
   libxcrypt/4.5.2-aca7p7l
   libxml2/2.15.3-3rlsf7j	(D)
   mpich/5.0.1-fxetsgw
   ncurses/6.6-romubi2		(D)
   netlib-lapack/3.12.1-77mpzq5
   numactl/2.0.19-vn7wvoe
   openblas/0.3.33-uh2kn3c
   openmpi/5.0.10-s6hcsd5
   openssh/10.3p1-plgdjpa
   openssl/3.6.1-mladgc5
   pigz/2.8-7ppidie		(D)
   pmix/6.1.0-2vgljh5
   prrte/4.1.0-wwm2ix6
   py-pybind11/3.0.2-jv2364q
   python-venv/1.0-iyp4doo
   python/3.14.5-vuoqpmo
   readline/8.3-teh56fo		(D)
   sqlite/3.53.1-qy73hym
   tar/1.35-u62ty4d		(D)
   util-linux-uuid/2.41-2wcnb6n
   xz/5.8.3-rrvhely		(D)
   zlib-ng/2.3.3-gc236kh	(D)
   zstd/1.5.7-mzuiikv

---- /home/spack/stacks/modules/linux-ubuntu26.04-x86_64/Core ----
   berkeley-db/18.1.40-27d4iyp
   binutils/2.46.1-6ene6zr	   (L)
   bzip2/1.0.8-cbtgjrh
   compiler-wrapper/1.1.0-mmywg7x
   curl/8.20.0-j2fa7xl
   gcc-runtime/15.2.0-xm76mt3	   (L)
   gcc-runtime/16.1.0-nn3xbub	   (D)
   gcc/15.2.0-yjlog5x
   gcc/16.1.0-hkkcdmj		   (L,D)
   gdbm/1.26-cq4dkuc
   gettext/1.0-sle3ix4		   (D)
   glibc/2.43-yc4n2pp		   (L)
   gmp/6.3.0-xpmsy5x		   (L)
   less/692-ohmdb2l
   libiconv/1.18-vbwvgwx
   libxcrypt/4.5.2-bkzcu2s	   (D)
   libxml2/2.15.3-ujlg2ua
   lmod/8.7.67-wf22jsg
   lua-luafilesystem/1.9.0-su5b2hs
   lua-luaposix/36.3-egzwtik
   lua/5.4.8-q66mudl
   mpc/1.4.1-gxmoxea		   (L)
   mpfr/4.2.2-melg7ga		   (L)
   ncurses/6.6-ekvivpv
   nghttp2/1.67.1-4lyasaz
   openssl/3.6.1-33yozp5	   (D)
   perl/5.42.0-bvphs3b
   pigz/2.8-afklka7
   pkgconf/2.5.1-yvl6jpi
   readline/8.3-k25xiih
   tar/1.35-f5xe4px
   tcl/8.6.17-d4ibosb
   unzip/6.0-4qgksuy
   xz/5.8.3-hhjyyqy
   zlib-ng/2.3.3-g72d7i3	   (L)
   zstd/1.5.7-63aruxk		   (L,D)

  Where:
   D:  Default Module
   L:  Module is loaded

If the avail list is too long
consider trying:

"module --default avail" or
"ml -d av" to just list the
default modules.
"module overview" or "ml ov"
to display the number of
modules for each name.

Use "module spider" to find
all possible modules and
extensions.
Use "module keyword key1 key2
..." to search for all
possible modules matching any
of the "keys".


$ module unload gcc

The default configuration has a few rough edges, though:

  • dependency modules from gcc clutter the listing with packages users will never load directly

  • hashes in module names make scripts non-portable across similar environments

  • some modules may need custom environment variables specific to site policy

To address all these needs, we can extend our modules configuration:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  definitions:
  - mpis: [mpich]
  - mpis: [openmpi]
    when: 'env.get("SPACK_STACK_USE_OPENMPI", "") == "1"'
  - lapacks: [openblas, netlib-lapack]
  - mpi_packages: [netlib-scalapack]
  - serial_packages: [py-scipy]

  specs:
  - group: compiler
    specs:
    - gcc@16
    - lmod@8.7.67
    override:
      packages:
        c:
          prefer: [gcc@15]
        cxx:
          prefer: [gcc@15]
        fortran:
          prefer: [gcc@15]
  - group: stack
    needs: [compiler]
    specs:
    - matrix:
      - [$mpi_packages]
      - [$^mpis]
      - [$^lapacks]
    - matrix:
      - [$serial_packages]
      - [$^lapacks]
      exclude:
      - "py-scipy ^netlib-lapack"
    override:
      packages:
        c:
          prefer: [gcc@16]
        cxx:
          prefer: [gcc@16]
        fortran:
          prefer: [gcc@16]
      concretizer:
        unify: false
  view: false
  modules:
    default:
      enable:
      - lmod
      roots:
        lmod: modules
      lmod:
        hierarchy:
        - mpi
        - lapack
        core_specs:
        - '%gcc@15'
        hash_length: 0
        include:
        - gcc
        exclude:
        - '%gcc@15'
        - autoconf
        all:
          environment:
            set:
              '{name}_ROOT': '{prefix}'
        openmpi:
          environment:
            set:
              SLURM_MPI_TYPE: pmi2
              OMPI_MCA_btl_openib_warn_default_gid_prefix: '0'
        projections:
          all: '{name}/{version}'

Regenerate the modules:

$ spack module lmod refresh --delete-tree -y
==> Regenerating lmod module files

Now we have module files without hashes, a correct hierarchy, and our custom environment variables:

$ module load gcc
$ module load openmpi openblas netlib-scalapack py-scipy
$ module av

---- /home/spack/stacks/modules/linux-ubuntu26.04-x86_64/gcc/16.1.0 ----
   bzip2/1.0.8
   expat/2.8.1
   gdbm/1.26
   gettext/1.0
   hwloc/2.13.0
   krb5/1.22.2
   libbsd/0.12.2
   libedit/3.1-20251016
   libevent/2.1.12
   libfabric/2.5.1
   libffi/3.5.2
   libiconv/1.18
   libmd/1.1.0
   libpciaccess/0.17
   libxcrypt/4.5.2
   libxml2/2.15.3
   mpich/5.0.1
   ncurses/6.6
   netlib-lapack/3.12.1
   numactl/2.0.19
   openblas/0.3.33
   openmpi/5.0.10
   openssh/10.3p1
   openssl/3.6.1
   pigz/2.8
   pmix/6.1.0
   prrte/4.1.0
   py-pybind11/3.0.2
   python-venv/1.0
   python/3.14.5
   readline/8.3
   sqlite/3.53.1
   tar/1.35
   util-linux-uuid/2.41
   xz/5.8.3
   zlib-ng/2.3.3
   zstd/1.5.7

---- /home/spack/stacks/modules/linux-ubuntu26.04-x86_64/Core ----
   compiler-wrapper/1.1.0
   gcc-runtime/16.1.0
   gcc/15.2.0
   gcc/16.1.0		  (L,D)
   glibc/2.43		  (L)

  Where:
   D:  Default Module
   L:  Module is loaded

If the avail list is too long
consider trying:

"module --default avail" or
"ml -d av" to just list the
default modules.
"module overview" or "ml ov"
to display the number of
modules for each name.

Use "module spider" to find
all possible modules and
extensions.
Use "module keyword key1 key2
..." to search for all
possible modules matching any
of the "keys".


$ module load mpich

Lmod is automatically
replacing "openmpi/5.0.10"
with "mpich/5.0.1".


Due to MODULEPATH changes, the following have been reloaded:
  1) netlib-scalapack/2.2.3

$ module load netlib-lapack

Lmod is automatically
replacing "openblas/0.3.33"
with "netlib-lapack/3.12.1".


Due to MODULEPATH changes, the following have been reloaded:
  1) netlib-scalapack/2.2.3

$ module purge

This concludes the stacks tutorial.

Summary

In this tutorial we built a realistic HPC software stack: a cross-product of MPI and LAPACK libraries, compiled with a newer compiler than the system default. Along the way we used:

  • Spec groups with needs and override to control build order and compiler selection per group

  • Spec matrices and definitions to express the cross-product concisely

  • Conditional definitions to make the MPI selection configurable at concretization time

  • Filesystem views and module files to make the software accessible to users