Tutorial setup

If you have not done the prior sections, you’ll need to start the docker image:

docker run -it ghcr.io/spack/tutorial:isc26

and then set Spack up like this:

git clone --depth=2 --branch=releases/v1.2 https://github.com/spack/spack
. spack/share/spack/setup-env.sh
spack mirror add --unsigned tutorial /mirror
spack bootstrap now
spack compiler find

See the Basic Installation Tutorial for full details on setup. For more help, join us in the #tutorial channel on Slack – get an invitation at slack.spack.io

Stacks Tutorial

Spack environments are a powerful tool not just for managing a single user’s project, but for deploying entire software stacks. In this tutorial we will see how to use them to manage large deployments, a common pattern in HPC centers.

What usually characterizes these deployments, compared to a typical single-user environment, is the need to provide the same set of packages built against a variety of configurations: different MPI libraries, LAPACK implementations, or compilers.

Below, we’ll build a representative example of such a deployment. Our goal is to install netlib-scalapack compiled with gcc@16, which is newer than the system-provided gcc@15, and linked against:

  • two MPI libraries (openmpi and mpich)

  • two LAPACK providers (openblas and netlib-lapack)

We’ll also install py-scipy linked against openblas.

We’ll first focus on how to configure and install the software correctly. Then we’ll discuss how to make it accessible to users through filesystem views and module files.

Installing a Software Stack

Let’s start by creating an environment in a directory of our choice:

$ spack env activate --create ~/stacks
$ spack env status
==> In environment /home/spack/stacks

The first step is to install the compiler we want to use. Let’s write a minimal spack.yaml that contains just the compiler and lmod, which we’ll need later to generate module files:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  specs:
  - gcc@16 %gcc@15
  - lmod@8.7.67 %gcc@15
  view: false

Concretize and install:

$ spack concretize
==> Concretized 2 specs:
[b]  vlwdxlj  gcc@16.1.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs+profiled+strip build_system=autotools build_type=Release languages:='c,c++,fortran' platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  cuzgmus	  ^binutils@2.46.0~debuginfod+gas~gprofng~headers~interwork+ld~libiberty~lto~nls~pgo+plugins build_system=autotools compress_debug_sections=zlib libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  mmywg7x	  ^compiler-wrapper@1.1.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  ao2onuz	  ^diffutils@3.12 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  vbwvgwx	      ^libiconv@1.18 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  z7wqeps	  ^gawk@5.4.0~nls build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  k25xiih	      ^readline@8.3 build_system=autotools patches:=21f0a03,72dee13,e273643 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[e]  yjlog5x	  ^gcc@15.2.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs~profiled~strip build_system=autotools build_type=RelWithDebInfo languages:='c,c++,fortran' platform=linux os=ubuntu26.04 target=x86_64_v3
[+]  xm76mt3	  ^gcc-runtime@15.2.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[e]  yc4n2pp	  ^glibc@2.43 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  r4lhaok	  ^gmake@4.4.1~guile build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  xpmsy5x	  ^gmp@6.3.0+cxx build_system=autotools libs:=shared,static patches:=28a6673 platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  jo3eg4r	      ^autoconf@2.72 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  yhkgfai	      ^automake@1.18.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  y6uqrto	      ^m4@1.4.21+sigsegv build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  phcmfqk		  ^libsigsegv@2.15 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  3gdq456	  ^libtool@2.5.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  mguwetc	      ^file@5.46+static build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  hhjyyqy		  ^xz@5.8.3~pic build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  punbqrx	      ^findutils@4.10.0 build_system=autotools patches:=440b954 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  gxmoxea	  ^mpc@1.4.1 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  melg7ga	  ^mpfr@4.2.2 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  sf5iutv	      ^autoconf-archive@2024.10.16 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  bvphs3b	  ^perl@5.42.0+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  27d4iyp	      ^berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  cbtgjrh	      ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  cq4dkuc	      ^gdbm@1.26 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  ohmdb2l	      ^less@692 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  jvlaabk	  ^texinfo@7.2~xs build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  sle3ix4	      ^gettext@1.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  ujlg2ua		  ^libxml2@2.15.3+pic~python+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  f5xe4px		  ^tar@1.35 build_system=autotools zip=pigz platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  afklka7		      ^pigz@2.8 build_system=makefile platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  ekvivpv	      ^ncurses@6.6~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  g72d7i3	  ^zlib-ng@2.3.3+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  63aruxk	  ^zstd@1.5.7+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  wf22jsg  lmod@8.7.67+auto_swap~redirect build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  ywyq4f5	  ^bc@1.08.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  litd2mh	      ^ed@1.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  q66mudl	  ^lua@5.4.8+shared build_system=makefile fetcher=curl platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  j2fa7xl	      ^curl@8.20.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2+nghttp2 build_system=autotools libs:=shared,static tls:=openssl platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  4lyasaz		  ^nghttp2@1.67.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  33yozp5		  ^openssl@3.6.1~docs+shared build_system=generic certs=mozilla platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  axeuqg3		      ^ca-certificates-mozilla@2026-03-19 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  4qgksuy	      ^unzip@6.0 build_system=makefile patches:=179330d,24582ff,251d575,3371314,44599c8,47e9def,4e5a081,59c0983,64f6498,74bc961,7d8e5c7,81ca46c,881d2ed,aced0f2,b6f64d7,b7a14c3,c9a863e,ee9e260,f6f6236,f88b9d4,fde8f9d platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  su5b2hs	  ^lua-luafilesystem@1.9.0 build_system=lua platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  egzwtik	  ^lua-luaposix@36.3 build_system=lua platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  bkzcu2s	      ^libxcrypt@4.5.2~obsolete_api build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  yvl6jpi	  ^pkgconf@2.5.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  ci53fkm	  ^procps@4.0.6+nls build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  a27iqiv	  ^sed@4.9 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  d4ibosb	  ^tcl@8.6.17 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0

$ spack install
[ ] d4ibosb tcl@8.6.17 fetching from build cache (0s)
[ ] 4qgksuy unzip@6.0 fetching from build cache (0s)
[ ] k25xiih readline@8.3 fetching from build cache (0s)
[ ] 4qgksuy unzip@6.0 relocating (0s)
[ ] 4lyasaz nghttp2@1.67.1 fetching from build cache (0s)
[ ] vlwdxlj gcc@16.1.0 fetching from build cache (0s)
[ ] d4ibosb tcl@8.6.17 relocating (0s)
[ ] k25xiih readline@8.3 relocating (0s)
[ ] 4lyasaz nghttp2@1.67.1 relocating (0s)
[+] 4qgksuy unzip@6.0 /home/spack/spack/opt/spack/linux-x86_64_v3/unzip-6.0-4qgksuyohalb7cfo76mqqjlmndeh3tyg (0s)
[+] k25xiih readline@8.3 /home/spack/spack/opt/spack/linux-x86_64_v3/readline-8.3-k25xiihp5dun6qwe3e5ouzdigwr5yuvz (0s)
[+] 4lyasaz nghttp2@1.67.1 /home/spack/spack/opt/spack/linux-x86_64_v3/nghttp2-1.67.1-4lyasazuklpnvidiw4ty5wvybxvkh7na (0s)
[ ] vlwdxlj gcc@16.1.0 relocating (0s)
[ ] j2fa7xl curl@8.20.0 fetching from build cache (0s)
[ ] j2fa7xl curl@8.20.0 relocating (0s)
[+] j2fa7xl curl@8.20.0 /home/spack/spack/opt/spack/linux-x86_64_v3/curl-8.20.0-j2fa7xlzdvkagqked76erodvcmcd2oqp (0s)
[ ] q66mudl lua@5.4.8 fetching from build cache (0s)
[ ] q66mudl lua@5.4.8 relocating (0s)
[+] q66mudl lua@5.4.8 /home/spack/spack/opt/spack/linux-x86_64_v3/lua-5.4.8-q66mudlsappt33u2frzcijegjay54ep3 (0s)
[ ] egzwtik lua-luaposix@36.3 fetching from build cache (0s)
[ ] su5b2hs lua-luafilesystem@1.9.0 fetching from build cache (0s)
[ ] egzwtik lua-luaposix@36.3 relocating (0s)
[ ] su5b2hs lua-luafilesystem@1.9.0 relocating (0s)
[+] su5b2hs lua-luafilesystem@1.9.0 /home/spack/spack/opt/spack/linux-x86_64_v3/lua-luafilesystem-1.9.0-su5b2hsgwgk2a4b7hkys2r23kf3me6lh (0s)
[+] egzwtik lua-luaposix@36.3 /home/spack/spack/opt/spack/linux-x86_64_v3/lua-luaposix-36.3-egzwtikjsfubsjiqpwdjemys3xqv2ytf (1s)
[+] d4ibosb tcl@8.6.17 /home/spack/spack/opt/spack/linux-x86_64_v3/tcl-8.6.17-d4ibosbgfm7536ij2dmdu6nnll5g5zpm (1s)
[ ] wf22jsg lmod@8.7.67 fetching from build cache (0s)
[ ] wf22jsg lmod@8.7.67 relocating (0s)
[+] wf22jsg lmod@8.7.67 /home/spack/spack/opt/spack/linux-x86_64_v3/lmod-8.7.67-wf22jsggn5vfsyoyuqm6ban76y7333we (0s)
[+] vlwdxlj gcc@16.1.0 /home/spack/spack/opt/spack/linux-x86_64_v3/gcc-16.1.0-vlwdxljzbis42fjrlecvyo4tmuq5eebn (1s)

Spec groups

We are now ready to build the software stack on top of this compiler. To do that cleanly, we’ll use spec groups, which allow you to organize specs into named collections, each with its own configuration and concretization order.

Let’s start with a single spec and a two-group spack.yaml:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  specs:
  - group: compiler
    specs:
    - gcc@16 %gcc@15
    - lmod@8.7.67 %gcc@15
  - group: stack
    needs: [compiler]
    specs:
    - netlib-scalapack %gcc@16 ^openblas ^openmpi
  view: false

The environment declares two groups:

  1. The compiler group contains GCC 16 and lmod (both already installed).

  2. The stack group lists the software we want to deploy.

The needs: [compiler] declaration ensures that the compiler group is always concretized before the stack group, making its specs available for reuse.

Let’s concretize to see this in action:

$ spack concretize
==> Concretizing the 'compiler' group of specs
==> Concretizing the 'stack' group of specs
==> Concretized 3 specs:
[+]  vlwdxlj  gcc@16.1.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs+profiled+strip build_system=autotools build_type=Release languages:='c,c++,fortran' platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  cuzgmus	  ^binutils@2.46.0~debuginfod+gas~gprofng~headers~interwork+ld~libiberty~lto~nls~pgo+plugins build_system=autotools compress_debug_sections=zlib libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  mmywg7x	  ^compiler-wrapper@1.1.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  ao2onuz	  ^diffutils@3.12 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  vbwvgwx	      ^libiconv@1.18 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  z7wqeps	  ^gawk@5.4.0~nls build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  k25xiih	      ^readline@8.3 build_system=autotools patches:=21f0a03,72dee13,e273643 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[e]  yjlog5x	  ^gcc@15.2.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs~profiled~strip build_system=autotools build_type=RelWithDebInfo languages:='c,c++,fortran' platform=linux os=ubuntu26.04 target=x86_64_v3
[+]  xm76mt3	  ^gcc-runtime@15.2.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[e]  yc4n2pp	  ^glibc@2.43 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  r4lhaok	  ^gmake@4.4.1~guile build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  xpmsy5x	  ^gmp@6.3.0+cxx build_system=autotools libs:=shared,static patches:=28a6673 platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  jo3eg4r	      ^autoconf@2.72 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  yhkgfai	      ^automake@1.18.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  y6uqrto	      ^m4@1.4.21+sigsegv build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  phcmfqk		  ^libsigsegv@2.15 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  3gdq456	  ^libtool@2.5.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  mguwetc	      ^file@5.46+static build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  hhjyyqy		  ^xz@5.8.3~pic build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  punbqrx	      ^findutils@4.10.0 build_system=autotools patches:=440b954 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  gxmoxea	  ^mpc@1.4.1 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  melg7ga	  ^mpfr@4.2.2 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  sf5iutv	      ^autoconf-archive@2024.10.16 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  bvphs3b	  ^perl@5.42.0+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  27d4iyp	      ^berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  cbtgjrh	      ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  cq4dkuc	      ^gdbm@1.26 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  ohmdb2l	      ^less@692 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  jvlaabk	  ^texinfo@7.2~xs build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  sle3ix4	      ^gettext@1.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  ujlg2ua		  ^libxml2@2.15.3+pic~python+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  f5xe4px		  ^tar@1.35 build_system=autotools zip=pigz platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  afklka7		      ^pigz@2.8 build_system=makefile platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  ekvivpv	      ^ncurses@6.6~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  g72d7i3	  ^zlib-ng@2.3.3+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  63aruxk	  ^zstd@1.5.7+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  wf22jsg  lmod@8.7.67+auto_swap~redirect build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  ywyq4f5	  ^bc@1.08.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  litd2mh	      ^ed@1.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  q66mudl	  ^lua@5.4.8+shared build_system=makefile fetcher=curl platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  j2fa7xl	      ^curl@8.20.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2+nghttp2 build_system=autotools libs:=shared,static tls:=openssl platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  4lyasaz		  ^nghttp2@1.67.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  33yozp5		  ^openssl@3.6.1~docs+shared build_system=generic certs=mozilla platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  axeuqg3		      ^ca-certificates-mozilla@2026-03-19 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[+]  4qgksuy	      ^unzip@6.0 build_system=makefile patches:=179330d,24582ff,251d575,3371314,44599c8,47e9def,4e5a081,59c0983,64f6498,74bc961,7d8e5c7,81ca46c,881d2ed,aced0f2,b6f64d7,b7a14c3,c9a863e,ee9e260,f6f6236,f88b9d4,fde8f9d platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  su5b2hs	  ^lua-luafilesystem@1.9.0 build_system=lua platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  egzwtik	  ^lua-luaposix@36.3 build_system=lua platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  bkzcu2s	      ^libxcrypt@4.5.2~obsolete_api build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  yvl6jpi	  ^pkgconf@2.5.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  ci53fkm	  ^procps@4.0.6+nls build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  a27iqiv	  ^sed@4.9 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  d4ibosb	  ^tcl@8.6.17 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  b4nkwcf  netlib-scalapack@2.2.3~ipo~pic+shared build_system=cmake build_type=Release generator=make platform=linux os=ubuntu26.04 target=x86_64_v3 %c,fortran=gcc@16.1.0
[b]  yonurps	  ^cmake@3.31.11~doc+ncurses+ownlibs~qtgui build_system=generic build_type=Release platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  6b5h3dv	      ^curl@8.20.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2+nghttp2 build_system=autotools libs:=shared,static tls:=openssl platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  iiwpnp7		  ^nghttp2@1.67.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  djpmji7		      ^diffutils@3.12 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  yoq4amm		  ^openssl@3.6.1~docs+shared build_system=generic certs=mozilla platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  iehcspo	      ^ncurses@6.6~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  yd7lgm7	      ^zlib-ng@2.3.3+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  ghfita3	  ^gcc-runtime@16.1.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  6hzfsbp	  ^gmake@4.4.1~guile build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  kdylbqh	  ^openblas@0.3.33~bignuma~consistent_fpcsr+dynamic_dispatch+fortran~ilp64+locking+pic+shared~static build_system=makefile patches:=723ddc1 symbol_suffix=none threads=none platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx,fortran=gcc@16.1.0
[b]  nzr6pd7	  ^openmpi@5.0.10+atomics~cuda~debug+fortran~gpfs~internal-hwloc~internal-libevent~internal-pmix~ipv6~java~lustre~memchecker~openshmem~rocm~romio+rsh~static~two_level_namespace+vt+wrapper-rpath build_system=autotools fabrics:=none patches:=646eb1a romio-filesystem:=none schedulers:=none platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx,fortran=gcc@16.1.0
[b]  yg7nv7p	      ^autoconf@2.72 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  47vgtt5		  ^m4@1.4.21+sigsegv build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  r6vd65h		      ^libsigsegv@2.15 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  pdslnit	      ^automake@1.18.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  nn3ymay	      ^hwloc@2.13.0~cairo~cuda~gl~level_zero~libudev+libxml2~nvml~opencl+pci~rocm build_system=autotools libs:=shared,static patches:=b4db98b platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  cmgnxt6		  ^libpciaccess@0.17 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  cy6lybd		      ^util-macros@1.20.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  cxesrxa		  ^libxml2@2.15.3+pic~python+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  qs6av53		      ^libiconv@1.18 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  bf6e7x2		      ^xz@5.8.3~pic build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  m5ts755	      ^libevent@2.1.12+openssl build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  raf5qk6	      ^libtool@2.5.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  fc5ezqn		  ^file@5.46+static build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  lq6tsmz		      ^zstd@1.5.7+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  l3if3dy		  ^findutils@4.10.0 build_system=autotools patches:=440b954 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  g3gt6ze		      ^gettext@1.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  tw6253j			  ^tar@1.35 build_system=autotools zip=pigz platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  4diropt			      ^pigz@2.8 build_system=makefile platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  swhw6f5	      ^numactl@2.0.19 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  kjx2nfr	      ^openssh@10.3p1+gssapi build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  awic2yh		  ^krb5@1.22.2+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  yws52fh		      ^bison@3.8.2~color build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  4c4uuub		  ^libedit@3.1-20251016 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  44jhzjg		  ^libxcrypt@4.5.2~obsolete_api build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  xxg272q	      ^perl@5.42.0+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  aev2xm7		  ^berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  ofahxlj		  ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  2af63my		  ^gdbm@1.26 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  cbnh7bw		      ^readline@8.3 build_system=autotools patches:=21f0a03,72dee13,e273643 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  3w5zzn3		  ^less@692 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  xp6fyih	      ^pkgconf@2.5.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  qcxnkrd	      ^pmix@6.1.0~munge~python build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  cj2ohvh	      ^prrte@4.1.0 build_system=autotools patches:=64faa1a,91b28f5 schedulers:=none platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  4p7olya		  ^flex@2.6.3+lex~nls build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0

Notice that the compiler group is processed first, and gcc@16 is immediately reused by the stack group.

We just used %gcc@16 to pin the compiler, but it’s a coarse constraint that must be repeated on every spec and does not distinguish between C, C++, and Fortran compilers. Spec groups offer a better tool: the override block, which scopes any Spack configuration to a single group.

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  specs:
  - group: compiler
    specs:
    - gcc@16
    - lmod@8.7.67
    override:
      packages:
        c:
          prefer: [gcc@15]
        cxx:
          prefer: [gcc@15]
        fortran:
          prefer: [gcc@15]
  - group: stack
    needs: [compiler]
    specs:
    - netlib-scalapack ^openblas ^openmpi
    override:
      packages:
        c:
          prefer: [gcc@16]
        cxx:
          prefer: [gcc@16]
        fortran:
          prefer: [gcc@16]
  view: false

Here we used the override block to set compiler preferences at the language level, once for each group:

  1. The compiler group prefers gcc@15

  2. The stack group prefers gcc@16

These language preferences are expressed once and apply automatically to every spec in the group.

Tuning concretizer options

Now we add a second MPI variant to the stack, listing both ^openmpi and ^mpich variants of netlib-scalapack:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  specs:
  - group: compiler
    specs:
    - gcc@16
    - lmod@8.7.67
    override:
      packages:
        c:
          prefer: [gcc@15]
        cxx:
          prefer: [gcc@15]
        fortran:
          prefer: [gcc@15]
  - group: stack
    needs: [compiler]
    specs:
    - netlib-scalapack ^openblas ^openmpi
    - netlib-scalapack ^openblas ^mpich
    override:
      packages:
        c:
          prefer: [gcc@16]
        cxx:
          prefer: [gcc@16]
        fortran:
          prefer: [gcc@16]
  view: false

The stack group now has two conflicting configurations, so, before concretizing, let’s check the concretizer:unify option:

$ spack config get concretizer | grep unify
  unify: true

With unify: true, the concretizer restricts the environment to a single configuration of each package within its root unification set: the nodes reachable from the root specs via link or run edges. Pure build dependencies, which fall outside the set, are not affected:

_images/stacks-unify.svg

This is the right default for a single-user environment, since it ensures a filesystem view can be built without conflicts. A software stack, however, requires multiple configurations of the same package by design, so we need to relax this constraint.

Since only the stack group has conflicting configurations, we add unify: false to its override block and concretize:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  specs:
  - group: compiler
    specs:
    - gcc@16
    - lmod@8.7.67
    override:
      packages:
        c:
          prefer: [gcc@15]
        cxx:
          prefer: [gcc@15]
        fortran:
          prefer: [gcc@15]
  - group: stack
    needs: [compiler]
    specs:
    - netlib-scalapack ^openblas ^openmpi
    - netlib-scalapack ^openblas ^mpich
    override:
      packages:
        c:
          prefer: [gcc@16]
        cxx:
          prefer: [gcc@16]
        fortran:
          prefer: [gcc@16]
      concretizer:
        unify: false
  view: false
$ spack concretize
==> Concretizing the 'compiler' group of specs
==> Concretizing the 'stack' group of specs
==> Starting concretization pool with 2 processes
==> Concretized 4 specs:
[+]  vlwdxlj  gcc@16.1.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs+profiled+strip build_system=autotools build_type=Release languages:='c,c++,fortran' platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  cuzgmus	  ^binutils@2.46.0~debuginfod+gas~gprofng~headers~interwork+ld~libiberty~lto~nls~pgo+plugins build_system=autotools compress_debug_sections=zlib libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  mmywg7x	  ^compiler-wrapper@1.1.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  ao2onuz	  ^diffutils@3.12 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  vbwvgwx	      ^libiconv@1.18 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  z7wqeps	  ^gawk@5.4.0~nls build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  k25xiih	      ^readline@8.3 build_system=autotools patches:=21f0a03,72dee13,e273643 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[e]  yjlog5x	  ^gcc@15.2.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs~profiled~strip build_system=autotools build_type=RelWithDebInfo languages:='c,c++,fortran' platform=linux os=ubuntu26.04 target=x86_64_v3
[+]  xm76mt3	  ^gcc-runtime@15.2.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[e]  yc4n2pp	  ^glibc@2.43 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  r4lhaok	  ^gmake@4.4.1~guile build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  xpmsy5x	  ^gmp@6.3.0+cxx build_system=autotools libs:=shared,static patches:=28a6673 platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  jo3eg4r	      ^autoconf@2.72 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  yhkgfai	      ^automake@1.18.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  y6uqrto	      ^m4@1.4.21+sigsegv build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  phcmfqk		  ^libsigsegv@2.15 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  3gdq456	  ^libtool@2.5.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  mguwetc	      ^file@5.46+static build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  hhjyyqy		  ^xz@5.8.3~pic build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  punbqrx	      ^findutils@4.10.0 build_system=autotools patches:=440b954 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  gxmoxea	  ^mpc@1.4.1 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  melg7ga	  ^mpfr@4.2.2 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  sf5iutv	      ^autoconf-archive@2024.10.16 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  bvphs3b	  ^perl@5.42.0+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  27d4iyp	      ^berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  cbtgjrh	      ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  cq4dkuc	      ^gdbm@1.26 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  ohmdb2l	      ^less@692 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  jvlaabk	  ^texinfo@7.2~xs build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  sle3ix4	      ^gettext@1.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  ujlg2ua		  ^libxml2@2.15.3+pic~python+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  f5xe4px		  ^tar@1.35 build_system=autotools zip=pigz platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  afklka7		      ^pigz@2.8 build_system=makefile platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  ekvivpv	      ^ncurses@6.6~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  g72d7i3	  ^zlib-ng@2.3.3+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  63aruxk	  ^zstd@1.5.7+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  wf22jsg  lmod@8.7.67+auto_swap~redirect build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  ywyq4f5	  ^bc@1.08.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  litd2mh	      ^ed@1.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  q66mudl	  ^lua@5.4.8+shared build_system=makefile fetcher=curl platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  j2fa7xl	      ^curl@8.20.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2+nghttp2 build_system=autotools libs:=shared,static tls:=openssl platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  4lyasaz		  ^nghttp2@1.67.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  33yozp5		  ^openssl@3.6.1~docs+shared build_system=generic certs=mozilla platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  axeuqg3		      ^ca-certificates-mozilla@2026-03-19 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[+]  4qgksuy	      ^unzip@6.0 build_system=makefile patches:=179330d,24582ff,251d575,3371314,44599c8,47e9def,4e5a081,59c0983,64f6498,74bc961,7d8e5c7,81ca46c,881d2ed,aced0f2,b6f64d7,b7a14c3,c9a863e,ee9e260,f6f6236,f88b9d4,fde8f9d platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  su5b2hs	  ^lua-luafilesystem@1.9.0 build_system=lua platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  egzwtik	  ^lua-luaposix@36.3 build_system=lua platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  bkzcu2s	      ^libxcrypt@4.5.2~obsolete_api build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  yvl6jpi	  ^pkgconf@2.5.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  ci53fkm	  ^procps@4.0.6+nls build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  a27iqiv	  ^sed@4.9 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  d4ibosb	  ^tcl@8.6.17 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  x425zhl  netlib-scalapack@2.2.3~ipo~pic+shared build_system=cmake build_type=Release generator=make platform=linux os=ubuntu26.04 target=x86_64_v3 %c,fortran=gcc@16.1.0
[b]  yonurps	  ^cmake@3.31.11~doc+ncurses+ownlibs~qtgui build_system=generic build_type=Release platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  6b5h3dv	      ^curl@8.20.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2+nghttp2 build_system=autotools libs:=shared,static tls:=openssl platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  iiwpnp7		  ^nghttp2@1.67.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  djpmji7		      ^diffutils@3.12 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  yoq4amm		  ^openssl@3.6.1~docs+shared build_system=generic certs=mozilla platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  iehcspo	      ^ncurses@6.6~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  yd7lgm7	      ^zlib-ng@2.3.3+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  ghfita3	  ^gcc-runtime@16.1.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  6hzfsbp	  ^gmake@4.4.1~guile build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  bovk7dt	  ^mpich@5.0.1~argobots~cuda+fortran+hwloc+hydra~level_zero+libxml2+pci~rocm+romio~slurm~vci~verbs+wrapperrpath~xpmem build_system=autotools datatype-engine=auto device=ch4 netmod=ofi pmi=default platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx,fortran=gcc@16.1.0
[b]  l3if3dy	      ^findutils@4.10.0 build_system=autotools patches:=440b954 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  g3gt6ze		  ^gettext@1.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  tw6253j		      ^tar@1.35 build_system=autotools zip=pigz platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  4diropt			  ^pigz@2.8 build_system=makefile platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  nn3ymay	      ^hwloc@2.13.0~cairo~cuda~gl~level_zero~libudev+libxml2~nvml~opencl+pci~rocm build_system=autotools libs:=shared,static patches:=b4db98b platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  s6cemsy	      ^libfabric@2.5.1~asan~cuda~debug~kdreg~level_zero~lsan~rocm~tsan~ubsan~uring build_system=autotools fabrics:=sockets,tcp,udp platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  cmgnxt6	      ^libpciaccess@0.17 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  cy6lybd		  ^util-macros@1.20.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  cxesrxa	      ^libxml2@2.15.3+pic~python+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  qs6av53		  ^libiconv@1.18 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  bf6e7x2		  ^xz@5.8.3~pic build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  xp6fyih	      ^pkgconf@2.5.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  afnel7p	      ^python@3.14.5+bz2+ctypes+dbm~debug~freethreading+libxml2+lzma~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~static~tests~tkinter+uuid+zlib+zstd build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  ofahxlj		  ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  xkvh6gd		  ^expat@2.8.1+libbsd build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  t623nqz		      ^libbsd@0.12.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  w3w22wx			  ^libmd@1.1.0 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  2af63my		  ^gdbm@1.26 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  jgqwv65		  ^libffi@3.5.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  cbnh7bw		  ^readline@8.3 build_system=autotools patches:=21f0a03,72dee13,e273643 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  uliuxbf		  ^sqlite@3.53.1+column_metadata+fts+rtree build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  v3sdaph		  ^util-linux-uuid@2.41 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  lq6tsmz		  ^zstd@1.5.7+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  kdylbqh	  ^openblas@0.3.33~bignuma~consistent_fpcsr+dynamic_dispatch+fortran~ilp64+locking+pic+shared~static build_system=makefile patches:=723ddc1 symbol_suffix=none threads=none platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx,fortran=gcc@16.1.0
[b]  b4nkwcf  netlib-scalapack@2.2.3~ipo~pic+shared build_system=cmake build_type=Release generator=make platform=linux os=ubuntu26.04 target=x86_64_v3 %c,fortran=gcc@16.1.0
[b]  nzr6pd7	  ^openmpi@5.0.10+atomics~cuda~debug+fortran~gpfs~internal-hwloc~internal-libevent~internal-pmix~ipv6~java~lustre~memchecker~openshmem~rocm~romio+rsh~static~two_level_namespace+vt+wrapper-rpath build_system=autotools fabrics:=none patches:=646eb1a romio-filesystem:=none schedulers:=none platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx,fortran=gcc@16.1.0
[b]  yg7nv7p	      ^autoconf@2.72 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  47vgtt5		  ^m4@1.4.21+sigsegv build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  r6vd65h		      ^libsigsegv@2.15 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  pdslnit	      ^automake@1.18.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  m5ts755	      ^libevent@2.1.12+openssl build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  raf5qk6	      ^libtool@2.5.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  fc5ezqn		  ^file@5.46+static build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  swhw6f5	      ^numactl@2.0.19 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  kjx2nfr	      ^openssh@10.3p1+gssapi build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  awic2yh		  ^krb5@1.22.2+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  yws52fh		      ^bison@3.8.2~color build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  4c4uuub		  ^libedit@3.1-20251016 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  44jhzjg		  ^libxcrypt@4.5.2~obsolete_api build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  xxg272q	      ^perl@5.42.0+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  aev2xm7		  ^berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  3w5zzn3		  ^less@692 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  qcxnkrd	      ^pmix@6.1.0~munge~python build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  cj2ohvh	      ^prrte@4.1.0 build_system=autotools patches:=64faa1a,91b28f5 schedulers:=none platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  4p7olya		  ^flex@2.6.3+lex~nls build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0

With unify: false, Spack concretizes each root spec independently and merges the results, which is exactly what a software stack needs.

Note

If the stack is expected to have only a few duplicate nodes, when_possible is a less aggressive alternative:

override:
  concretizer:
    unify: when_possible

With this option Spack tries to unify the group in an eager, multi-round process. The concretization at round n contains all the specs that could not be unified at round n-1, and considers all the specs from previous rounds for reuse.

Spec matrices

Let’s further expand our stack to link against different LAPACK providers as well. Adding them explicitly would be tedious, so we use a matrix to express the cross-product compactly:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  specs:
  - group: compiler
    specs:
    - gcc@16
    - lmod@8.7.67
    override:
      packages:
        c:
          prefer: [gcc@15]
        cxx:
          prefer: [gcc@15]
        fortran:
          prefer: [gcc@15]
  - group: stack
    needs: [compiler]
    specs:
    - matrix:
      - [netlib-scalapack]
      - [^openmpi, ^mpich]
      - [^openblas, ^netlib-lapack]
    override:
      packages:
        c:
          prefer: [gcc@16]
        cxx:
          prefer: [gcc@16]
        fortran:
          prefer: [gcc@16]
      concretizer:
        unify: false
  view: false

A matrix expands to the cross-product of its rows. This matrix:

- matrix:
  - [netlib-scalapack]
  - [^openmpi, ^mpich]
  - [^openblas, ^netlib-lapack]

is equivalent to this explicit list of specs:

- "netlib-scalapack ^openblas ^openmpi"
- "netlib-scalapack ^openblas ^mpich"
- "netlib-scalapack ^netlib-lapack ^openmpi"
- "netlib-scalapack ^netlib-lapack ^mpich"

No compiler annotation appears in the matrix rows: the override block in the stack group handles that. Let’s concretize to verify our four variants:

$ spack concretize
==> Concretizing the 'stack' group of specs
==> Starting concretization pool with 2 processes
==> Concretized 2 specs:
[b]  v7thz7n  netlib-scalapack@2.2.3~ipo~pic+shared build_system=cmake build_type=Release generator=make platform=linux os=ubuntu26.04 target=x86_64_v3 %c,fortran=gcc@16.1.0
[b]  yonurps	  ^cmake@3.31.11~doc+ncurses+ownlibs~qtgui build_system=generic build_type=Release platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  6b5h3dv	      ^curl@8.20.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2+nghttp2 build_system=autotools libs:=shared,static tls:=openssl platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  iiwpnp7		  ^nghttp2@1.67.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  djpmji7		      ^diffutils@3.12 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  yoq4amm		  ^openssl@3.6.1~docs+shared build_system=generic certs=mozilla platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  axeuqg3		      ^ca-certificates-mozilla@2026-03-19 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  iehcspo	      ^ncurses@6.6~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  yd7lgm7	      ^zlib-ng@2.3.3+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[+]  mmywg7x	  ^compiler-wrapper@1.1.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[+]  vlwdxlj	  ^gcc@16.1.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs+profiled+strip build_system=autotools build_type=Release languages:='c,c++,fortran' platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  cuzgmus	      ^binutils@2.46.0~debuginfod+gas~gprofng~headers~interwork+ld~libiberty~lto~nls~pgo+plugins build_system=autotools compress_debug_sections=zlib libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  yvl6jpi		  ^pkgconf@2.5.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  ao2onuz	      ^diffutils@3.12 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  vbwvgwx		  ^libiconv@1.18 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  z7wqeps	      ^gawk@5.4.0~nls build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  k25xiih		  ^readline@8.3 build_system=autotools patches:=21f0a03,72dee13,e273643 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[e]  yjlog5x	      ^gcc@15.2.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs~profiled~strip build_system=autotools build_type=RelWithDebInfo languages:='c,c++,fortran' platform=linux os=ubuntu26.04 target=x86_64_v3
[+]  xm76mt3	      ^gcc-runtime@15.2.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  r4lhaok	      ^gmake@4.4.1~guile build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  xpmsy5x	      ^gmp@6.3.0+cxx build_system=autotools libs:=shared,static patches:=28a6673 platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  jo3eg4r		  ^autoconf@2.72 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  yhkgfai		  ^automake@1.18.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  y6uqrto		  ^m4@1.4.21+sigsegv build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  phcmfqk		      ^libsigsegv@2.15 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  3gdq456	      ^libtool@2.5.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  mguwetc		  ^file@5.46+static build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  hhjyyqy		      ^xz@5.8.3~pic build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  punbqrx		  ^findutils@4.10.0 build_system=autotools patches:=440b954 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  gxmoxea	      ^mpc@1.4.1 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  melg7ga	      ^mpfr@4.2.2 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  sf5iutv		  ^autoconf-archive@2024.10.16 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  bvphs3b	      ^perl@5.42.0+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  27d4iyp		  ^berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  cbtgjrh		  ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  cq4dkuc		  ^gdbm@1.26 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  ohmdb2l		  ^less@692 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  jvlaabk	      ^texinfo@7.2~xs build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  sle3ix4		  ^gettext@1.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  ujlg2ua		      ^libxml2@2.15.3+pic~python+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  f5xe4px		      ^tar@1.35 build_system=autotools zip=pigz platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  afklka7			  ^pigz@2.8 build_system=makefile platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  ekvivpv		  ^ncurses@6.6~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  g72d7i3	      ^zlib-ng@2.3.3+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  63aruxk	      ^zstd@1.5.7+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  ghfita3	  ^gcc-runtime@16.1.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[e]  yc4n2pp	  ^glibc@2.43 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  6hzfsbp	  ^gmake@4.4.1~guile build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  bovk7dt	  ^mpich@5.0.1~argobots~cuda+fortran+hwloc+hydra~level_zero+libxml2+pci~rocm+romio~slurm~vci~verbs+wrapperrpath~xpmem build_system=autotools datatype-engine=auto device=ch4 netmod=ofi pmi=default platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx,fortran=gcc@16.1.0
[b]  l3if3dy	      ^findutils@4.10.0 build_system=autotools patches:=440b954 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  g3gt6ze		  ^gettext@1.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  tw6253j		      ^tar@1.35 build_system=autotools zip=pigz platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  4diropt			  ^pigz@2.8 build_system=makefile platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  nn3ymay	      ^hwloc@2.13.0~cairo~cuda~gl~level_zero~libudev+libxml2~nvml~opencl+pci~rocm build_system=autotools libs:=shared,static patches:=b4db98b platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  s6cemsy	      ^libfabric@2.5.1~asan~cuda~debug~kdreg~level_zero~lsan~rocm~tsan~ubsan~uring build_system=autotools fabrics:=sockets,tcp,udp platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  cmgnxt6	      ^libpciaccess@0.17 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  cy6lybd		  ^util-macros@1.20.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  cxesrxa	      ^libxml2@2.15.3+pic~python+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  qs6av53		  ^libiconv@1.18 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  bf6e7x2		  ^xz@5.8.3~pic build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  xp6fyih	      ^pkgconf@2.5.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  afnel7p	      ^python@3.14.5+bz2+ctypes+dbm~debug~freethreading+libxml2+lzma~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~static~tests~tkinter+uuid+zlib+zstd build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  ofahxlj		  ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  xkvh6gd		  ^expat@2.8.1+libbsd build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  t623nqz		      ^libbsd@0.12.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  w3w22wx			  ^libmd@1.1.0 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  2af63my		  ^gdbm@1.26 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  jgqwv65		  ^libffi@3.5.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  cbnh7bw		  ^readline@8.3 build_system=autotools patches:=21f0a03,72dee13,e273643 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  uliuxbf		  ^sqlite@3.53.1+column_metadata+fts+rtree build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  v3sdaph		  ^util-linux-uuid@2.41 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  lq6tsmz		  ^zstd@1.5.7+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  zknpbz3	  ^netlib-lapack@3.12.1~external-blas~ipo+lapacke+pic+shared~xblas build_system=cmake build_type=Release generator=make patches:=3059ebf,b1af8b6,e318340 platform=linux os=ubuntu26.04 target=x86_64_v3 %c,fortran=gcc@16.1.0
[b]  2zi5wtu  netlib-scalapack@2.2.3~ipo~pic+shared build_system=cmake build_type=Release generator=make platform=linux os=ubuntu26.04 target=x86_64_v3 %c,fortran=gcc@16.1.0
[b]  nzr6pd7	  ^openmpi@5.0.10+atomics~cuda~debug+fortran~gpfs~internal-hwloc~internal-libevent~internal-pmix~ipv6~java~lustre~memchecker~openshmem~rocm~romio+rsh~static~two_level_namespace+vt+wrapper-rpath build_system=autotools fabrics:=none patches:=646eb1a romio-filesystem:=none schedulers:=none platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx,fortran=gcc@16.1.0
[b]  yg7nv7p	      ^autoconf@2.72 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  47vgtt5		  ^m4@1.4.21+sigsegv build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  r6vd65h		      ^libsigsegv@2.15 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  pdslnit	      ^automake@1.18.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  m5ts755	      ^libevent@2.1.12+openssl build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  raf5qk6	      ^libtool@2.5.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  fc5ezqn		  ^file@5.46+static build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  swhw6f5	      ^numactl@2.0.19 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  kjx2nfr	      ^openssh@10.3p1+gssapi build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  awic2yh		  ^krb5@1.22.2+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  yws52fh		      ^bison@3.8.2~color build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  4c4uuub		  ^libedit@3.1-20251016 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  44jhzjg		  ^libxcrypt@4.5.2~obsolete_api build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  xxg272q	      ^perl@5.42.0+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  aev2xm7		  ^berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  3w5zzn3		  ^less@692 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  qcxnkrd	      ^pmix@6.1.0~munge~python build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  cj2ohvh	      ^prrte@4.1.0 build_system=autotools patches:=64faa1a,91b28f5 schedulers:=none platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  4p7olya		  ^flex@2.6.3+lex~nls build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0

We now have four variations of netlib-scalapack ready to install.

Reusable definitions

So far each matrix row has been written out inline. When multiple matrices share rows, duplicating them becomes a maintenance burden. Spack allows defining lists of constraints under the definitions attribute and reusing them across matrices. Let’s rewrite our manifest using definitions:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  definitions:
  - mpis: [openmpi, mpich]
  - lapacks: [openblas, netlib-lapack]
  - mpi_packages: [netlib-scalapack]

  specs:
  - group: compiler
    specs:
    - gcc@16
    - lmod@8.7.67
    override:
      packages:
        c:
          prefer: [gcc@15]
        cxx:
          prefer: [gcc@15]
        fortran:
          prefer: [gcc@15]
  - group: stack
    needs: [compiler]
    specs:
    - matrix:
      - [$mpi_packages]
      - [$^mpis]
      - [$^lapacks]
    override:
      packages:
        c:
          prefer: [gcc@16]
        cxx:
          prefer: [gcc@16]
        fortran:
          prefer: [gcc@16]
      concretizer:
        unify: false
  view: false

Concretizing confirms the result is identical:

$ spack concretize
==> No new specs to concretize.

Now that we have definitions, we can add a second matrix for serial packages that link against LAPACK but not MPI. In a real deployment this list might include many packages, but here we use just py-scipy as an example. Not every combination is meaningful though: py-scipy ^netlib-lapack is not a useful build, so we use exclude to drop that specific entry from the cross-product:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  definitions:
  - mpis: [openmpi, mpich]
  - lapacks: [openblas, netlib-lapack]
  - mpi_packages: [netlib-scalapack]
  - serial_packages: [py-scipy]

  specs:
  - group: compiler
    specs:
    - gcc@16
    - lmod@8.7.67
    override:
      packages:
        c:
          prefer: [gcc@15]
        cxx:
          prefer: [gcc@15]
        fortran:
          prefer: [gcc@15]
  - group: stack
    needs: [compiler]
    specs:
    - matrix:
      - [$mpi_packages]
      - [$^mpis]
      - [$^lapacks]
    - matrix:
      - [$serial_packages]
      - [$^lapacks]
      exclude:
      - "py-scipy ^netlib-lapack"
    override:
      packages:
        c:
          prefer: [gcc@16]
        cxx:
          prefer: [gcc@16]
        fortran:
          prefer: [gcc@16]
      concretizer:
        unify: false
  view: false

Concretize the environment again:

$ spack concretize
==> Concretizing the 'stack' group of specs
==> Starting concretization
==> Concretized 1 spec:
[b]  gouabck  py-scipy@1.17.1 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx,fortran=gcc@16.1.0
[+]  mmywg7x	  ^compiler-wrapper@1.1.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[+]  vlwdxlj	  ^gcc@16.1.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs+profiled+strip build_system=autotools build_type=Release languages:='c,c++,fortran' platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  cuzgmus	      ^binutils@2.46.0~debuginfod+gas~gprofng~headers~interwork+ld~libiberty~lto~nls~pgo+plugins build_system=autotools compress_debug_sections=zlib libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  yvl6jpi		  ^pkgconf@2.5.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  ao2onuz	      ^diffutils@3.12 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  vbwvgwx		  ^libiconv@1.18 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  z7wqeps	      ^gawk@5.4.0~nls build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  k25xiih		  ^readline@8.3 build_system=autotools patches:=21f0a03,72dee13,e273643 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[e]  yjlog5x	      ^gcc@15.2.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs~profiled~strip build_system=autotools build_type=RelWithDebInfo languages:='c,c++,fortran' platform=linux os=ubuntu26.04 target=x86_64_v3
[+]  xm76mt3	      ^gcc-runtime@15.2.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  r4lhaok	      ^gmake@4.4.1~guile build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  xpmsy5x	      ^gmp@6.3.0+cxx build_system=autotools libs:=shared,static patches:=28a6673 platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  jo3eg4r		  ^autoconf@2.72 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  yhkgfai		  ^automake@1.18.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  y6uqrto		  ^m4@1.4.21+sigsegv build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  phcmfqk		      ^libsigsegv@2.15 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  3gdq456	      ^libtool@2.5.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  mguwetc		  ^file@5.46+static build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  hhjyyqy		      ^xz@5.8.3~pic build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  punbqrx		  ^findutils@4.10.0 build_system=autotools patches:=440b954 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  gxmoxea	      ^mpc@1.4.1 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  melg7ga	      ^mpfr@4.2.2 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  sf5iutv		  ^autoconf-archive@2024.10.16 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  bvphs3b	      ^perl@5.42.0+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  27d4iyp		  ^berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  cbtgjrh		  ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  cq4dkuc		  ^gdbm@1.26 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  ohmdb2l		  ^less@692 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[b]  jvlaabk	      ^texinfo@7.2~xs build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  sle3ix4		  ^gettext@1.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  ujlg2ua		      ^libxml2@2.15.3+pic~python+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  f5xe4px		      ^tar@1.35 build_system=autotools zip=pigz platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  afklka7			  ^pigz@2.8 build_system=makefile platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@15.2.0
[+]  ekvivpv		  ^ncurses@6.6~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  g72d7i3	      ^zlib-ng@2.3.3+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[+]  63aruxk	      ^zstd@1.5.7+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@15.2.0
[b]  ghfita3	  ^gcc-runtime@16.1.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[e]  yc4n2pp	  ^glibc@2.43 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  6mel4bl	  ^meson@1.11.1 build_system=python_pip patches:=0f0b1bd platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  yqhdh2x	      ^ninja@1.13.2+re2c build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  rcco4zy		  ^re2c@4.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  ytwnkqf	      ^py-setuptools@82.0.1 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  kdylbqh	  ^openblas@0.3.33~bignuma~consistent_fpcsr+dynamic_dispatch+fortran~ilp64+locking+pic+shared~static build_system=makefile patches:=723ddc1 symbol_suffix=none threads=none platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx,fortran=gcc@16.1.0
[b]  6hzfsbp	      ^gmake@4.4.1~guile build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  xp6fyih	  ^pkgconf@2.5.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  3ggpdjv	  ^py-cython@3.2.4 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  c6oww3b	  ^py-meson-python@0.19.0 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  x55wsqe	      ^py-packaging@26.2 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  hn6pbjh		  ^py-flit-core@3.12.0 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  cuihsxt	      ^py-pyproject-metadata@0.11.0 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  7g2cva3	  ^py-numpy@2.4.6 build_system=python_pip patches:=873745d platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  5p3frq3	  ^py-pip@26.1 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  3iqsk4i	  ^py-pybind11@3.0.2+ipo build_system=cmake build_type=Release generator=ninja platform=linux os=ubuntu26.04 target=x86_64_v3 %cxx=gcc@16.1.0
[b]  yonurps	      ^cmake@3.31.11~doc+ncurses+ownlibs~qtgui build_system=generic build_type=Release platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  6b5h3dv		  ^curl@8.20.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2+nghttp2 build_system=autotools libs:=shared,static tls:=openssl platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  iiwpnp7		      ^nghttp2@1.67.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  5anzcvk	      ^py-scikit-build-core@0.12.2~pyproject build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx,fortran=gcc@16.1.0
[b]  4qnonii		  ^py-hatch-vcs@0.5.0 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  psffea6		      ^py-setuptools-scm@9.2.2+toml build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  loixb6c			  ^git@2.53.0+man+nls+perl+subtree~tcltk build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  yg7nv7p			      ^autoconf@2.72 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  pdslnit			      ^automake@1.18.1 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  6hwe4tz			      ^libidn2@2.3.8 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  iyz34h3				  ^libunistring@1.4.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  raf5qk6			      ^libtool@2.5.4 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  fc5ezqn				  ^file@5.46+static build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  l3if3dy				  ^findutils@4.10.0 build_system=autotools patches:=440b954 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  47vgtt5			      ^m4@1.4.21+sigsegv build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  r6vd65h				  ^libsigsegv@2.15 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  kjx2nfr			      ^openssh@10.3p1+gssapi build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  awic2yh				  ^krb5@1.22.2+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  yws52fh				      ^bison@3.8.2~color build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  4c4uuub				  ^libedit@3.1-20251016 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  44jhzjg				  ^libxcrypt@4.5.2~obsolete_api build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  wttcr2f			      ^pcre2@10.44~jit+multibyte+pic build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  dxgdks3		  ^py-hatchling@1.29.0 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  etdouzz		      ^py-pluggy@1.6.0 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  cj7d3wo		      ^py-trove-classifiers@2026.6.1.19 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  t7ufmfz			  ^py-calver@2025.10.20 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  ble4v64		  ^py-pathspec@1.1.1 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  36lyfl6	  ^py-pythran@0.18.1 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3 %cxx=gcc@16.1.0
[b]  k7zfexy	      ^py-beniget@0.4.2.post1 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  5nakdbl	      ^py-gast@0.6.0 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  kbtru35	      ^py-ply@3.11 build_system=python_pip platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  yygojb3	  ^py-wheel@0.45.1 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  afnel7p	  ^python@3.14.5+bz2+ctypes+dbm~debug~freethreading+libxml2+lzma~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~static~tests~tkinter+uuid+zlib+zstd build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  ofahxlj	      ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  djpmji7		  ^diffutils@3.12 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  xkvh6gd	      ^expat@2.8.1+libbsd build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  t623nqz		  ^libbsd@0.12.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  w3w22wx		      ^libmd@1.1.0 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  2af63my	      ^gdbm@1.26 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  g3gt6ze	      ^gettext@1.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  qs6av53		  ^libiconv@1.18 build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  cxesrxa		  ^libxml2@2.15.3+pic~python+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  tw6253j		  ^tar@1.35 build_system=autotools zip=pigz platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  4diropt		      ^pigz@2.8 build_system=makefile platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  jgqwv65	      ^libffi@3.5.2 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  iehcspo	      ^ncurses@6.6~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  yoq4amm	      ^openssl@3.6.1~docs+shared build_system=generic certs=mozilla platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  axeuqg3		  ^ca-certificates-mozilla@2026-03-19 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3
[b]  xxg272q		  ^perl@5.42.0+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  aev2xm7		      ^berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  3w5zzn3		      ^less@692 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  cbnh7bw	      ^readline@8.3 build_system=autotools patches:=21f0a03,72dee13,e273643 platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  uliuxbf	      ^sqlite@3.53.1+column_metadata+fts+rtree build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  v3sdaph	      ^util-linux-uuid@2.41 build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  bf6e7x2	      ^xz@5.8.3~pic build_system=autotools libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c=gcc@16.1.0
[b]  yd7lgm7	      ^zlib-ng@2.3.3+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  lq6tsmz	      ^zstd@1.5.7+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=ubuntu26.04 target=x86_64_v3 %c,cxx=gcc@16.1.0
[b]  bcclnnz	  ^python-venv@1.0 build_system=generic platform=linux os=ubuntu26.04 target=x86_64_v3

The exclusion worked: the environment contains only py-scipy ^openblas.

Conditional definitions

A common need in multi-site or multi-cluster deployments is enabling different software sets based on environment variables or host properties. Definitions support a when clause for exactly this. when is a Python expression evaluated at concretization time, with access to the following variables:

variable name

value

platform

The spack platform name for this machine

os

The default spack os name and version string for this machine

target

The default spack target string for this machine

architecture

The default spack architecture string platform-os-target for this machine

arch

Alias for architecture

env

A dictionary representing the user’s environment variables

re

The python re module for regex

hostname

The hostname of this node

Suppose we want to build with mpich by default, and add openmpi when a site-specific environment variable is set. We can express this with two definitions of mpis, the second guarded by a when clause:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  definitions:
  - mpis: [mpich]
  - mpis: [openmpi]
    when: 'env.get("SPACK_STACK_USE_OPENMPI", "") == "1"'
  - lapacks: [openblas, netlib-lapack]
  - mpi_packages: [netlib-scalapack]
  - serial_packages: [py-scipy]

  specs:
  - group: compiler
    specs:
    - gcc@16
    - lmod@8.7.67
    override:
      packages:
        c:
          prefer: [gcc@15]
        cxx:
          prefer: [gcc@15]
        fortran:
          prefer: [gcc@15]
  - group: stack
    needs: [compiler]
    specs:
    - matrix:
      - [$mpi_packages]
      - [$^mpis]
      - [$^lapacks]
    - matrix:
      - [$serial_packages]
      - [$^lapacks]
      exclude:
      - "py-scipy ^netlib-lapack"
    override:
      packages:
        c:
          prefer: [gcc@16]
        cxx:
          prefer: [gcc@16]
        fortran:
          prefer: [gcc@16]
      concretizer:
        unify: false
  view: false

When multiple definitions share the same name, they are concatenated, so the conditional entry simply appends openmpi to the list when the variable is set.

First, concretize without the variable:

$ spack concretize
==> No new specs to concretize.
$ spack find -cl netlib-scalapack
==> In environment /home/spack/stacks (5 root specs)
-- root specs / compiler ----------------------------------------
[+] vlwdxlj gcc@16  [+] wf22jsg lmod@8.7.67

-- root specs / stack -------------------------------------------
[b] v7thz7n netlib-scalapack  [b] x425zhl netlib-scalapack  [b] gouabck py-scipy

-- linux-ubuntu26.04-x86_64_v3 / %c,fortran=gcc@16.1.0 ----------
[b]  v7thz7n netlib-scalapack@2.2.3  [b]  x425zhl netlib-scalapack@2.2.3
==> 0 installed packages
==> 2 concretized packages to be installed

Only mpich is used. Now set the variable and re-concretize:

$ export SPACK_STACK_USE_OPENMPI=1
$ spack concretize
$ spack find -cl netlib-scalapack
==> In environment /home/spack/stacks (7 root specs)
-- root specs / compiler ----------------------------------------
[+] vlwdxlj gcc@16  [+] wf22jsg lmod@8.7.67

-- root specs / stack -------------------------------------------
[b] v7thz7n netlib-scalapack  [b] 2zi5wtu netlib-scalapack  [b] gouabck py-scipy
[b] x425zhl netlib-scalapack  [b] b4nkwcf netlib-scalapack

-- linux-ubuntu26.04-x86_64_v3 / %c,fortran=gcc@16.1.0 ----------
[b]  v7thz7n netlib-scalapack@2.2.3  [b]  2zi5wtu netlib-scalapack@2.2.3
[b]  x425zhl netlib-scalapack@2.2.3  [b]  b4nkwcf netlib-scalapack@2.2.3
==> 0 installed packages
==> 4 concretized packages to be installed

Both MPI providers are now active.

We now have a complete, well-structured environment. It’s time to install it:

$ spack install
[ ] ghfita3 gcc-runtime@16.1.0 fetching from build cache (0s)
[ ] ghfita3 gcc-runtime@16.1.0 relocating (0s)
[+] ghfita3 gcc-runtime@16.1.0 /home/spack/spack/opt/spack/linux-x86_64_v3/gcc-runtime-16.1.0-ghfita36diqcbps7me6eh5im5crijl3g (0s)
[ ] ofahxlj bzip2@1.0.8 fetching from build cache (0s)
[ ] ofahxlj bzip2@1.0.8 relocating (0s)
[ ] kdylbqh openblas@0.3.33 fetching from build cache (0s)
[ ] kdylbqh openblas@0.3.33 relocating (0s)
[ ] 44jhzjg libxcrypt@4.5.2 fetching from build cache (0s)
[ ] 44jhzjg libxcrypt@4.5.2 relocating (0s)
[ ] qs6av53 libiconv@1.18 fetching from build cache (0s)
[ ] qs6av53 libiconv@1.18 relocating (0s)
[ ] cmgnxt6 libpciaccess@0.17 fetching from build cache (0s)
[ ] cmgnxt6 libpciaccess@0.17 relocating (0s)
[ ] zknpbz3 netlib-lapack@3.12.1 fetching from build cache (0s)
[ ] zknpbz3 netlib-lapack@3.12.1 relocating (0s)
[ ] swhw6f5 numactl@2.0.19 fetching from build cache (0s)
[ ] swhw6f5 numactl@2.0.19 relocating (0s)
[ ] jgqwv65 libffi@3.5.2 fetching from build cache (0s)
[ ] jgqwv65 libffi@3.5.2 relocating (0s)
[ ] yd7lgm7 zlib-ng@2.3.3 fetching from build cache (0s)
[ ] yd7lgm7 zlib-ng@2.3.3 relocating (0s)
[ ] bf6e7x2 xz@5.8.3 fetching from build cache (0s)
[ ] bf6e7x2 xz@5.8.3 relocating (0s)
[ ] w3w22wx libmd@1.1.0 fetching from build cache (0s)
[ ] w3w22wx libmd@1.1.0 relocating (0s)
[ ] v3sdaph util-linux-uuid@2.41 fetching from build cache (0s)
[+] ofahxlj bzip2@1.0.8 /home/spack/spack/opt/spack/linux-x86_64_v3/bzip2-1.0.8-ofahxljg6r4dtlyfgtolxexpmnw4cnkz (0s)
[+] 44jhzjg libxcrypt@4.5.2 /home/spack/spack/opt/spack/linux-x86_64_v3/libxcrypt-4.5.2-44jhzjg42xausczvpn26wr6gyzsh2wnu (0s)
[ ] iehcspo ncurses@6.6 fetching from build cache (0s)
[+] cmgnxt6 libpciaccess@0.17 /home/spack/spack/opt/spack/linux-x86_64_v3/libpciaccess-0.17-cmgnxt6kpobrrgj7k5a4iweh6p2qemnw (0s)
[ ] lq6tsmz zstd@1.5.7 fetching from build cache (0s)
[ ] v3sdaph util-linux-uuid@2.41 relocating (0s)
[+] swhw6f5 numactl@2.0.19 /home/spack/spack/opt/spack/linux-x86_64_v3/numactl-2.0.19-swhw6f5gno25h4ap4h6htedy5thh2k55 (0s)
[+] qs6av53 libiconv@1.18 /home/spack/spack/opt/spack/linux-x86_64_v3/libiconv-1.18-qs6av53jc6wvykpse3q7o7sm2dazibpv (0s)
[ ] s6cemsy libfabric@2.5.1 fetching from build cache (0s)
[ ] iehcspo ncurses@6.6 relocating (0s)
[+] jgqwv65 libffi@3.5.2 /home/spack/spack/opt/spack/linux-x86_64_v3/libffi-3.5.2-jgqwv653kext2ew4mzjg3cbmlflyhdbb (0s)
[ ] lq6tsmz zstd@1.5.7 relocating (0s)
[ ] s6cemsy libfabric@2.5.1 relocating (0s)
[+] yd7lgm7 zlib-ng@2.3.3 /home/spack/spack/opt/spack/linux-x86_64_v3/zlib-ng-2.3.3-yd7lgm73h2wqb74e6pz2sdszwkubvpzd (0s)
[+] w3w22wx libmd@1.1.0 /home/spack/spack/opt/spack/linux-x86_64_v3/libmd-1.1.0-w3w22wxnstajwmg2wmtfm6nayugxqbvo (0s)
[ ] yoq4amm openssl@3.6.1 fetching from build cache (0s)
[+] bf6e7x2 xz@5.8.3 /home/spack/spack/opt/spack/linux-x86_64_v3/xz-5.8.3-bf6e7x2te7rdq6kt43maxjnnntvnkpza (0s)
[ ] 4diropt pigz@2.8 fetching from build cache (0s)
[+] lq6tsmz zstd@1.5.7 /home/spack/spack/opt/spack/linux-x86_64_v3/zstd-1.5.7-lq6tsmzmnrljkb52otacw2o4iqmskxn5 (0s)
[ ] t623nqz libbsd@0.12.2 fetching from build cache (0s)
[+] v3sdaph util-linux-uuid@2.41 /home/spack/spack/opt/spack/linux-x86_64_v3/util-linux-uuid-2.41-v3sdaph4k3onzfo5nbhk6gt3typaycl7 (0s)
[ ] yoq4amm openssl@3.6.1 relocating (0s)
[ ] 4diropt pigz@2.8 relocating (0s)
[+] zknpbz3 netlib-lapack@3.12.1 /home/spack/spack/opt/spack/linux-x86_64_v3/netlib-lapack-3.12.1-zknpbz3xor7rz2jo6ek5p4dba6ac4ujs (0s)
[ ] t623nqz libbsd@0.12.2 relocating (0s)
[ ] cxesrxa libxml2@2.15.3 fetching from build cache (0s)
[+] s6cemsy libfabric@2.5.1 /home/spack/spack/opt/spack/linux-x86_64_v3/libfabric-2.5.1-s6cemsy3l7cfqnetr3sjcrtcsln4quey (0s)
[ ] cxesrxa libxml2@2.15.3 relocating (0s)
[+] 4diropt pigz@2.8 /home/spack/spack/opt/spack/linux-x86_64_v3/pigz-2.8-4diroptd5kjfnmxyjlgutr4sh7m53ctf (0s)
[+] kdylbqh openblas@0.3.33 /home/spack/spack/opt/spack/linux-x86_64_v3/openblas-0.3.33-kdylbqhaa64fb44w6hbaoj4eibakcc6u (0s)
[ ] tw6253j tar@1.35 fetching from build cache (0s)
[+] t623nqz libbsd@0.12.2 /home/spack/spack/opt/spack/linux-x86_64_v3/libbsd-0.12.2-t623nqzi6s364pdvgvf7ooygtsagc7ue (0s)
[ ] tw6253j tar@1.35 relocating (0s)
[+] cxesrxa libxml2@2.15.3 /home/spack/spack/opt/spack/linux-x86_64_v3/libxml2-2.15.3-cxesrxalf2jcma3g5dew6ovwxyqqgdqn (0s)
[+] yoq4amm openssl@3.6.1 /home/spack/spack/opt/spack/linux-x86_64_v3/openssl-3.6.1-yoq4ammrtbfxojt6fms67bekd7a5izst (0s)
[ ] xkvh6gd expat@2.8.1 fetching from build cache (0s)
[ ] xkvh6gd expat@2.8.1 relocating (0s)
[ ] m5ts755 libevent@2.1.12 fetching from build cache (0s)
[+] tw6253j tar@1.35 /home/spack/spack/opt/spack/linux-x86_64_v3/tar-1.35-tw6253j6xqn7udhufev2afnhjepbunqh (0s)
[ ] m5ts755 libevent@2.1.12 relocating (0s)
[+] xkvh6gd expat@2.8.1 /home/spack/spack/opt/spack/linux-x86_64_v3/expat-2.8.1-xkvh6gdnqgwccukuhsyvfry63qseotlz (0s)
[+] iehcspo ncurses@6.6 /home/spack/spack/opt/spack/linux-x86_64_v3/ncurses-6.6-iehcspoirzcldubv7budljlbeiooqvpa (0s)
[ ] 4c4uuub libedit@3.1-20251016 fetching from build cache (0s)
[+] m5ts755 libevent@2.1.12 /home/spack/spack/opt/spack/linux-x86_64_v3/libevent-2.1.12-m5ts755lehs4a7r5psjeh5k6pjptap4v (0s)
[ ] g3gt6ze gettext@1.0 fetching from build cache (0s)
[ ] 4c4uuub libedit@3.1-20251016 relocating (0s)
[ ] cbnh7bw readline@8.3 fetching from build cache (0s)
[ ] nn3ymay hwloc@2.13.0 fetching from build cache (1s)
[ ] cbnh7bw readline@8.3 relocating (1s)
[ ] g3gt6ze gettext@1.0 relocating (1s)
[ ] nn3ymay hwloc@2.13.0 relocating (1s)
[+] 4c4uuub libedit@3.1-20251016 /home/spack/spack/opt/spack/linux-x86_64_v3/libedit-3.1-20251016-4c4uuubtih3trasplcywhk2ena3knzjk (1s)
[+] cbnh7bw readline@8.3 /home/spack/spack/opt/spack/linux-x86_64_v3/readline-8.3-cbnh7bwdfvbzqpoww77ltp2s54nizaxx (1s)
[ ] uliuxbf sqlite@3.53.1 fetching from build cache (0s)
[ ] 2af63my gdbm@1.26 fetching from build cache (0s)
[ ] uliuxbf sqlite@3.53.1 relocating (0s)
[ ] 2af63my gdbm@1.26 relocating (0s)
[+] nn3ymay hwloc@2.13.0 /home/spack/spack/opt/spack/linux-x86_64_v3/hwloc-2.13.0-nn3ymaytmizysjb5hfbyzn7migudvklf (1s)
[ ] bovk7dt mpich@5.0.1 fetching from build cache (0s)
[ ] qcxnkrd pmix@6.1.0 fetching from build cache (0s)
[+] 2af63my gdbm@1.26 /home/spack/spack/opt/spack/linux-x86_64_v3/gdbm-1.26-2af63myyx6xsgd5vuwqhj7ibwmnptibj (0s)
[ ] bovk7dt mpich@5.0.1 relocating (0s)
[ ] qcxnkrd pmix@6.1.0 relocating (0s)
[+] uliuxbf sqlite@3.53.1 /home/spack/spack/opt/spack/linux-x86_64_v3/sqlite-3.53.1-uliuxbfvj27y6podjxir7af7frdjuord (0s)
[+] qcxnkrd pmix@6.1.0 /home/spack/spack/opt/spack/linux-x86_64_v3/pmix-6.1.0-qcxnkrd6rle6hjxvjeevshakgvxxsi7g (0s)
[+] g3gt6ze gettext@1.0 /home/spack/spack/opt/spack/linux-x86_64_v3/gettext-1.0-g3gt6ze7gojves2utbt43mddfmx3evv3 (1s)
[ ] cj2ohvh prrte@4.1.0 fetching from build cache (0s)
[ ] awic2yh krb5@1.22.2 fetching from build cache (0s)
[ ] cj2ohvh prrte@4.1.0 relocating (0s)
[ ] afnel7p python@3.14.5 fetching from build cache (0s)
[ ] awic2yh krb5@1.22.2 relocating (0s)
[ ] afnel7p python@3.14.5 relocating (0s)
[+] awic2yh krb5@1.22.2 /home/spack/spack/opt/spack/linux-x86_64_v3/krb5-1.22.2-awic2yh3c7mlbqbamsoxcwrciv4h6utb (0s)
[+] cj2ohvh prrte@4.1.0 /home/spack/spack/opt/spack/linux-x86_64_v3/prrte-4.1.0-cj2ohvhvhiawcijsrkgs5nldcxi42hfd (0s)
[ ] kjx2nfr openssh@10.3p1 fetching from build cache (0s)
[ ] kjx2nfr openssh@10.3p1 relocating (0s)
[+] kjx2nfr openssh@10.3p1 /home/spack/spack/opt/spack/linux-x86_64_v3/openssh-10.3p1-kjx2nfrhrvxzutcvg3zyxjv7tvyqiat2 (0s)
[ ] nzr6pd7 openmpi@5.0.10 fetching from build cache (0s)
[+] bovk7dt mpich@5.0.1 /home/spack/spack/opt/spack/linux-x86_64_v3/mpich-5.0.1-bovk7dtifzhj3qyej4pmqssvi4oda2k4 (0s)
[ ] nzr6pd7 openmpi@5.0.10 relocating (0s)
[ ] x425zhl netlib-scalapack@2.2.3 fetching from build cache (0s)
[ ] v7thz7n netlib-scalapack@2.2.3 fetching from build cache (0s)
[ ] x425zhl netlib-scalapack@2.2.3 relocating (0s)
[ ] v7thz7n netlib-scalapack@2.2.3 relocating (0s)
[+] x425zhl netlib-scalapack@2.2.3 /home/spack/spack/opt/spack/linux-x86_64_v3/netlib-scalapack-2.2.3-x425zhlxzxtbvtol4voqnfnwkwi7i7jm (1s)
[+] v7thz7n netlib-scalapack@2.2.3 /home/spack/spack/opt/spack/linux-x86_64_v3/netlib-scalapack-2.2.3-v7thz7nq3c2kfw4rbqhsulbd3yprhrap (1s)
[+] afnel7p python@3.14.5 /home/spack/spack/opt/spack/linux-x86_64_v3/python-3.14.5-afnel7povb4oermf4jcb5pchgjqkmmux (1s)
[ ] bcclnnz python-venv@1.0 fetching from build cache (0s)
[+] nzr6pd7 openmpi@5.0.10 /home/spack/spack/opt/spack/linux-x86_64_v3/openmpi-5.0.10-nzr6pd7wjjwsa7eltwui6dzmv7klrucn (1s)
[ ] bcclnnz python-venv@1.0 relocating (0s)
[ ] b4nkwcf netlib-scalapack@2.2.3 fetching from build cache (0s)
[ ] 2zi5wtu netlib-scalapack@2.2.3 fetching from build cache (0s)
[ ] b4nkwcf netlib-scalapack@2.2.3 relocating (0s)
[ ] 2zi5wtu netlib-scalapack@2.2.3 relocating (0s)
[+] bcclnnz python-venv@1.0 /home/spack/spack/opt/spack/linux-x86_64_v3/python-venv-1.0-bcclnnzhc3vordjcpw474p7acng2prlx (0s)
[ ] 7g2cva3 py-numpy@2.4.6 fetching from build cache (0s)
[ ] 3iqsk4i py-pybind11@3.0.2 fetching from build cache (0s)
[ ] 7g2cva3 py-numpy@2.4.6 relocating (0s)
[ ] 3iqsk4i py-pybind11@3.0.2 relocating (0s)
[+] b4nkwcf netlib-scalapack@2.2.3 /home/spack/spack/opt/spack/linux-x86_64_v3/netlib-scalapack-2.2.3-b4nkwcfstoe3im2ia3x4sybstfpwfprz (0s)
[+] 2zi5wtu netlib-scalapack@2.2.3 /home/spack/spack/opt/spack/linux-x86_64_v3/netlib-scalapack-2.2.3-2zi5wtuh2jnzfsfjctcvwu7bxugildzo (0s)
[+] 3iqsk4i py-pybind11@3.0.2 /home/spack/spack/opt/spack/linux-x86_64_v3/py-pybind11-3.0.2-3iqsk4i75azrkcftvwmnyp6hezl35dl3 (0s)
[+] 7g2cva3 py-numpy@2.4.6 /home/spack/spack/opt/spack/linux-x86_64_v3/py-numpy-2.4.6-7g2cva3wwgq2nffqpvwujvc2rbbk57hp (0s)
[ ] gouabck py-scipy@1.17.1 fetching from build cache (0s)
[ ] gouabck py-scipy@1.17.1 relocating (0s)
[+] gouabck py-scipy@1.17.1 /home/spack/spack/opt/spack/linux-x86_64_v3/py-scipy-1.17.1-gouabckeot5bo6tkkjiz5o5grjhbhptd (1s)

Other useful features

Before moving to views and modules, two other features are worth knowing about.

The first is source mirrors. When the environment is active, creating one is as simple as:

$ spack mirror create --all -d ./stacks-mirror

This fetches all the tarballs listed in spack.lock into the given directory. You can then move the mirror to an air-gapped machine and register it with:

$ spack mirror add <name> <stacks-mirror>

This lets you rebuild the specs from source on the target machine. Alternatively, to create a buildcache you can:

$ spack gpg create <name> <e-mail>
$ spack buildcache push ./mirror

Don’t forget to set an appropriate value for the padding of the install tree; see how to set up relocation in our documentation.

Multiple Filesystem Views

Views give users access to the installed software through standard filesystem paths, without requiring them to interact with Spack directly. A stack, however, contains multiple configurations of the same package, which would conflict in a single merged directory tree. Instead, we define multiple views, each covering a curated subset of the stack, with optional projections that control the directory layout.

Let’s add view descriptors to our spack.yaml:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  definitions:
  - mpis: [mpich]
  - mpis: [openmpi]
    when: 'env.get("SPACK_STACK_USE_OPENMPI", "") == "1"'
  - lapacks: [openblas, netlib-lapack]
  - mpi_packages: [netlib-scalapack]
  - serial_packages: [py-scipy]

  specs:
  - group: compiler
    specs:
    - gcc@16
    - lmod@8.7.67
    override:
      packages:
        c:
          prefer: [gcc@15]
        cxx:
          prefer: [gcc@15]
        fortran:
          prefer: [gcc@15]
  - group: stack
    needs: [compiler]
    specs:
    - matrix:
      - [$mpi_packages]
      - [$^mpis]
      - [$^lapacks]
    - matrix:
      - [$serial_packages]
      - [$^lapacks]
      exclude:
      - "py-scipy ^netlib-lapack"
    override:
      packages:
        c:
          prefer: [gcc@16]
        cxx:
          prefer: [gcc@16]
        fortran:
          prefer: [gcc@16]
      concretizer:
        unify: false
  view:
    default:
      root: views/default
      select: ['%gcc@16']
      exclude: [^mpich, ^netlib-lapack]
    full:
      root: views/full
      projections:
        ^mpi^lapack: '{name}/{name}-{version}-{^c.name}-{^c.version}-{^mpi.name}-{^mpi.version}-{^lapack.name}-{^lapack.version}'
        ^lapack: '{name}/{name}-{version}-{^c.name}-{^c.version}-{^lapack.name}-{^lapack.version}'
        '%c': '{name}/{name}-{version}-{^c.name}-{^c.version}'
        '%cxx': '{name}/{name}-{version}-{^cxx.name}-{^cxx.version}'
        '%fortran': '{name}/{name}-{version}-{^fortran.name}-{^fortran.version}'
        all: '{name}/{name}-{version}'

We defined two views, default and full. The default view consists of all the packages compiled with gcc@16, excluding those that depend on mpich or netlib-lapack. View descriptors accept both select and exclude constraints.

The full view uses a more complex projection to place each spec into an appropriate subdirectory, according to the first constraint that matches. all is the default projection and always has the lowest priority, regardless of the order in which it appears. To avoid confusion, we advise always keeping it last in projections.

Concretize to regenerate the views and check their structure:

$ spack concretize
==> No new specs to concretize.
==> Updating view at /home/spack/stacks/views/default
==> Updating view at /home/spack/stacks/views/full
$ ls ~/stacks/views/default
bin  etc  include  lib	lib64  libexec	man  pyvenv.cfg	 sbin  share  var
$ ls ~/stacks/views/default/lib
cmake			libevent_core-2.1.so.7.0.1	libgettextpo.a		  libkrb5.so			       libmpich.so	       libpanel.so	       libtinfo_g.a
gettext			libevent_core.a			libgettextpo.so		  libkrb5.so.3			       libmpichcxx.so	       libpanel.so.6	       libtinfow.a
krb5			libevent_core.so		libgettextpo.so.0	  libkrb5.so.3.3		       libmpichf90.so	       libpanel.so.6.6	       libtinfow.so
libasan.so.8		libevent_extra-2.1.so.7		libgettextpo.so.0.6.0	  libkrb5support.so		       libmpicxx.a	       libpanel_g.a	       libtinfow.so.6
libasprintf.a		libevent_extra-2.1.so.7.0.1	libgettextsrc-1.0.so	  libkrb5support.so.0		       libmpicxx.so	       libpanelw.a	       libtinfow.so.6.6
libasprintf.so		libevent_extra.a		libgettextsrc.a		  libkrb5support.so.0.1		       libmpicxx.so.12	       libpanelw.so	       libtinfow_g.a
libasprintf.so.0	libevent_extra.so		libgettextsrc.so	  liblapack.so			       libmpicxx.so.12.6.1     libpanelw.so.6	       libtmglib.so
libasprintf.so.0.0.0	libevent_openssl-2.1.so.7	libgfortran.so.5	  liblapack.so.3		       libmpifort.a	       libpanelw.so.6.6	       libtmglib.so.3
libatomic.so.1		libevent_openssl-2.1.so.7.0.1	libgomp.so.1		  liblapack.so.3.12.0		       libmpifort.so	       libpanelw_g.a	       libtmglib.so.3.12.0
libblas.so		libevent_openssl.a		libgssapi_krb5.so	  liblapacke.so			       libmpifort.so.12	       libpciaccess.a	       libtsan.so.2
libblas.so.3		libevent_openssl.so		libgssapi_krb5.so.2	  liblapacke.so.3		       libmpifort.so.12.6.1    libpciaccess.so	       libubsan.so.1
libblas.so.3.12.0	libevent_pthreads-2.1.so.7	libgssapi_krb5.so.2.2	  liblapacke.so.3.12.0		       libmpl.so	       libpciaccess.so.0       libuuid.a
libbsd-ctor.a		libevent_pthreads-2.1.so.7.0.1	libgssrpc.so		  liblsan.so.0			       libncurses++.a	       libpciaccess.so.0.11.1  libuuid.so
libbsd.a		libevent_pthreads.a		libgssrpc.so.4		  liblzma.a			       libncurses++.so	       libpmix.a	       libuuid.so.1
libbsd.so		libevent_pthreads.so		libgssrpc.so.4.2	  liblzma.so			       libncurses++.so.6       libpmix.so	       libuuid.so.1.3.0
libbsd.so.0		libexpat.a			libhistory.a		  liblzma.so.5			       libncurses++.so.6.6     libpmix.so.2	       libverto.so
libbsd.so.0.12.2	libexpat.so			libhistory.so		  liblzma.so.5.8.3		       libncurses++_g.a	       libpmix.so.2.23.0       libverto.so.0
libbz2.a		libexpat.so.1			libhistory.so.8		  libmd.a			       libncurses++w.a	       libprrte.a	       libverto.so.0.0
libbz2.so		libexpat.so.1.12.1		libhistory.so.8.3	  libmd.so			       libncurses++w.so	       libprrte.so	       libxml2.so
libbz2.so.1		libfabric.a			libhwasan.so.0		  libmd.so.0			       libncurses++w.so.6      libprrte.so.3	       libxml2.so.16
libbz2.so.1.0		libfabric.so			libhwloc.a		  libmd.so.0.1.0		       libncurses++w.so.6.6    libprrte.so.3.1.0       libxml2.so.16.1.3
libbz2.so.1.0.8		libfabric.so.1			libhwloc.so		  libmenu.a			       libncurses++w_g.a       libpython3.14.so	       libz.a
libcblas.so		libfabric.so.1.31.1		libhwloc.so.15		  libmenu.so			       libncurses.a	       libpython3.14.so.1.0    libz.so
libcblas.so.3		libfmpich.so			libhwloc.so.15.10.2	  libmenu.so.6			       libncurses.so	       libpython3.so	       libz.so.1
libcblas.so.3.12.0	libform.a			libiconv.a		  libmenu.so.6.6		       libncurses.so.6	       libquadmath.so.0	       libz.so.1.3.1.zlib-ng
libcharset.a		libform.so			libiconv.so		  libmenu_g.a			       libncurses.so.6.6       libreadline.a	       libzstd.a
libcharset.so		libform.so.6			libiconv.so.2		  libmenuw.a			       libncurses_g.a	       libreadline.so	       libzstd.so
libcharset.so.1		libform.so.6.6			libiconv.so.2.7.0	  libmenuw.so			       libncursesw.a	       libreadline.so.8	       libzstd.so.1
libcharset.so.1.0.0	libform_g.a			libitm.so.1		  libmenuw.so.6			       libncursesw.so	       libreadline.so.8.3      libzstd.so.1.5.7
libcom_err.so		libformw.a			libk5crypto.so		  libmenuw.so.6.6		       libncursesw.so.6	       libscalapack.so	       mpi.mod
libcom_err.so.3		libformw.so			libk5crypto.so.3	  libmenuw_g.a			       libncursesw.so.6.6      libscalapack.so.2.2     mpi_ext.mod
libcom_err.so.3.0	libformw.so.6			libk5crypto.so.3.1	  libmpi.a			       libncursesw_g.a	       libscalapack.so.2.2.3   mpi_f08.mod
libcrypt.a		libformw.so.6.6			libkadm5clnt.so		  libmpi.so			       libnuma.a	       libsqlite3.a	       mpi_f08_callbacks.mod
libcrypt.so		libformw_g.a			libkadm5clnt_mit.so	  libmpi.so.12			       libnuma.so	       libsqlite3.so	       mpi_f08_ext.mod
libcrypt.so.2		libgcc_s.so.1			libkadm5clnt_mit.so.12	  libmpi.so.12.6.1		       libnuma.so.1	       libsqlite3.so.0	       mpi_f08_interfaces.mod
libcrypt.so.2.0.0	libgdbm.a			libkadm5clnt_mit.so.12.0  libmpi.so.40			       libnuma.so.1.0.0	       libsqlite3.so.3.53.1    mpi_f08_interfaces_callbacks.mod
libcurses.so		libgdbm.so			libkadm5srv.so		  libmpi.so.40.40.7		       libopa.so	       libssp.so.0	       mpi_f08_types.mod
libedit.a		libgdbm.so.6			libkadm5srv_mit.so	  libmpi_mpifh.so		       libopen-pal.so	       libstdc++.so.6	       mpi_types.mod
libedit.so		libgdbm.so.6.0.0		libkadm5srv_mit.so.12	  libmpi_mpifh.so.40		       libopen-pal.so.80       libtextstyle.a	       openmpi
libedit.so.0		libgdbm_compat.a		libkadm5srv_mit.so.12.0	  libmpi_mpifh.so.40.40.1	       libopen-pal.so.80.0.5   libtextstyle.so	       pkgconfig
libedit.so.0.0.76	libgdbm_compat.so		libkdb5.so		  libmpi_usempi_ignore_tkr.so	       libopenblas-r0.3.33.a   libtextstyle.so.0       pmpi_f08_interfaces.mod
libevent-2.1.so.7	libgdbm_compat.so.4		libkdb5.so.10		  libmpi_usempi_ignore_tkr.so.40       libopenblas-r0.3.33.so  libtextstyle.so.0.2.7   preloadable_libintl.so
libevent-2.1.so.7.0.1	libgdbm_compat.so.4.0.0		libkdb5.so.10.0		  libmpi_usempi_ignore_tkr.so.40.40.1  libopenblas.a	       libtinfo.a	       python3.14
libevent.a		libgettextlib-1.0.so		libkrad.so		  libmpi_usempif08.so		       libopenblas.so	       libtinfo.so	       terminfo
libevent.so		libgettextlib.a			libkrad.so.0		  libmpi_usempif08.so.40	       libopenblas.so.0	       libtinfo.so.6
libevent_core-2.1.so.7	libgettextlib.so		libkrad.so.0.0		  libmpi_usempif08.so.40.40.3	       libpanel.a	       libtinfo.so.6.6
$ ls ~/stacks/views/full
binutils  expat	       gdbm	hwloc	libedit	   libffi    libpciaccess  lmod	 mpfr	  netlib-lapack	    numactl   openssh  pmix    readline	 tcl		  xz
bzip2	  gcc	       gettext	krb5	libevent   libiconv  libxcrypt	   lua	 mpich	  netlib-scalapack  openblas  openssl  prrte   sqlite	 unzip		  zlib-ng
curl	  gcc-runtime  gmp	libbsd	libfabric  libmd     libxml2	   mpc	 ncurses  nghttp2	    openmpi   pigz     python  tar	 util-linux-uuid  zstd
$ ls ~/stacks/views/full/gcc/
gcc-16.1.0-gcc-15.2.0
$ ls ~/stacks/views/full/gcc/gcc-16.1.0-gcc-15.2.0
bin  include  lib  lib64  libexec  share

By default, a view includes all packages, link and run dependencies included. Setting link: roots restricts it to root specs only:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  definitions:
  - mpis: [mpich]
  - mpis: [openmpi]
    when: 'env.get("SPACK_STACK_USE_OPENMPI", "") == "1"'
  - lapacks: [openblas, netlib-lapack]
  - mpi_packages: [netlib-scalapack]
  - serial_packages: [py-scipy]

  specs:
  - group: compiler
    specs:
    - gcc@16
    - lmod@8.7.67
    override:
      packages:
        c:
          prefer: [gcc@15]
        cxx:
          prefer: [gcc@15]
        fortran:
          prefer: [gcc@15]
  - group: stack
    needs: [compiler]
    specs:
    - matrix:
      - [$mpi_packages]
      - [$^mpis]
      - [$^lapacks]
    - matrix:
      - [$serial_packages]
      - [$^lapacks]
      exclude:
      - "py-scipy ^netlib-lapack"
    override:
      packages:
        c:
          prefer: [gcc@16]
        cxx:
          prefer: [gcc@16]
        fortran:
          prefer: [gcc@16]
      concretizer:
        unify: false
  view:
    default:
      root: views/default
      select: ['%gcc@16']
      exclude: [^mpich, ^netlib-lapack]
      link: roots
    full:
      root: views/full
      projections:
        ^mpi^lapack: '{name}/{name}-{version}-{^c.name}-{^c.version}-{^mpi.name}-{^mpi.version}-{^lapack.name}-{^lapack.version}'
        ^lapack: '{name}/{name}-{version}-{^c.name}-{^c.version}-{^lapack.name}-{^lapack.version}'
        '%c': '{name}/{name}-{version}-{^c.name}-{^c.version}'
        '%cxx': '{name}/{name}-{version}-{^cxx.name}-{^cxx.version}'
        '%fortran': '{name}/{name}-{version}-{^fortran.name}-{^fortran.version}'
        all: '{name}/{name}-{version}'
$ spack concretize
==> No new specs to concretize.
==> Updating view at /home/spack/stacks/views/default
$ ls ~/stacks/views/default
lib
$ ls ~/stacks/views/default/lib
cmake  libscalapack.so	libscalapack.so.2.2  libscalapack.so.2.2.3  pkgconfig  python3.14
$ ls ~/stacks/views/full
binutils  expat	       gdbm	hwloc	libedit	   libffi    libpciaccess  lmod	 mpfr	  netlib-lapack	    numactl   openssh  pmix    readline	 tcl		  xz
bzip2	  gcc	       gettext	krb5	libevent   libiconv  libxcrypt	   lua	 mpich	  netlib-scalapack  openblas  openssl  prrte   sqlite	 unzip		  zlib-ng
curl	  gcc-runtime  gmp	libbsd	libfabric  libmd     libxml2	   mpc	 ncurses  nghttp2	    openmpi   pigz     python  tar	 util-linux-uuid  zstd

Now we see only the root packages in the default view. The rest are hidden, but are still available in the full view. Full documentation on views is available in the Spack environments guide.

Module Files

Module files are the standard mechanism for managing multiple software versions on HPC systems. In this section we’ll show how to configure and generate a hierarchical module structure with lmod.

Note

A more in-depth tutorial, focused only on module files, can be found at Module Files Tutorial. It covers the general architecture and differences between environment-modules and lmod that are out of scope here.

Since lmod is already part of our compiler group and has been installed, we just need to add the module command to our shell:

$ . $(spack location -i lmod)/lmod/lmod/init/bash

You should now have the module command available:

$ module --version

Modules based on Lua: Version 8.7.67 2025-10-04 20:16 -05:00
    by Robert McLay mclay@tacc.utexas.edu

The next step is to add some basic configuration to our spack.yaml to generate module files:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  definitions:
  - mpis: [mpich]
  - mpis: [openmpi]
    when: 'env.get("SPACK_STACK_USE_OPENMPI", "") == "1"'
  - lapacks: [openblas, netlib-lapack]
  - mpi_packages: [netlib-scalapack]
  - serial_packages: [py-scipy]

  specs:
  - group: compiler
    specs:
    - gcc@16
    - lmod@8.7.67
    override:
      packages:
        c:
          prefer: [gcc@15]
        cxx:
          prefer: [gcc@15]
        fortran:
          prefer: [gcc@15]
  - group: stack
    needs: [compiler]
    specs:
    - matrix:
      - [$mpi_packages]
      - [$^mpis]
      - [$^lapacks]
    - matrix:
      - [$serial_packages]
      - [$^lapacks]
      exclude:
      - "py-scipy ^netlib-lapack"
    override:
      packages:
        c:
          prefer: [gcc@16]
        cxx:
          prefer: [gcc@16]
        fortran:
          prefer: [gcc@16]
      concretizer:
        unify: false
  view: false
  modules:
    default:
      enable:
      - lmod
      roots:
        lmod: modules
      lmod:
        hierarchy:
        - mpi
        - lapack
        core_specs:
        - '%gcc@15'

This configuration tells Spack to:

  • generate lmod module files under modules/, with a hierarchy based on mpi and lapack

  • place all specs built with the system compiler (%gcc@15) into the Core designation

We can generate the module files and use them with the following commands:

$ spack module lmod refresh -y
$ module use $PWD/stacks/modules/linux-ubuntu26.04-x86_64/Core

Let’s check the generated module files:

$ module av

----------------------------------------------------------------------- /home/spack/stacks/modules/linux-ubuntu26.04-x86_64/Core -----------------------------------------------------------------------
   binutils/2.46.0-cuzgmus		 gcc/15.2.0-yjlog5x	      libxcrypt/4.5.2-bkzcu2s		 mpc/1.4.1-gxmoxea	   pkgconf/2.5.1-yvl6jpi    zlib-ng/2.3.3-g72d7i3
   bzip2/1.0.8-cbtgjrh			 gcc/16.1.0-vlwdxlj    (D)    libxml2/2.15.3-ujlg2ua		 mpfr/4.2.2-melg7ga	   readline/8.3-k25xiih	    zstd/1.5.7-63aruxk
   compiler-wrapper/1.1.0-mmywg7x	 gettext/1.0-sle3ix4	      lmod/8.7.67-wf22jsg		 ncurses/6.6-ekvivpv	   tar/1.35-f5xe4px
   curl/8.20.0-j2fa7xl			 glibc/2.43-yc4n2pp	      lua-luafilesystem/1.9.0-su5b2hs	 nghttp2/1.67.1-4lyasaz	   tcl/8.6.17-d4ibosb
   gcc-runtime/15.2.0-xm76mt3		 gmp/6.3.0-xpmsy5x	      lua-luaposix/36.3-egzwtik		 openssl/3.6.1-33yozp5	   unzip/6.0-4qgksuy
   gcc-runtime/16.1.0-ghfita3	  (D)	 libiconv/1.18-vbwvgwx	      lua/5.4.8-q66mudl			 pigz/2.8-afklka7	   xz/5.8.3-hhjyyqy

  Where:
   D:  Default Module

If the avail list is too long consider trying:

"module --default avail" or "ml -d av" to just list the default modules.
"module overview" or "ml ov" to display the number of modules for each name.

Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

The set of modules is already usable, and the hierarchy already works. For instance, we can load the gcc compiler and check that gcc is in our path and that many modules are available, all compiled with gcc@16:

$ module load gcc
$ which gcc
/home/spack/spack/opt/spack/linux-x86_64_v3/gcc-16.1.0-vlwdxljzbis42fjrlecvyo4tmuq5eebn/bin/gcc
$ gcc --version
gcc (Spack GCC) 16.1.0
Copyright (C) 2026 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ module av

-------------------------------------------------------------------- /home/spack/stacks/modules/linux-ubuntu26.04-x86_64/gcc/16.1.0 --------------------------------------------------------------------
   bzip2/1.0.8-ofahxlj	 (D)	libedit/3.1-20251016-4c4uuub	libxcrypt/4.5.2-44jhzjg	     (D)    openmpi/5.0.10-nzr6pd7	     python-venv/1.0-bcclnnz		 zlib-ng/2.3.3-yd7lgm7 (D)
   expat/2.8.1-xkvh6gd		libevent/2.1.12-m5ts755		libxml2/2.15.3-cxesrxa		    openssh/10.3p1-kjx2nfr	     python/3.14.5-afnel7p		 zstd/1.5.7-lq6tsmz
   gdbm/1.26-2af63my		libfabric/2.5.1-s6cemsy		mpich/5.0.1-bovk7dt		    openssl/3.6.1-yoq4amm	     readline/8.3-cbnh7bw
   gettext/1.0-g3gt6ze		libffi/3.5.2-jgqwv65		ncurses/6.6-iehcspo	     (D)    pigz/2.8-4diropt	      (D)    sqlite/3.53.1-uliuxbf
   hwloc/2.13.0-nn3ymay		libiconv/1.18-qs6av53		netlib-lapack/3.12.1-zknpbz3	    pmix/6.1.0-qcxnkrd		     tar/1.35-tw6253j		  (D)
   krb5/1.22.2-awic2yh		libmd/1.1.0-w3w22wx		numactl/2.0.19-swhw6f5		    prrte/4.1.0-cj2ohvh		     util-linux-uuid/2.41-v3sdaph
   libbsd/0.12.2-t623nqz	libpciaccess/0.17-cmgnxt6	openblas/0.3.33-kdylbqh		    py-pybind11/3.0.2-3iqsk4i	     xz/5.8.3-bf6e7x2

----------------------------------------------------------------------- /home/spack/stacks/modules/linux-ubuntu26.04-x86_64/Core -----------------------------------------------------------------------
   binutils/2.46.0-cuzgmus	  (L)	 gcc/15.2.0-yjlog5x		libxcrypt/4.5.2-bkzcu2s		       mpc/1.4.1-gxmoxea      (L)    pkgconf/2.5.1-yvl6jpi	  zlib-ng/2.3.3-g72d7i3 (L)
   bzip2/1.0.8-cbtgjrh			 gcc/16.1.0-vlwdxlj    (L,D)	libxml2/2.15.3-ujlg2ua		(D)    mpfr/4.2.2-melg7ga     (L)    readline/8.3-k25xiih  (D)	  zstd/1.5.7-63aruxk	(L,D)
   compiler-wrapper/1.1.0-mmywg7x	 gettext/1.0-sle3ix4   (D)	lmod/8.7.67-wf22jsg		       ncurses/6.6-ekvivpv	     tar/1.35-f5xe4px
   curl/8.20.0-j2fa7xl			 glibc/2.43-yc4n2pp    (L)	lua-luafilesystem/1.9.0-su5b2hs	       nghttp2/1.67.1-4lyasaz	     tcl/8.6.17-d4ibosb
   gcc-runtime/15.2.0-xm76mt3	  (L)	 gmp/6.3.0-xpmsy5x     (L)	lua-luaposix/36.3-egzwtik	       openssl/3.6.1-33yozp5  (D)    unzip/6.0-4qgksuy
   gcc-runtime/16.1.0-ghfita3	  (D)	 libiconv/1.18-vbwvgwx (D)	lua/5.4.8-q66mudl		       pigz/2.8-afklka7		     xz/5.8.3-hhjyyqy	   (D)

  Where:
   D:  Default Module
   L:  Module is loaded

If the avail list is too long consider trying:

"module --default avail" or "ml -d av" to just list the default modules.
"module overview" or "ml ov" to display the number of modules for each name.

Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".


$ module unload gcc

The default configuration has a few rough edges, though:

  • dependency modules from gcc clutter the listing with packages users will never load directly

  • hashes in module names make scripts non-portable across similar environments

  • some modules may need custom environment variables specific to site policy

To address all these needs, we can extend our modules configuration:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  definitions:
  - mpis: [mpich]
  - mpis: [openmpi]
    when: 'env.get("SPACK_STACK_USE_OPENMPI", "") == "1"'
  - lapacks: [openblas, netlib-lapack]
  - mpi_packages: [netlib-scalapack]
  - serial_packages: [py-scipy]

  specs:
  - group: compiler
    specs:
    - gcc@16
    - lmod@8.7.67
    override:
      packages:
        c:
          prefer: [gcc@15]
        cxx:
          prefer: [gcc@15]
        fortran:
          prefer: [gcc@15]
  - group: stack
    needs: [compiler]
    specs:
    - matrix:
      - [$mpi_packages]
      - [$^mpis]
      - [$^lapacks]
    - matrix:
      - [$serial_packages]
      - [$^lapacks]
      exclude:
      - "py-scipy ^netlib-lapack"
    override:
      packages:
        c:
          prefer: [gcc@16]
        cxx:
          prefer: [gcc@16]
        fortran:
          prefer: [gcc@16]
      concretizer:
        unify: false
  view: false
  modules:
    default:
      enable:
      - lmod
      roots:
        lmod: modules
      lmod:
        hierarchy:
        - mpi
        - lapack
        core_specs:
        - '%gcc@15'
        hash_length: 0
        include:
        - gcc
        exclude:
        - '%gcc@15'
        - autoconf
        all:
          environment:
            set:
              '{name}_ROOT': '{prefix}'
        openmpi:
          environment:
            set:
              SLURM_MPI_TYPE: pmi2
              OMPI_MCA_btl_openib_warn_default_gid_prefix: '0'
        projections:
          all: '{name}/{version}'

Regenerate the modules:

$ spack module lmod refresh --delete-tree -y
==> Regenerating lmod module files

Now we have module files without hashes, a correct hierarchy, and our custom environment variables:

$ module load gcc
$ module load openmpi openblas netlib-scalapack py-scipy
$ module av

-------------------------------------------------------------------- /home/spack/stacks/modules/linux-ubuntu26.04-x86_64/gcc/16.1.0 --------------------------------------------------------------------
   bzip2/1.0.8	  hwloc/2.13.0		  libevent/2.1.12    libmd/1.1.0	  mpich/5.0.1		  openblas/0.3.33    pigz/2.8		  python-venv/1.0    tar/1.35		     zstd/1.5.7
   expat/2.8.1	  krb5/1.22.2		  libfabric/2.5.1    libpciaccess/0.17	  ncurses/6.6		  openmpi/5.0.10     pmix/6.1.0		  python/3.14.5	     util-linux-uuid/2.41
   gdbm/1.26	  libbsd/0.12.2		  libffi/3.5.2	     libxcrypt/4.5.2	  netlib-lapack/3.12.1	  openssh/10.3p1     prrte/4.1.0	  readline/8.3	     xz/5.8.3
   gettext/1.0	  libedit/3.1-20251016	  libiconv/1.18	     libxml2/2.15.3	  numactl/2.0.19	  openssl/3.6.1	     py-pybind11/3.0.2	  sqlite/3.53.1	     zlib-ng/2.3.3

----------------------------------------------------------------------- /home/spack/stacks/modules/linux-ubuntu26.04-x86_64/Core -----------------------------------------------------------------------
   compiler-wrapper/1.1.0    gcc-runtime/16.1.0	   gcc/15.2.0	 gcc/16.1.0 (L,D)    glibc/2.43 (L)

  Where:
   D:  Default Module
   L:  Module is loaded

If the avail list is too long consider trying:

"module --default avail" or "ml -d av" to just list the default modules.
"module overview" or "ml ov" to display the number of modules for each name.

Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".


$ module load mpich

Lmod is automatically replacing "openmpi/5.0.10" with "mpich/5.0.1".


Due to MODULEPATH changes, the following have been reloaded:
  1) netlib-scalapack/2.2.3

$ module load netlib-lapack

Lmod is automatically replacing "openblas/0.3.33" with "netlib-lapack/3.12.1".


Due to MODULEPATH changes, the following have been reloaded:
  1) netlib-scalapack/2.2.3

$ module purge

This concludes the stacks tutorial.

Summary

In this tutorial we built a realistic HPC software stack: a cross-product of MPI and LAPACK libraries, compiled with a newer compiler than the system default. Along the way we used:

  • Spec groups with needs and override to control build order and compiler selection per group

  • Spec matrices and definitions to express the cross-product concisely

  • Conditional definitions to make the MPI selection configurable at concretization time

  • Filesystem views and module files to make the software accessible to users