Module Files

In this tutorial, we’ll introduce a few concepts that are fundamental to the generation of module files with Spack, and we’ll guide you through the customization of both module files content and their layout on disk. In the end you should have a clear understanding of:

  • What are module files and how they work

  • How Spack generates them

  • Which commands are available to ease their maintenance

  • How it is possible to customize them in all aspects

Modules at a Glance

Let’s start by summarizing what module files are and how you can use them to modify your environment. The idea is to give enough information so that people without any previous exposure to them will be able to follow the tutorial later on. We’ll also give a high-level view of how module files are generated in Spack. If you are already familiar with these topics you can quickly skim through this section or move directly to Setup for the Tutorial.

What are module files?

Module files are an easy way to modify your environment in a controlled manner during a shell session. In general, they contain the information needed to run an application or use a library, and they work in conjunction with a tool that interprets them. Typical module files instruct this tool to modify the environment variables when a module file is loaded:

$ module show zlib
-------------------------------------------------------------------
/home/mculpo/PycharmProjects/spack/share/spack/modules/linux-ubuntu14.04-x86_64/zlib/1.2.11-gcc-7.2.0-linux-ubuntu14.04-x86_64-co2px3k:

module-whatis       A free, general-purpose, legally unencumbered lossless data-compression library.
prepend-path        MANPATH /home/mculpo/PycharmProjects/spack/opt/spack/linux-ubuntu14.04-x86_64/gcc-7.2.0/zlib-1.2.11-co2px3k53m76lm6tofylh2mur2hnicux/share/man
prepend-path        LIBRARY_PATH /home/mculpo/PycharmProjects/spack/opt/spack/linux-ubuntu14.04-x86_64/gcc-7.2.0/zlib-1.2.11-co2px3k53m76lm6tofylh2mur2hnicux/lib
prepend-path        LD_LIBRARY_PATH /home/mculpo/PycharmProjects/spack/opt/spack/linux-ubuntu14.04-x86_64/gcc-7.2.0/zlib-1.2.11-co2px3k53m76lm6tofylh2mur2hnicux/lib
prepend-path        CPATH /home/mculpo/PycharmProjects/spack/opt/spack/linux-ubuntu14.04-x86_64/gcc-7.2.0/zlib-1.2.11-co2px3k53m76lm6tofylh2mur2hnicux/include
prepend-path        PKG_CONFIG_PATH /home/mculpo/PycharmProjects/spack/opt/spack/linux-ubuntu14.04-x86_64/gcc-7.2.0/zlib-1.2.11-co2px3k53m76lm6tofylh2mur2hnicux/lib/pkgconfig
prepend-path        CMAKE_PREFIX_PATH /home/mculpo/PycharmProjects/spack/opt/spack/linux-ubuntu14.04-x86_64/gcc-7.2.0/zlib-1.2.11-co2px3k53m76lm6tofylh2mur2hnicux/
-------------------------------------------------------------------

$ echo $LD_LIBRARY_PATH

$ module load zlib
$ echo $LD_LIBRARY_PATH
/home/mculpo/PycharmProjects/spack/opt/spack/linux-ubuntu14.04-x86_64/gcc-7.2.0/zlib-1.2.11-co2px3k53m76lm6tofylh2mur2hnicux/lib

and to undo the modifications when the same module file is unloaded:

$ module unload zlib
$ echo $LD_LIBRARY_PATH

$

Different formats exist for module files, and different tools provide various levels of support for them. Spack can natively generate:

  1. Non-hierarchical module files written in TCL

  2. Hierarchical module files written in Lua

and can build environment-modules and lmod as support tools. Which of the formats or tools best suits one’s needs depends on each particular use-case. For the sake of illustration, we’ll be working on both formats using lmod.

See also

Environment modules

This is the original tool that provided modules support. Its first version was coded in C in the early ’90s and was later substituted by a version completely coded in TCL - the one Spack is distributing. More details on its features are given in the homepage of the project or in its github page. The tool is able to interpret the non-hierarchical TCL modulefiles written by Spack.

Lmod

Lmod is a module system written in Lua, designed to easily handle hierarchies of module files. It’s a drop-in replacement of Environment Modules and works with both of the module file formats generated by Spack. Despite being fully compatible with Environment Modules there are many features that are unique to Lmod. These features are either targeted towards safety or meant to extend the module system functionality.

How do we generate module files?

Before we dive into the hands-on sections it’s worth spending a couple of words to explain how module files are generated by Spack. The following diagram provides a high-level view of the process:

_images/module_file_generation.svg

The red dashed line above represents Spack’s boundaries, the blue one Spack’s dependencies 1. Module files are generated by combining:

  • the configuration details in config.yaml and modules.yaml

  • the information contained in Spack packages (and processed by the module subpackage)

  • a set of template files

with Jinja2, an external template engine that stamps out each particular module file. As Spack serves very diverse needs this process has many points of customization, and we’ll explore most of them in the next sections.

1

Spack vendors its dependencies! This means that Spack comes with a copy of each one of its dependencies, including Jinja2, and is already configured to use them.

Setup for the Tutorial

In order to showcase the capabilities of Spack’s module file generation, we need a representative set of software to work with. This set must include different flavors of the same packages installed alongside each other and some external packages.

The purpose of this setup is not to make our life harder but to demonstrate how Spack can help with similar situations, as they will happen on real HPC clusters. For instance, it’s often preferable for Spack to use vendor-provided MPI implementations than to build one itself.

To keep the set of software we’re dealing with manageable, we’re going to uninstall everything from earlier in the tutorial.

Build a module tool

The first thing that we need is the module tool. In this case we choose lmod as it can work with both hierarchical and non-hierarchical module file layouts.

$ spack install lmod

Once the module tool is installed we need to have it available in the current shell. As the installation directories are definitely not easy to remember, we’ll employ the command spack location to retrieve the lmod prefix directly from Spack:

$ . $(spack location -i lmod)/lmod/lmod/init/bash

Now we can re-source the setup file and Spack modules will be put in our module path.

$ . share/spack/setup-env.sh

Add a new compiler

The second step is to build a recent compiler. On first use, Spack scans the environment and automatically locates the compiler(s) already available on the system. For this tutorial, however, we want to use gcc@8.3.0.

$ spack install gcc@8.3.0

Once gcc is installed we can use shell support to load it and make it readily available:

$ spack load gcc@8.3.0

It may not be apparent, but the last command employed the module files generated automatically by Spack. What happens under the hood when you use the spack load command is:

  1. the spec passed as argument is translated into a module file name

  2. the current module tool is used to load that module file

You can use this command to double check:

$ module list

Currently Loaded Modules:
  1) gcc-8.3.0-gcc-7.4.0-rvoysuv

Note that the 7-digit hash at the end of the generated module may vary depending on architecture or package version. Now that we have gcc@8.3.0 in PATH we can finally add it to the list of compilers known to Spack:

$ spack compiler add
==> Added 1 new compiler to /home/spack/.spack/linux/compilers.yaml
    gcc@8.3.0
==> Compilers are defined in the following files:
    /home/spack/.spack/linux/compilers.yaml

$ spack compiler list
==> Available compilers
-- clang ubuntu18.04-x86_64 -------------------------------------
clang@6.0.0

-- gcc ubuntu18.04-x86_64 ---------------------------------------
gcc@8.3.0  gcc@7.4.0  gcc@6.5.0

Build the software that will be used in the tutorial

Finally, we should use Spack to install the packages used in the examples:

$ spack install netlib-scalapack ^openmpi ^openblas
$ spack install netlib-scalapack ^mpich ^openblas
$ spack install netlib-scalapack ^openmpi ^netlib-lapack
$ spack install netlib-scalapack ^mpich ^netlib-lapack
$ spack install py-scipy ^openblas

Non-hierarchical Module Files

If you arrived to this point you should have an environment that looks similar to:

$ module avail

----------------------------------------- /home/spack/spack/share/spack/modules/linux-ubuntu18.04-x86_64 -----------------------------------------
   autoconf-2.69-gcc-7.4.0-g23qful          libiconv-1.16-gcc-8.3.0-brkkjge                openmpi-3.1.4-gcc-8.3.0-dorc4s4
   autoconf-2.69-gcc-8.3.0-6nmk4ju          libpciaccess-0.13.5-gcc-8.3.0-rqqclxg          openssl-1.1.1d-gcc-7.4.0-jujqjv5
   automake-1.16.1-gcc-7.4.0-io3tplo        libsigsegv-2.12-gcc-7.4.0-3khohgm              openssl-1.1.1d-gcc-8.3.0-vwjhpks
   automake-1.16.1-gcc-8.3.0-hooeon5        libsigsegv-2.12-gcc-8.3.0-oaiujfn              pcre-8.42-gcc-7.4.0-qa75ghf
   bzip2-1.0.8-gcc-7.4.0-g2ghsbb            libtool-2.4.6-gcc-7.4.0-4neu5jw                perl-5.30.0-gcc-7.4.0-cxcj6ei
   bzip2-1.0.8-gcc-8.3.0-jx7bite            libtool-2.4.6-gcc-8.3.0-too4ft7                perl-5.30.0-gcc-8.3.0-odoz5y2
   cmake-3.15.4-gcc-8.3.0-fnleprp           libxml2-2.9.9-gcc-7.4.0-fg5evg4                pkgconf-1.6.3-gcc-7.4.0-eifxmps
   curl-7.63.0-gcc-7.4.0-opvvtzf            libxml2-2.9.9-gcc-8.3.0-twvbznc                pkgconf-1.6.3-gcc-8.3.0-cysthwv
   diffutils-3.7-gcc-7.4.0-vku7yph          lmod-8.1.5-gcc-7.4.0-vozl7wk                   py-numpy-1.17.3-gcc-8.3.0-yntcyvh
   diffutils-3.7-gcc-8.3.0-5fvw3jh          lua-5.3.5-gcc-7.4.0-dakrkq7                    py-scipy-1.3.1-gcc-8.3.0-userj6l
   expat-2.2.9-gcc-7.4.0-o7ka63e            lua-luafilesystem-1_7_0_2-gcc-7.4.0-ugz4w6v    py-setuptools-41.4.0-gcc-8.3.0-tlap5m6
   expat-2.2.9-gcc-8.3.0-igk6juo            lua-luaposix-33.4.0-gcc-7.4.0-5i3eeji          python-3.7.4-gcc-8.3.0-rip43dt
   findutils-4.6.0-gcc-8.3.0-gxxeusv        m4-1.4.18-gcc-7.4.0-ut64la6                    readline-8.0-gcc-7.4.0-hzwkvqa
   gcc-8.3.0-gcc-7.4.0-rvoysuv       (L)    m4-1.4.18-gcc-8.3.0-65odbgi                    readline-8.0-gcc-8.3.0-kpjhsbz
   gdbm-1.18.1-gcc-7.4.0-surdjxd            mpc-1.1.0-gcc-7.4.0-7uvv4z6                    sqlite-3.30.1-gcc-8.3.0-z7tyt2d
   gdbm-1.18.1-gcc-8.3.0-xbglh3w            mpfr-3.1.6-gcc-7.4.0-joz6bhq                   tar-1.32-gcc-7.4.0-iyu6ntr
   gettext-0.20.1-gcc-7.4.0-4uqpp5g         mpich-3.3.1-gcc-8.3.0-shejyq6                  tar-1.32-gcc-8.3.0-fk4qp4i
   gettext-0.20.1-gcc-8.3.0-iyyrl57         ncurses-6.1-gcc-7.4.0-s4rsior                  tcl-8.6.8-gcc-7.4.0-t3gp773
   git-2.21.0-gcc-7.4.0-gfhbvln             ncurses-6.1-gcc-8.3.0-ros3avk                  texinfo-6.5-gcc-8.3.0-lclslv5
   gmp-6.1.2-gcc-7.4.0-fz3lzqi              netlib-lapack-3.8.0-gcc-8.3.0-kbk7dg3          unzip-6.0-gcc-7.4.0-fxlf2hu
   hwloc-1.11.11-gcc-8.3.0-4ievkpi          netlib-scalapack-2.0.2-gcc-8.3.0-2p75lzq       util-macros-1.19.1-gcc-8.3.0-obeg5kx
   isl-0.18-gcc-7.4.0-f4xq2ne               netlib-scalapack-2.0.2-gcc-8.3.0-2zaeexe       xz-5.2.4-gcc-7.4.0-ur2jffe
   libbsd-0.9.1-gcc-7.4.0-4xrvxug           netlib-scalapack-2.0.2-gcc-8.3.0-4hv3hb4       xz-5.2.4-gcc-8.3.0-jj3rmi7
   libbsd-0.9.1-gcc-8.3.0-m4ib6l2           netlib-scalapack-2.0.2-gcc-8.3.0-hamor5v       zlib-1.2.11-gcc-7.4.0-o2viq7y
   libffi-3.2.1-gcc-8.3.0-dnijwsa           numactl-2.0.12-gcc-8.3.0-tusy4nf               zlib-1.2.11-gcc-8.3.0-2icaxiy
   libiconv-1.16-gcc-7.4.0-zvmmgjb          openblas-0.3.7-gcc-8.3.0-ldv4b4h

  Where:
   L:  Module is loaded

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

The non-hierarchical module files that have been generated so far follow the default rules for module generation. Taking a look at the gcc module you’ll see, for example:

$ module show gcc-8.3.0-gcc-7.4.0-rvoysuv
----------------------------------------------------------------------------------------------------------------------------------------------
   /home/spack/spack/share/spack/modules/linux-ubuntu18.04-x86_64/gcc-8.3.0-gcc-7.4.0-rvoysuv:
----------------------------------------------------------------------------------------------------------------------------------------------
whatis("The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages. ")
prepend_path("PATH","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/bin")
prepend_path("MANPATH","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/share/man")
prepend_path("LD_LIBRARY_PATH","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/lib")
prepend_path("LIBRARY_PATH","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/lib")
prepend_path("LD_LIBRARY_PATH","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/lib64")
prepend_path("LIBRARY_PATH","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/lib64")
prepend_path("CPATH","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/include")
prepend_path("CMAKE_PREFIX_PATH","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/")
setenv("CC","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/bin/gcc")
setenv("CXX","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/bin/g++")
setenv("FC","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/bin/gfortran")
setenv("F77","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/bin/gfortran")
help([[The GNU Compiler Collection includes front ends for C, C++, Objective-C,
Fortran, Ada, and Go, as well as libraries for these languages.
]])

As expected, a few environment variables representing paths will be modified by the module file according to the default prefix inspection rules.

Filter unwanted modifications to the environment

Now consider the case that your site has decided that CPATH and LIBRARY_PATH modifications should not be present in module files. What you can do to abide by the rules is to create a configuration file ${SPACK_ROOT}/etc/spack/modules.yaml with the following content:

modules:
  tcl:
    all:
      filter:
        environment_blacklist: ['CPATH', 'LIBRARY_PATH']

Next you should regenerate all the module files:

$ spack module tcl refresh
==> You are about to regenerate tcl module files for:

-- linux-ubuntu18.04-x86_64 / gcc@7.4.0 -------------------------
g23qful autoconf@2.69    surdjxd gdbm@1.18.1     3khohgm libsigsegv@2.12            ut64la6 m4@1.4.18       eifxmps pkgconf@1.6.3
io3tplo automake@1.16.1  4uqpp5g gettext@0.20.1  4neu5jw libtool@2.4.6              7uvv4z6 mpc@1.1.0       hzwkvqa readline@8.0
g2ghsbb bzip2@1.0.8      gfhbvln git@2.21.0      fg5evg4 libxml2@2.9.9              joz6bhq mpfr@3.1.6      iyu6ntr tar@1.32
opvvtzf curl@7.63.0      fz3lzqi gmp@6.1.2       vozl7wk lmod@8.1.5                 s4rsior ncurses@6.1     t3gp773 tcl@8.6.8
vku7yph diffutils@3.7    f4xq2ne isl@0.18        dakrkq7 lua@5.3.5                  jujqjv5 openssl@1.1.1d  fxlf2hu unzip@6.0
o7ka63e expat@2.2.9      4xrvxug libbsd@0.9.1    ugz4w6v lua-luafilesystem@1_7_0_2  qa75ghf pcre@8.42       ur2jffe xz@5.2.4
rvoysuv gcc@8.3.0        zvmmgjb libiconv@1.16   5i3eeji lua-luaposix@33.4.0        cxcj6ei perl@5.30.0     o2viq7y zlib@1.2.11

-- linux-ubuntu18.04-x86_64 / gcc@8.3.0 -------------------------
6nmk4ju autoconf@2.69    4ievkpi hwloc@1.11.11        shejyq6 mpich@3.3.1             dorc4s4 openmpi@3.1.4         z7tyt2d sqlite@3.30.1
hooeon5 automake@1.16.1  m4ib6l2 libbsd@0.9.1         ros3avk ncurses@6.1             vwjhpks openssl@1.1.1d        fk4qp4i tar@1.32
jx7bite bzip2@1.0.8      dnijwsa libffi@3.2.1         kbk7dg3 netlib-lapack@3.8.0     odoz5y2 perl@5.30.0           lclslv5 texinfo@6.5
fnleprp cmake@3.15.4     brkkjge libiconv@1.16        2p75lzq netlib-scalapack@2.0.2  cysthwv pkgconf@1.6.3         obeg5kx util-macros@1.19.1
5fvw3jh diffutils@3.7    rqqclxg libpciaccess@0.13.5  4hv3hb4 netlib-scalapack@2.0.2  yntcyvh py-numpy@1.17.3       jj3rmi7 xz@5.2.4
igk6juo expat@2.2.9      oaiujfn libsigsegv@2.12      2zaeexe netlib-scalapack@2.0.2  userj6l py-scipy@1.3.1        2icaxiy zlib@1.2.11
gxxeusv findutils@4.6.0  too4ft7 libtool@2.4.6        hamor5v netlib-scalapack@2.0.2  tlap5m6 py-setuptools@41.4.0
xbglh3w gdbm@1.18.1      twvbznc libxml2@2.9.9        tusy4nf numactl@2.0.12          rip43dt python@3.7.4
iyyrl57 gettext@0.20.1   65odbgi m4@1.4.18            ldv4b4h openblas@0.3.7          kpjhsbz readline@8.0

==> Do you want to proceed? [y/n] y
==> Regenerating tcl module files

If you take a look now at the module for gcc you’ll see that the unwanted paths have disappeared:

$ module show gcc-8.3.0-gcc-7.4.0-rvoysuv
----------------------------------------------------------------------------------------------------------------------------------------------
   /home/spack/spack/share/spack/modules/linux-ubuntu18.04-x86_64/gcc-8.3.0-gcc-7.4.0-rvoysuv:
----------------------------------------------------------------------------------------------------------------------------------------------
whatis("The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages. ")
prepend_path("PATH","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/bin")
prepend_path("MANPATH","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/share/man")
prepend_path("LD_LIBRARY_PATH","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/lib")
prepend_path("LD_LIBRARY_PATH","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/lib64")
prepend_path("CMAKE_PREFIX_PATH","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/")
setenv("CC","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/bin/gcc")
setenv("CXX","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/bin/g++")
setenv("FC","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/bin/gfortran")
setenv("F77","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/bin/gfortran")
help([[The GNU Compiler Collection includes front ends for C, C++, Objective-C,
Fortran, Ada, and Go, as well as libraries for these languages.
]])

Prevent some module files from being generated

Another common request at many sites is to avoid exposing software that is only needed as an intermediate step when building a newer stack. Let’s try to prevent the generation of module files for anything that is compiled with gcc@7.4.0 (the OS provided compiler).

To do this you should add a blacklist keyword to ${SPACK_ROOT}/etc/spack/modules.yaml:

modules:
  tcl:
    blacklist:
      -  '%gcc@7.4.0'
    all:
      filter:
        environment_blacklist: ['CPATH', 'LIBRARY_PATH']

and regenerate the module files:

This time it is convenient to pass the option --delete-tree to the command that regenerates the module files to instruct it to delete the existing tree and regenerate a new one instead of overwriting the files in the existing directory.

$ spack module tcl refresh --delete-tree
==> You are about to regenerate tcl module files for:

-- linux-ubuntu18.04-x86_64 / gcc@7.4.0 -------------------------
g23qful autoconf@2.69    surdjxd gdbm@1.18.1     3khohgm libsigsegv@2.12            ut64la6 m4@1.4.18       eifxmps pkgconf@1.6.3
io3tplo automake@1.16.1  4uqpp5g gettext@0.20.1  4neu5jw libtool@2.4.6              7uvv4z6 mpc@1.1.0       hzwkvqa readline@8.0
g2ghsbb bzip2@1.0.8      gfhbvln git@2.21.0      fg5evg4 libxml2@2.9.9              joz6bhq mpfr@3.1.6      iyu6ntr tar@1.32
opvvtzf curl@7.63.0      fz3lzqi gmp@6.1.2       vozl7wk lmod@8.1.5                 s4rsior ncurses@6.1     t3gp773 tcl@8.6.8
vku7yph diffutils@3.7    f4xq2ne isl@0.18        dakrkq7 lua@5.3.5                  jujqjv5 openssl@1.1.1d  fxlf2hu unzip@6.0
o7ka63e expat@2.2.9      4xrvxug libbsd@0.9.1    ugz4w6v lua-luafilesystem@1_7_0_2  qa75ghf pcre@8.42       ur2jffe xz@5.2.4
rvoysuv gcc@8.3.0        zvmmgjb libiconv@1.16   5i3eeji lua-luaposix@33.4.0        cxcj6ei perl@5.30.0     o2viq7y zlib@1.2.11

-- linux-ubuntu18.04-x86_64 / gcc@8.3.0 -------------------------
6nmk4ju autoconf@2.69    4ievkpi hwloc@1.11.11        shejyq6 mpich@3.3.1             dorc4s4 openmpi@3.1.4         z7tyt2d sqlite@3.30.1
hooeon5 automake@1.16.1  m4ib6l2 libbsd@0.9.1         ros3avk ncurses@6.1             vwjhpks openssl@1.1.1d        fk4qp4i tar@1.32
jx7bite bzip2@1.0.8      dnijwsa libffi@3.2.1         kbk7dg3 netlib-lapack@3.8.0     odoz5y2 perl@5.30.0           lclslv5 texinfo@6.5
fnleprp cmake@3.15.4     brkkjge libiconv@1.16        2p75lzq netlib-scalapack@2.0.2  cysthwv pkgconf@1.6.3         obeg5kx util-macros@1.19.1
5fvw3jh diffutils@3.7    rqqclxg libpciaccess@0.13.5  4hv3hb4 netlib-scalapack@2.0.2  yntcyvh py-numpy@1.17.3       jj3rmi7 xz@5.2.4
igk6juo expat@2.2.9      oaiujfn libsigsegv@2.12      2zaeexe netlib-scalapack@2.0.2  userj6l py-scipy@1.3.1        2icaxiy zlib@1.2.11
gxxeusv findutils@4.6.0  too4ft7 libtool@2.4.6        hamor5v netlib-scalapack@2.0.2  tlap5m6 py-setuptools@41.4.0
xbglh3w gdbm@1.18.1      twvbznc libxml2@2.9.9        tusy4nf numactl@2.0.12          rip43dt python@3.7.4
iyyrl57 gettext@0.20.1   65odbgi m4@1.4.18            ldv4b4h openblas@0.3.7          kpjhsbz readline@8.0

==> Do you want to proceed? [y/n] y
==> Regenerating tcl module files

$ module avail

----------------------------------------- /home/spack/spack/share/spack/modules/linux-ubuntu18.04-x86_64 -----------------------------------------
   autoconf-2.69-gcc-8.3.0-6nmk4ju          libsigsegv-2.12-gcc-8.3.0-oaiujfn           openssl-1.1.1d-gcc-8.3.0-vwjhpks
   automake-1.16.1-gcc-8.3.0-hooeon5        libtool-2.4.6-gcc-8.3.0-too4ft7             perl-5.30.0-gcc-8.3.0-odoz5y2
   bzip2-1.0.8-gcc-8.3.0-jx7bite            libxml2-2.9.9-gcc-8.3.0-twvbznc             pkgconf-1.6.3-gcc-8.3.0-cysthwv
   cmake-3.15.4-gcc-8.3.0-fnleprp           m4-1.4.18-gcc-8.3.0-65odbgi                 py-numpy-1.17.3-gcc-8.3.0-yntcyvh
   diffutils-3.7-gcc-8.3.0-5fvw3jh          mpich-3.3.1-gcc-8.3.0-shejyq6               py-scipy-1.3.1-gcc-8.3.0-userj6l
   expat-2.2.9-gcc-8.3.0-igk6juo            ncurses-6.1-gcc-8.3.0-ros3avk               py-setuptools-41.4.0-gcc-8.3.0-tlap5m6
   findutils-4.6.0-gcc-8.3.0-gxxeusv        netlib-lapack-3.8.0-gcc-8.3.0-kbk7dg3       python-3.7.4-gcc-8.3.0-rip43dt
   gdbm-1.18.1-gcc-8.3.0-xbglh3w            netlib-scalapack-2.0.2-gcc-8.3.0-2p75lzq    readline-8.0-gcc-8.3.0-kpjhsbz
   gettext-0.20.1-gcc-8.3.0-iyyrl57         netlib-scalapack-2.0.2-gcc-8.3.0-2zaeexe    sqlite-3.30.1-gcc-8.3.0-z7tyt2d
   hwloc-1.11.11-gcc-8.3.0-4ievkpi          netlib-scalapack-2.0.2-gcc-8.3.0-4hv3hb4    tar-1.32-gcc-8.3.0-fk4qp4i
   libbsd-0.9.1-gcc-8.3.0-m4ib6l2           netlib-scalapack-2.0.2-gcc-8.3.0-hamor5v    texinfo-6.5-gcc-8.3.0-lclslv5
   libffi-3.2.1-gcc-8.3.0-dnijwsa           numactl-2.0.12-gcc-8.3.0-tusy4nf            util-macros-1.19.1-gcc-8.3.0-obeg5kx
   libiconv-1.16-gcc-8.3.0-brkkjge          openblas-0.3.7-gcc-8.3.0-ldv4b4h            xz-5.2.4-gcc-8.3.0-jj3rmi7
   libpciaccess-0.13.5-gcc-8.3.0-rqqclxg    openmpi-3.1.4-gcc-8.3.0-dorc4s4             zlib-1.2.11-gcc-8.3.0-2icaxiy

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

If you look closely you’ll see though that we went too far in blacklisting modules: the module for gcc@8.3.0 disappeared as it was bootstrapped with gcc@7.4.0. To specify exceptions to the blacklist rules you can use whitelist:

modules:
  tcl:
    whitelist:
      -  gcc
    blacklist:
      -  '%gcc@7.4.0'
    all:
      filter:
        environment_blacklist: ['CPATH', 'LIBRARY_PATH']

whitelist rules always have precedence over blacklist rules. If you regenerate the modules again:

$ spack module tcl refresh -y
==> Regenerating tcl module files

you’ll see that now the module for gcc@8.3.0 has reappeared:

$ module avail gcc-8.3.0-gcc-7.4.0-rvoysuv

----------------------------------------- /home/spack/spack/share/spack/modules/linux-ubuntu18.04-x86_64 -----------------------------------------
   gcc-8.3.0-gcc-7.4.0-rvoysuv (L)

  Where:
   L:  Module is loaded

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

An additional possibility that you can leverage to unclutter the environment is that of preventing the generation of module files for implicitly installed packages. In this case all one needs to do is to add the following line:

modules:
  tcl:
    blacklist_implicits: true
    whitelist:
      -  gcc
    blacklist:
      -  '%gcc@7.4.0'
    all:
      filter:
        environment_blacklist: ['CPATH', 'LIBRARY_PATH']

to modules.yaml and regenerate the module file tree as above.

Change module file naming

The next step in making module files more user-friendly is to improve their naming scheme. To reduce the length of the hash or remove it altogether you can use the hash_length keyword in the configuration file:

modules:
  tcl:
    hash_length: 0
    whitelist:
      -  gcc
    blacklist:
      -  '%gcc@7.4.0'
    all:
      filter:
        environment_blacklist: ['CPATH', 'LIBRARY_PATH']

If you try to regenerate the module files now you will get an error:

$ spack module tcl refresh --delete-tree -y
==> Error: Name clashes detected in module files:

file: /home/spack/spack/share/spack/modules/linux-ubuntu18.04-x86_64/netlib-scalapack-2.0.2-gcc-8.3.0
spec: netlib-scalapack@2.0.2%gcc@8.3.0 build_type=RelWithDebInfo patches=22ebf4e3d5a6356cd6086ea65bfdf30f9d0a2038136127590cd269d15bdb03af,e8f30dd1f26e523dfb552f8d7b8ad26ac88fc0c8d72e3d4f9a9717a3383e0b33 ~pic+shared arch=linux-ubuntu18.04-x86_64
spec: netlib-scalapack@2.0.2%gcc@8.3.0 build_type=RelWithDebInfo patches=22ebf4e3d5a6356cd6086ea65bfdf30f9d0a2038136127590cd269d15bdb03af,e8f30dd1f26e523dfb552f8d7b8ad26ac88fc0c8d72e3d4f9a9717a3383e0b33 ~pic+shared arch=linux-ubuntu18.04-x86_64
spec: netlib-scalapack@2.0.2%gcc@8.3.0 build_type=RelWithDebInfo patches=22ebf4e3d5a6356cd6086ea65bfdf30f9d0a2038136127590cd269d15bdb03af,e8f30dd1f26e523dfb552f8d7b8ad26ac88fc0c8d72e3d4f9a9717a3383e0b33 ~pic+shared arch=linux-ubuntu18.04-x86_64
spec: netlib-scalapack@2.0.2%gcc@8.3.0 build_type=RelWithDebInfo patches=22ebf4e3d5a6356cd6086ea65bfdf30f9d0a2038136127590cd269d15bdb03af,e8f30dd1f26e523dfb552f8d7b8ad26ac88fc0c8d72e3d4f9a9717a3383e0b33 ~pic+shared arch=linux-ubuntu18.04-x86_64

==> Error: Operation aborted

Note

We try to check for errors upfront!

In Spack we check for errors upfront whenever possible, so don’t worry about your module files: as a name clash was detected nothing has been changed on disk.

The problem here is that without the hashes the four different flavors of netlib-scalapack map to the same module file name. We can add suffixes to differentiate them:

 modules:
   tcl:
     hash_length: 0
     whitelist:
       -  gcc
     blacklist:
       -  '%gcc@7.4.0'
     all:
       suffixes:
         '^openblas': openblas
         '^netlib-lapack': netlib
       filter:
         environment_blacklist: ['CPATH', 'LIBRARY_PATH']
     netlib-scalapack:
       suffixes:
         '^openmpi': openmpi
         '^mpich': mpich

As you can see it is possible to specify rules that apply only to a restricted set of packages using anonymous specs. Regenerating module files now we obtain:

$ spack module tcl refresh --delete-tree -y
==> Regenerating tcl module files
$ module avail

----------------------------------------- /home/spack/spack/share/spack/modules/linux-ubuntu18.04-x86_64 -----------------------------------------
   autoconf-2.69-gcc-8.3.0          libsigsegv-2.12-gcc-8.3.0                            perl-5.30.0-gcc-8.3.0
   automake-1.16.1-gcc-8.3.0        libtool-2.4.6-gcc-8.3.0                              pkgconf-1.6.3-gcc-8.3.0
   bzip2-1.0.8-gcc-8.3.0            libxml2-2.9.9-gcc-8.3.0                              py-numpy-1.17.3-gcc-8.3.0-openblas
   cmake-3.15.4-gcc-8.3.0           m4-1.4.18-gcc-8.3.0                                  py-scipy-1.3.1-gcc-8.3.0-openblas
   diffutils-3.7-gcc-8.3.0          mpich-3.3.1-gcc-8.3.0                                py-setuptools-41.4.0-gcc-8.3.0
   expat-2.2.9-gcc-8.3.0            ncurses-6.1-gcc-8.3.0                                python-3.7.4-gcc-8.3.0
   findutils-4.6.0-gcc-8.3.0        netlib-lapack-3.8.0-gcc-8.3.0                        readline-8.0-gcc-8.3.0
   gcc-8.3.0-gcc-7.4.0              netlib-scalapack-2.0.2-gcc-8.3.0-netlib-mpich        sqlite-3.30.1-gcc-8.3.0
   gdbm-1.18.1-gcc-8.3.0            netlib-scalapack-2.0.2-gcc-8.3.0-netlib-openmpi      tar-1.32-gcc-8.3.0
   gettext-0.20.1-gcc-8.3.0         netlib-scalapack-2.0.2-gcc-8.3.0-openblas-mpich      texinfo-6.5-gcc-8.3.0
   hwloc-1.11.11-gcc-8.3.0          netlib-scalapack-2.0.2-gcc-8.3.0-openblas-openmpi    util-macros-1.19.1-gcc-8.3.0
   libbsd-0.9.1-gcc-8.3.0           numactl-2.0.12-gcc-8.3.0                             xz-5.2.4-gcc-8.3.0
   libffi-3.2.1-gcc-8.3.0           openblas-0.3.7-gcc-8.3.0                             zlib-1.2.11-gcc-8.3.0
   libiconv-1.16-gcc-8.3.0          openmpi-3.1.4-gcc-8.3.0
   libpciaccess-0.13.5-gcc-8.3.0    openssl-1.1.1d-gcc-8.3.0

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

Finally we can set a naming_scheme to prevent users from loading modules that refer to different flavors of the same library/application:

modules:
  tcl:
    hash_length: 0
    naming_scheme: '{name}/{version}-{compiler.name}-{compiler.version}'
    whitelist:
      -  gcc
    blacklist:
      -  '%gcc@7.4.0'
    all:
      conflict:
        - '{name}'
      suffixes:
        '^openblas': openblas
        '^netlib-lapack': netlib
      filter:
        environment_blacklist: ['CPATH', 'LIBRARY_PATH']
    netlib-scalapack:
      suffixes:
        '^openmpi': openmpi
        '^mpich': mpich

The final result should look like:

$ spack module tcl refresh --delete-tree -y
==> Regenerating tcl module files
$ module avail

----------------------------------------- /home/spack/spack/share/spack/modules/linux-ubuntu18.04-x86_64 -----------------------------------------
   autoconf/2.69-gcc-8.3.0          libsigsegv/2.12-gcc-8.3.0                                perl/5.30.0-gcc-8.3.0
   automake/1.16.1-gcc-8.3.0        libtool/2.4.6-gcc-8.3.0                                  pkgconf/1.6.3-gcc-8.3.0
   bzip2/1.0.8-gcc-8.3.0            libxml2/2.9.9-gcc-8.3.0                                  py-numpy/1.17.3-gcc-8.3.0-openblas
   cmake/3.15.4-gcc-8.3.0           m4/1.4.18-gcc-8.3.0                                      py-scipy/1.3.1-gcc-8.3.0-openblas
   diffutils/3.7-gcc-8.3.0          mpich/3.3.1-gcc-8.3.0                                    py-setuptools/41.4.0-gcc-8.3.0
   expat/2.2.9-gcc-8.3.0            ncurses/6.1-gcc-8.3.0                                    python/3.7.4-gcc-8.3.0
   findutils/4.6.0-gcc-8.3.0        netlib-lapack/3.8.0-gcc-8.3.0                            readline/8.0-gcc-8.3.0
   gcc/8.3.0-gcc-7.4.0              netlib-scalapack/2.0.2-gcc-8.3.0-netlib-mpich            sqlite/3.30.1-gcc-8.3.0
   gdbm/1.18.1-gcc-8.3.0            netlib-scalapack/2.0.2-gcc-8.3.0-netlib-openmpi          tar/1.32-gcc-8.3.0
   gettext/0.20.1-gcc-8.3.0         netlib-scalapack/2.0.2-gcc-8.3.0-openblas-mpich          texinfo/6.5-gcc-8.3.0
   hwloc/1.11.11-gcc-8.3.0          netlib-scalapack/2.0.2-gcc-8.3.0-openblas-openmpi (D)    util-macros/1.19.1-gcc-8.3.0
   libbsd/0.9.1-gcc-8.3.0           numactl/2.0.12-gcc-8.3.0                                 xz/5.2.4-gcc-8.3.0
   libffi/3.2.1-gcc-8.3.0           openblas/0.3.7-gcc-8.3.0                                 zlib/1.2.11-gcc-8.3.0
   libiconv/1.16-gcc-8.3.0          openmpi/3.1.4-gcc-8.3.0
   libpciaccess/0.13.5-gcc-8.3.0    openssl/1.1.1d-gcc-8.3.0

  Where:
   D:  Default Module

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

Note

TCL specific directive

The directives naming_scheme and conflict are TCL specific and can’t be used in the lmod section of the configuration file.

Add custom environment modifications

At many sites it is customary to set an environment variable in a package’s module file that points to the folder in which the package is installed. You can achieve this with Spack by adding an environment directive to the configuration file:

modules:
  tcl:
    hash_length: 0
    naming_scheme: '{name}/{version}-{compiler.name}-{compiler.version}'
    whitelist:
      -  gcc
    blacklist:
      -  '%gcc@7.4.0'
    all:
      conflict:
        - '{name}'
      suffixes:
        '^openblas': openblas
        '^netlib-lapack': netlib
      filter:
        environment_blacklist: ['CPATH', 'LIBRARY_PATH']
      environment:
        set:
          '{name}_ROOT': '{prefix}'
    netlib-scalapack:
      suffixes:
        '^openmpi': openmpi
        '^mpich': mpich

Under the hood Spack uses the format() API to substitute tokens in either environment variable names or values. There are two caveats though:

  • The set of allowed tokens in variable names is restricted to name, version, compiler, compiler.name, compiler.version, architecture

  • Any token expanded in a variable name is made uppercase, but other than that case sensitivity is preserved

Regenerating the module files results in something like:

$ spack module tcl refresh -y
==> Regenerating tcl module files

$ module show gcc
----------------------------------------------------------------------------------------------------------------------------------------------
   /home/spack/spack/share/spack/modules/linux-ubuntu18.04-x86_64/gcc/8.3.0-gcc-7.4.0:
----------------------------------------------------------------------------------------------------------------------------------------------
whatis("The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages. ")
conflict("gcc")
prepend_path("PATH","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/bin")
prepend_path("MANPATH","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/share/man")
prepend_path("LD_LIBRARY_PATH","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/lib")
prepend_path("LD_LIBRARY_PATH","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/lib64")
prepend_path("CMAKE_PREFIX_PATH","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/")
setenv("CC","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/bin/gcc")
setenv("CXX","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/bin/g++")
setenv("FC","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/bin/gfortran")
setenv("F77","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k/bin/gfortran")
setenv("GCC_ROOT","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.4.0/gcc-8.3.0-rvoysuvia7pirmb3kee6xjh7zcmhbi5k")
help([[The GNU Compiler Collection includes front ends for C, C++, Objective-C,
Fortran, Ada, and Go, as well as libraries for these languages.
]])

As you can see, the gcc module has the environment variable GCC_ROOT set.

Sometimes it’s also useful to apply environment modifications selectively and target only certain packages. You can for instance apply modifications to the openmpi module as follows:

modules:
  tcl:
    hash_length: 0
    naming_scheme: '{name}/{version}-{compiler.name}-{compiler.version}'
    whitelist:
      - gcc
    blacklist:
      - '%gcc@7.4.0'
    all:
      conflict:
        - '{name}'
      suffixes:
        '^openblas': openblas
        '^netlib-lapack': netlib
      filter:
        environment_blacklist: ['CPATH', 'LIBRARY_PATH']
      environment:
        set:
          '{name}_ROOT': '{prefix}'
    openmpi:
      environment:
        set:
          SLURM_MPI_TYPE: pmi2
          OMPI_MCA_btl_openib_warn_default_gid_prefix: '0'
    netlib-scalapack:
      suffixes:
        '^openmpi': openmpi
        '^mpich': mpich

This time we will be more selective and regenerate only the openmpi module file:

$ spack module tcl refresh -y openmpi
==> Regenerating tcl module files

$ module show openmpi
----------------------------------------------------------------------------------------------------------------------------------------------
   /home/spack/spack/share/spack/modules/linux-ubuntu18.04-x86_64/openmpi/3.1.4-gcc-8.3.0:
----------------------------------------------------------------------------------------------------------------------------------------------
whatis("An open source Message Passing Interface implementation. ")
conflict("openmpi")
prepend_path("PATH","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-8.3.0/openmpi-3.1.4-dorc4s4cuegnr44fmgl72bddrtbv76yd/bin")
prepend_path("MANPATH","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-8.3.0/openmpi-3.1.4-dorc4s4cuegnr44fmgl72bddrtbv76yd/share/man")
prepend_path("LD_LIBRARY_PATH","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-8.3.0/openmpi-3.1.4-dorc4s4cuegnr44fmgl72bddrtbv76yd/lib")
prepend_path("PKG_CONFIG_PATH","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-8.3.0/openmpi-3.1.4-dorc4s4cuegnr44fmgl72bddrtbv76yd/lib/pkgconfig")
prepend_path("CMAKE_PREFIX_PATH","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-8.3.0/openmpi-3.1.4-dorc4s4cuegnr44fmgl72bddrtbv76yd/")
setenv("OPENMPI_ROOT","/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-8.3.0/openmpi-3.1.4-dorc4s4cuegnr44fmgl72bddrtbv76yd")
setenv("SLURM_MPI_TYPE","pmi2")
setenv("OMPI_MCA_btl_openib_warn_default_gid_prefix","0")
help([[An open source Message Passing Interface implementation. The Open MPI
Project is an open source Message Passing Interface implementation that
is developed and maintained by a consortium of academic, research, and
industry partners. Open MPI is therefore able to combine the expertise,
technologies, and resources from all across the High Performance
Computing community in order to build the best MPI library available.
Open MPI offers advantages for system and software vendors, application
developers and computer science researchers.
]])

Autoload dependencies

Spack can also generate module files that contain code to load the dependencies automatically. You can, for instance generate python modules that load their dependencies by adding the autoload directive and assigning it the value direct:

modules:
  tcl:
    verbose: True
    hash_length: 0
    naming_scheme: '{name}/{version}-{compiler.name}-{compiler.version}'
    whitelist:
      - gcc
    blacklist:
      - '%gcc@7.4.0'
    all:
      conflict:
        - '{name}'
      suffixes:
        '^openblas': openblas
        '^netlib-lapack': netlib
      filter:
        environment_blacklist: ['CPATH', 'LIBRARY_PATH']
      environment:
        set:
          '{name}_ROOT': '{prefix}'
    gcc:
      environment:
        set:
          CC: gcc
          CXX: g++
          FC: gfortran
          F90: gfortran
          F77: gfortran
    openmpi:
      environment:
        set:
          SLURM_MPI_TYPE: pmi2
          OMPI_MCA_btl_openib_warn_default_gid_prefix: '0'
    netlib-scalapack:
      suffixes:
        '^openmpi': openmpi
        '^mpich': mpich
    ^python:
      autoload:  direct

and regenerating the module files for every package that depends on python:

$ spack module tcl refresh -y ^python
==> Regenerating tcl module files

Now the py-scipy module will be:

#%Module1.0
## Module file created by spack (https://github.com/spack/spack) on 2019-11-13 10:02:18.588422
##
## py-scipy@1.3.1%gcc@8.3.0 arch=linux-ubuntu18.04-x86_64/userj6l
##


module-whatis "SciPy (pronounced 'Sigh Pie') is a Scientific Library for Python. It provides many user-friendly and efficient numerical routines such as routines for numerical integration and optimization."

proc ModulesHelp { } {
puts stderr "SciPy (pronounced "Sigh Pie") is a Scientific Library for Python. It"
puts stderr "provides many user-friendly and efficient numerical routines such as"
puts stderr "routines for numerical integration and optimization."
}

if { [ module-info mode load ] && ![ is-loaded openblas/0.3.7-gcc-8.3.0 ] } {
    puts stderr "Autoloading openblas/0.3.7-gcc-8.3.0"
    module load openblas/0.3.7-gcc-8.3.0
}
if { [ module-info mode load ] && ![ is-loaded py-numpy/1.17.3-gcc-8.3.0-openblas ] } {
    puts stderr "Autoloading py-numpy/1.17.3-gcc-8.3.0-openblas"
    module load py-numpy/1.17.3-gcc-8.3.0-openblas
}
if { [ module-info mode load ] && ![ is-loaded python/3.7.4-gcc-8.3.0 ] } {
    puts stderr "Autoloading python/3.7.4-gcc-8.3.0"
    module load python/3.7.4-gcc-8.3.0
}
conflict py-scipy

prepend-path LD_LIBRARY_PATH "/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-8.3.0/py-scipy-1.3.1-userj6lfsdr4nhgfdmvuzrw3pihz6rcm/lib"
prepend-path CMAKE_PREFIX_PATH "/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-8.3.0/py-scipy-1.3.1-userj6lfsdr4nhgfdmvuzrw3pihz6rcm/"
prepend-path PYTHONPATH "/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-8.3.0/py-scipy-1.3.1-userj6lfsdr4nhgfdmvuzrw3pihz6rcm/lib/python3.7/site-packages"
setenv PY_SCIPY_ROOT "/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-8.3.0/py-scipy-1.3.1-userj6lfsdr4nhgfdmvuzrw3pihz6rcm"

and will contain code to autoload all the dependencies:

$ module load py-scipy
Autoloading openblas/0.3.7-gcc-8.3.0
Autoloading py-numpy/1.17.3-gcc-8.3.0-openblas
Autoloading python/3.7.4-gcc-8.3.0
Autoloading python/3.7.4-gcc-8.3.0

In case messages are unwanted during the autoload procedure, it will be sufficient to omit the line setting verbose: True in the configuration file above.

Hierarchical Module Files

So far we worked with non-hierarchical module files, i.e. with module files that are all generated in the same root directory and don’t attempt to dynamically modify the MODULEPATH. This results in a flat module structure where all the software is visible at the same time:

$ module avail

----------------------------------------- /home/spack/spack/share/spack/modules/linux-ubuntu18.04-x86_64 -----------------------------------------
   autoconf/2.69-gcc-8.3.0          libsigsegv/2.12-gcc-8.3.0                                perl/5.30.0-gcc-8.3.0
   automake/1.16.1-gcc-8.3.0        libtool/2.4.6-gcc-8.3.0                                  pkgconf/1.6.3-gcc-8.3.0
   bzip2/1.0.8-gcc-8.3.0            libxml2/2.9.9-gcc-8.3.0                                  py-numpy/1.17.3-gcc-8.3.0-openblas (L)
   cmake/3.15.4-gcc-8.3.0           m4/1.4.18-gcc-8.3.0                                      py-scipy/1.3.1-gcc-8.3.0-openblas  (L)
   diffutils/3.7-gcc-8.3.0          mpich/3.3.1-gcc-8.3.0                                    py-setuptools/41.4.0-gcc-8.3.0
   expat/2.2.9-gcc-8.3.0            ncurses/6.1-gcc-8.3.0                                    python/3.7.4-gcc-8.3.0             (L)
   findutils/4.6.0-gcc-8.3.0        netlib-lapack/3.8.0-gcc-8.3.0                            readline/8.0-gcc-8.3.0
   gcc/8.3.0-gcc-7.4.0              netlib-scalapack/2.0.2-gcc-8.3.0-netlib-mpich            sqlite/3.30.1-gcc-8.3.0
   gdbm/1.18.1-gcc-8.3.0            netlib-scalapack/2.0.2-gcc-8.3.0-netlib-openmpi          tar/1.32-gcc-8.3.0
   gettext/0.20.1-gcc-8.3.0         netlib-scalapack/2.0.2-gcc-8.3.0-openblas-mpich          texinfo/6.5-gcc-8.3.0
   hwloc/1.11.11-gcc-8.3.0          netlib-scalapack/2.0.2-gcc-8.3.0-openblas-openmpi (D)    util-macros/1.19.1-gcc-8.3.0
   libbsd/0.9.1-gcc-8.3.0           numactl/2.0.12-gcc-8.3.0                                 xz/5.2.4-gcc-8.3.0
   libffi/3.2.1-gcc-8.3.0           openblas/0.3.7-gcc-8.3.0                          (L)    zlib/1.2.11-gcc-8.3.0
   libiconv/1.16-gcc-8.3.0          openmpi/3.1.4-gcc-8.3.0
   libpciaccess/0.13.5-gcc-8.3.0    openssl/1.1.1d-gcc-8.3.0

  Where:
   L:  Module is loaded
   D:  Default Module

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

This layout is quite simple to deploy, but you can see from the above snippet that nothing prevents users from loading incompatible sets of modules:

$ module purge
$ module load netlib-lapack openblas
$ module list

Currently Loaded Modules:
  1) netlib-lapack/3.8.0-gcc-8.3.0   2) openblas/0.3.7-gcc-8.3.0

Even if conflicts directives are carefully placed in module files, they:

  • won’t enforce a consistent environment, but will just report an error

  • need constant updates, for instance as soon as a new compiler or MPI library is installed

Hierarchical module files try to overcome these shortcomings by showing at start-up only a restricted view of what is available on the system: more specifically only the software that has been installed with OS provided compilers. Among this software there will be other - usually more recent - compilers that, once loaded, will prepend new directories to MODULEPATH unlocking all the software that was compiled with them. This “unlocking” idea can then be extended arbitrarily to virtual dependencies, as we’ll see in the following section.

Core/Compiler/MPI

The most widely used hierarchy is the so called Core/Compiler/MPI where, on top of the compilers, different MPI libraries also unlock software linked to them. There are just a few steps needed to adapt the modules.yaml file we used previously:

  1. enable the lmod file generator

  2. change the tcl tag to lmod

  3. remove tcl specific directives (naming_scheme and conflict)

  4. declare which compilers are considered core_compilers

  5. remove the mpi related suffixes (as they will be substituted by hierarchies)

After these modifications your configuration file should look like:

modules:
  enable::
    - lmod
  lmod:
    core_compilers:
      - 'gcc@7.4.0'
    hierarchy:
      - mpi
    hash_length: 0
    whitelist:
      - gcc
    blacklist:
      - '%gcc@7.4.0'
    all:
      suffixes:
        '^openblas': openblas
        '^netlib-lapack': netlib
      filter:
        environment_blacklist: ['CPATH', 'LIBRARY_PATH']
      environment:
        set:
          '{name}_ROOT': '{prefix}'
    openmpi:
      environment:
        set:
          SLURM_MPI_TYPE: pmi2
          OMPI_MCA_btl_openib_warn_default_gid_prefix: '0'

Note

Double colon in configuration files

The double colon after enable is intentional and it serves the purpose of overriding the default list of enabled generators so that only lmod will be active (see Overriding entire sections for more details).

The directive core_compilers accepts a list of compilers. Everything built using these compilers will create a module in the Core part of the hierarchy, which is the entry point for hierarchical module files. It is common practice to put the OS provided compilers in the list and only build common utilities and other compilers with them.

If we now regenerate the module files:

$ spack module lmod refresh --delete-tree -y
==> Regenerating lmod module files

and update MODULEPATH to point to the Core:

$ module purge
$ module unuse $HOME/spack/share/spack/modules/linux-ubuntu18.04-x86_64
$ module use $HOME/spack/share/spack/lmod/linux-ubuntu18.04-x86_64/Core

asking for the available modules will return:

$ module avail

---------------------------------------- /home/spack/spack/share/spack/lmod/linux-ubuntu18.04-x86_64/Core ----------------------------------------
   gcc/8.3.0

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

Unsurprisingly, the only visible module is gcc. Loading that we’ll unlock the Compiler part of the hierarchy:

$ module load gcc
$ module avail

------------------------------------- /home/spack/spack/share/spack/lmod/linux-ubuntu18.04-x86_64/gcc/8.3.0 --------------------------------------
   autoconf/2.69      gdbm/1.18.1            libsigsegv/2.12        numactl/2.0.12              py-scipy/1.3.1-openblas    util-macros/1.19.1
   automake/1.16.1    gettext/0.20.1         libtool/2.4.6          openblas/0.3.7              py-setuptools/41.4.0       xz/5.2.4
   bzip2/1.0.8        hwloc/1.11.11          libxml2/2.9.9          openmpi/3.1.4               python/3.7.4               zlib/1.2.11
   cmake/3.15.4       libbsd/0.9.1           m4/1.4.18              openssl/1.1.1d              readline/8.0
   diffutils/3.7      libffi/3.2.1           mpich/3.3.1            perl/5.30.0                 sqlite/3.30.1
   expat/2.2.9        libiconv/1.16          ncurses/6.1            pkgconf/1.6.3               tar/1.32
   findutils/4.6.0    libpciaccess/0.13.5    netlib-lapack/3.8.0    py-numpy/1.17.3-openblas    texinfo/6.5

---------------------------------------- /home/spack/spack/share/spack/lmod/linux-ubuntu18.04-x86_64/Core ----------------------------------------
   gcc/8.3.0 (L)

  Where:
   L:  Module is loaded

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

The same holds true also for the MPI part, that you can enable by loading either mpich or openmpi. Let’s start by loading mpich:

$ module load mpich
$ module avail

--------------------------- /home/spack/spack/share/spack/lmod/linux-ubuntu18.04-x86_64/mpich/3.3.1-shejyq6/gcc/8.3.0 ----------------------------
   netlib-scalapack/2.0.2-netlib-mpich    netlib-scalapack/2.0.2-openblas-mpich (D)

------------------------------------- /home/spack/spack/share/spack/lmod/linux-ubuntu18.04-x86_64/gcc/8.3.0 --------------------------------------
   autoconf/2.69      gettext/0.20.1         libxml2/2.9.9              openssl/1.1.1d              sqlite/3.30.1
   automake/1.16.1    hwloc/1.11.11          m4/1.4.18                  perl/5.30.0                 tar/1.32
   bzip2/1.0.8        libbsd/0.9.1           mpich/3.3.1         (L)    pkgconf/1.6.3               texinfo/6.5
   cmake/3.15.4       libffi/3.2.1           ncurses/6.1                py-numpy/1.17.3-openblas    util-macros/1.19.1
   diffutils/3.7      libiconv/1.16          netlib-lapack/3.8.0        py-scipy/1.3.1-openblas     xz/5.2.4
   expat/2.2.9        libpciaccess/0.13.5    numactl/2.0.12             py-setuptools/41.4.0        zlib/1.2.11
   findutils/4.6.0    libsigsegv/2.12        openblas/0.3.7             python/3.7.4
   gdbm/1.18.1        libtool/2.4.6          openmpi/3.1.4              readline/8.0

---------------------------------------- /home/spack/spack/share/spack/lmod/linux-ubuntu18.04-x86_64/Core ----------------------------------------
   gcc/8.3.0 (L)

  Where:
   L:  Module is loaded
   D:  Default Module

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".


$ module load openblas netlib-scalapack/2.0.2-openblas
$ module list

Currently Loaded Modules:
  1) gcc/8.3.0   2) mpich/3.3.1   3) openblas/0.3.7   4) netlib-scalapack/2.0.2-openblas

At this point we can showcase the improved consistency that a hierarchical layout provides over a non-hierarchical one:

$ module load openmpi

Lmod is automatically replacing "mpich/3.3.1" with "openmpi/3.1.4".


Due to MODULEPATH changes, the following have been reloaded:
  1) netlib-scalapack/2.0.2-openblas

Lmod took care of swapping the MPI provider for us, and it also substituted the netlib-scalapack module to conform to the change in the MPI. In this way we can’t accidentally pull-in two different MPI providers at the same time or load a module file for a package linked to openmpi when mpich is also loaded. Consistency for compilers and MPI is ensured by the tool.

Add LAPACK to the hierarchy

The hierarchy just shown is already a great improvement over non-hierarchical layouts, but it still has an asymmetry: LAPACK providers cover the same semantic role as MPI providers, but yet they are not part of the hierarchy.

To be more practical, this means that although we have gained an improved consistency in our environment when it comes to MPI, we still have the same problems as we had before for LAPACK implementations:

$ module list

Currently Loaded Modules:
  1) gcc/8.3.0   2) openblas/0.3.7   3) openmpi/3.1.4   4) netlib-scalapack/2.0.2-openblas

$ module load netlib-scalapack/2.0.2-netlib

The following have been reloaded with a version change:
  1) netlib-scalapack/2.0.2-openblas => netlib-scalapack/2.0.2-netlib

$ module list

Currently Loaded Modules:
  1) gcc/8.3.0   2) openblas/0.3.7   3) openmpi/3.1.4   4) netlib-scalapack/2.0.2-netlib

Hierarchies that are deeper than Core/Compiler/MPI are probably still considered “unusual” or “impractical” at many sites, mainly because module files are written manually and keeping track of the combinations among multiple providers quickly becomes quite involved.

For instance, having both MPI and LAPACK in the hierarchy means we must classify software into one of four categories:

  1. Software that doesn’t depend on MPI or LAPACK

  2. Software that depends only on MPI

  3. Software that depends only on LAPACK

  4. Software that depends on both

to decide when to show it to the user. The situation becomes more involved as the number of virtual dependencies in the hierarchy increases.

We can take advantage of the DAG that Spack maintains for the installed software and solve this combinatorial problem in a clean and automated way. In some sense Spack’s ability to manage this combinatorial complexity makes deeper hierarchies feasible.

Coming back to our example, let’s add lapack to the hierarchy and remove any remaining suffix:

modules:
  enable::
    - lmod
  lmod:
    core_compilers:
      - 'gcc@7.4.0'
    hierarchy:
      - mpi
      - lapack
    hash_length: 0
    whitelist:
      - gcc
    blacklist:
      - '%gcc@7.4.0'
    all:
      filter:
        environment_blacklist: ['CPATH', 'LIBRARY_PATH']
      environment:
        set:
          '{name}_ROOT': '{prefix}'
    openmpi:
      environment:
        set:
          SLURM_MPI_TYPE: pmi2
          OMPI_MCA_btl_openib_warn_default_gid_prefix: '0'

After module files have been regenerated as usual:

$ module purge

$ spack module lmod refresh --delete-tree -y
==> Regenerating lmod module files

we can see that now we have additional components in the hierarchy:

$ module load gcc
$ module load openblas
$ module avail

-------------------------- /home/spack/spack/share/spack/lmod/linux-ubuntu18.04-x86_64/openblas/0.3.7-ldv4b4h/gcc/8.3.0 --------------------------
   py-numpy/1.17.3    py-scipy/1.3.1

------------------------------------- /home/spack/spack/share/spack/lmod/linux-ubuntu18.04-x86_64/gcc/8.3.0 --------------------------------------
   autoconf/2.69      findutils/4.6.0    libiconv/1.16          mpich/3.3.1                openssl/1.1.1d          sqlite/3.30.1
   automake/1.16.1    gdbm/1.18.1        libpciaccess/0.13.5    ncurses/6.1                perl/5.30.0             tar/1.32
   bzip2/1.0.8        gettext/0.20.1     libsigsegv/2.12        netlib-lapack/3.8.0        pkgconf/1.6.3           texinfo/6.5
   cmake/3.15.4       hwloc/1.11.11      libtool/2.4.6          numactl/2.0.12             py-setuptools/41.4.0    util-macros/1.19.1
   diffutils/3.7      libbsd/0.9.1       libxml2/2.9.9          openblas/0.3.7      (L)    python/3.7.4            xz/5.2.4
   expat/2.2.9        libffi/3.2.1       m4/1.4.18              openmpi/3.1.4              readline/8.0            zlib/1.2.11

---------------------------------------- /home/spack/spack/share/spack/lmod/linux-ubuntu18.04-x86_64/Core ----------------------------------------
   gcc/8.3.0 (L)

  Where:
   L:  Module is loaded

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".


$ module load openmpi
$ module avail

--------------- /home/spack/spack/share/spack/lmod/linux-ubuntu18.04-x86_64/openmpi/3.1.4-dorc4s4/openblas/0.3.7-ldv4b4h/gcc/8.3.0 ---------------
   netlib-scalapack/2.0.2

-------------------------- /home/spack/spack/share/spack/lmod/linux-ubuntu18.04-x86_64/openblas/0.3.7-ldv4b4h/gcc/8.3.0 --------------------------
   py-numpy/1.17.3    py-scipy/1.3.1

------------------------------------- /home/spack/spack/share/spack/lmod/linux-ubuntu18.04-x86_64/gcc/8.3.0 --------------------------------------
   autoconf/2.69      findutils/4.6.0    libiconv/1.16          mpich/3.3.1                openssl/1.1.1d          sqlite/3.30.1
   automake/1.16.1    gdbm/1.18.1        libpciaccess/0.13.5    ncurses/6.1                perl/5.30.0             tar/1.32
   bzip2/1.0.8        gettext/0.20.1     libsigsegv/2.12        netlib-lapack/3.8.0        pkgconf/1.6.3           texinfo/6.5
   cmake/3.15.4       hwloc/1.11.11      libtool/2.4.6          numactl/2.0.12             py-setuptools/41.4.0    util-macros/1.19.1
   diffutils/3.7      libbsd/0.9.1       libxml2/2.9.9          openblas/0.3.7      (L)    python/3.7.4            xz/5.2.4
   expat/2.2.9        libffi/3.2.1       m4/1.4.18              openmpi/3.1.4       (L)    readline/8.0            zlib/1.2.11

---------------------------------------- /home/spack/spack/share/spack/lmod/linux-ubuntu18.04-x86_64/Core ----------------------------------------
   gcc/8.3.0 (L)

  Where:
   L:  Module is loaded

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

Both MPI and LAPACK providers will now benefit from the same safety features:

$ module load py-numpy netlib-scalapack
$ module load mpich

Lmod is automatically replacing "openmpi/3.1.4" with "mpich/3.3.1".


Due to MODULEPATH changes, the following have been reloaded:
  1) netlib-scalapack/2.0.2

$ module load netlib-lapack

Lmod is automatically replacing "openblas/0.3.7" with "netlib-lapack/3.8.0".


Inactive Modules:
  1) py-numpy

Due to MODULEPATH changes, the following have been reloaded:
  1) netlib-scalapack/2.0.2

Because we only compiled py-numpy with openblas the module is made inactive when we switch the LAPACK provider. The user environment is now consistent by design!

Working with Templates

As briefly mentioned in the introduction, Spack uses Jinja2 to generate each individual module file. This means that you have all of its flexibility and power when it comes to customizing what gets generated!

Module file templates

The templates that Spack uses to generate module files are stored in the share/spack/templates/module directory within the Spack prefix, and they all share the same common structure. Usually, they start with a header that identifies the type of module being generated. In the case of hierarchical module files it’s:

-- -*- lua -*-
-- Module file created by spack (https://github.com/spack/spack) on {{ timestamp }}
--
-- {{ spec.short_spec }}
--

The statements within double curly brackets {{ ... }} denote expressions that will be evaluated and substituted at module generation time. The rest of the file is then divided into blocks that can be overridden or extended by users, if need be. Control structures , delimited by {% ... %}, are also permitted in the template language:

{% block environment %}
{% for command_name, cmd in environment_modifications %}
{% if command_name == 'PrependPath' %}
prepend_path("{{ cmd.name }}", "{{ cmd.value }}", "{{ cmd.separator }}")
{% elif command_name == 'AppendPath' %}
append_path("{{ cmd.name }}", "{{ cmd.value }}", "{{ cmd.separator }}")
{% elif command_name == 'RemovePath' %}
remove_path("{{ cmd.name }}", "{{ cmd.value }}", "{{ cmd.separator }}")
{% elif command_name == 'SetEnv' %}
setenv("{{ cmd.name }}", "{{ cmd.value }}")
{% elif command_name == 'UnsetEnv' %}
unsetenv("{{ cmd.name }}")
{% endif %}
{% endfor %}
{% endblock %}

The locations where Spack looks for templates are specified in config.yaml:

  # Locations where templates should be found
  template_dirs:
    - $spack/share/spack/templates

and can be extended by users to employ custom templates, as we’ll see next.

Extend the default templates

Let’s assume one of our software is protected by group membership: allowed users belong to the same linux group, and access is granted at group level. Wouldn’t it be nice if people that are not yet entitled to use it could receive a helpful message at module load time that tells them who to contact in your organization to be inserted in the group?

To automate the generation of module files with such site-specific behavior we’ll start by extending the list of locations where Spack looks for module files. Let’s create the file ${SPACK_ROOT}/etc/spack/config.yaml with the content:

config:
  template_dirs:
    - $HOME/.spack/templates

This tells Spack to also search another location when looking for template files. Next, we need to create our custom template extension in the folder listed above:

{% extends "modules/modulefile.lua" %}
{% block footer %}
-- Access is granted only to specific groups
if not isDir("{{ spec.prefix }}") then
    LmodError (
        "You don't have the necessary rights to run \"{{ spec.name }}\".\n\n",
        "\tPlease write an e-mail to 1234@foo.com if you need further information on how to get access to it.\n"
    )
end
{% endblock %}

Let’s name this file group-restricted.lua. The line:

{% extends "modules/modulefile.lua" %}

tells Jinja2 that we are reusing the standard template for hierarchical module files. The section:

{% block footer %}
-- Access is granted only to specific groups
if not isDir("{{ spec.prefix }}") then
    LmodError (
        "You don't have the necessary rights to run \"{{ spec.name }}\".\n\n",
        "\tPlease write an e-mail to 1234@foo.com if you need further information on how to get access to it.\n"
    )
end
{% endblock %}

overrides the footer block. Finally, we need to add a couple of lines in modules.yaml to tell Spack which specs need to use the new custom template. For the sake of illustration let’s assume it’s netlib-scalapack:

modules:
  enable::
    - lmod
  lmod:
    core_compilers:
      - 'gcc@7.4.0'
    hierarchy:
      - mpi
      - lapack
    hash_length: 0
    whitelist:
      - gcc
    blacklist:
      - '%gcc@7.4.0'
      - readline
    all:
      filter:
        environment_blacklist: ['CPATH', 'LIBRARY_PATH']
      environment:
        set:
          '{name}_ROOT': '{prefix}'
    openmpi:
      environment:
        set:
          SLURM_MPI_TYPE: pmi2
          OMPI_MCA_btl_openib_warn_default_gid_prefix: '0'
    netlib-scalapack:
      template: 'group-restricted.lua'

If we regenerate the module files one last time:

$ spack module lmod refresh -y netlib-scalapack
==> Regenerating lmod module files

we’ll find the following at the end of each netlib-scalapack module file:

-- Access is granted only to specific groups
if not isDir("/home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-8.3.0/netlib-scalapack-2.0.2-2p75lzqjbsnev7d2j2osgpkz7ib33oca") then
    LmodError (
        "You don't have the necessary rights to run \"netlib-scalapack\".\n\n",
        "\tPlease write an e-mail to 1234@foo.com if you need further information on how to get access to it.\n"
    )
end

and every user that doesn’t have access to the software will now be redirected to the right e-mail address where to ask for it!