Tutorial setup
If you have not done the prior sections, you’ll need to set Spack up like this:
git clone https://github.com/spack/spack
. spack/share/spack/setup-env.sh
spack tutorial
See the Basic Installation Tutorial for full details on setup. For more
help join us in the #tutorial
channel on Slack – get an
invitation at spackpm.herokuapp.com
Environments Tutorial¶
So far, we’ve shown you the basics of how to install and remove packages with Spack. You can use spack install to install packages, spack uninstall to remove them, and spack find to look at and query what is installed.
If you build a lot of software, or if you work on multiple projects,
managing everything in one place can be overwhelming. The default spack
find
output may contain many packages, but you may want to just focus
on packages for a particular project. Moreover, you may want to include
special configuration with your package groups, e.g., to build all the
packages in the same group the same way. Or, you may want to build up
development workflows around a subset of packages.
Spack environments provide a way to handle these problems.
Environment Basics¶
Let’s look at the output of spack find
at this point in the tutorial.
$ spack find
==> 63 installed packages
-- linux-ubuntu18.04-x86_64 / clang@6.0.0 -----------------------
zlib@1.2.11
-- linux-ubuntu18.04-x86_64 / gcc@7.5.0 -------------------------
autoconf@2.69 hdf5@1.10.7 libxml2@2.9.10 netcdf-c@4.7.4 suite-sparse@5.8.1
autoconf-archive@2019.01.06 hdf5@1.10.7 m4@1.4.18 netlib-scalapack@2.1.0 tcl@8.6.10
automake@1.16.2 hdf5@1.10.7 matio@1.5.17 netlib-scalapack@2.1.0 tcl@8.6.10
berkeley-db@18.1.40 hdf5@1.10.7 matio@1.5.17 numactl@2.0.14 texinfo@6.5
boost@1.74.0 hwloc@1.11.11 metis@5.1.0 openblas@0.3.12 trilinos@13.0.1
bzip2@1.0.8 hwloc@2.2.0 mpc@1.1.0 openmpi@3.1.6 util-macros@1.19.1
cmake@3.18.4 hypre@2.20.0 mpfr@3.1.6 openssl@1.1.1h xz@5.2.5
diffutils@3.7 hypre@2.20.0 mpfr@4.0.2 parmetis@4.0.3 zlib@1.2.8
findutils@4.6.0 isl@0.18 mpich@3.3.2 parmetis@4.0.3 zlib@1.2.8
gcc@8.3.0 libiconv@1.16 mumps@5.3.3 patchelf@0.10 zlib@1.2.11
gdbm@1.18.1 libpciaccess@0.16 mumps@5.3.3 perl@5.32.0
glm@0.9.7.1 libsigsegv@2.12 ncurses@6.2 pkgconf@1.7.3
gmp@6.1.2 libtool@2.4.6 netcdf-c@4.7.4 readline@8.0
This is a complete, but cluttered view. There are packages built with
both openmpi
and mpich
, as well as multiple variants of other
packages, like zlib
. The query mechanism we learned about in spack
find
can help, but it would be nice if we could start from a clean
slate without losing what we’ve already done.
Creating and activating environments¶
The spack env
command can help. Let’s create a new environment:
$ spack env create myproject
==> Updating view at /home/spack/spack/var/spack/environments/myproject/.spack-env/view
==> Created environment 'myproject' in /home/spack/spack/var/spack/environments/myproject
==> You can activate this environment with:
==> spack env activate myproject
An environment is a virtualized spack
instance that you can use for a
specific purpose. The environment also has an associated view, which
is a single prefix where all packages from the environment are linked.
You can see the environments we’ve created so far like this:
$ spack env list
==> 1 environments
myproject
And you can activate an environment with spack env activate
:
$ spack env activate myproject
Once you enter an environment, spack find
shows only what is in the
current environment. We just created this environment, so we have a
clean slate – 0 packages:
$ spack find
==> In environment myproject
==> No root specs
==> 0 installed packages
The spack find
output is still slightly different. It tells you
that you’re in the myproject
environment, so that you don’t panic
when you see that there is nothing installed. It also says that there
are no root specs. We’ll get back to what that means later.
If you only want to check what environment you are in, you can use
spack env status
:
$ spack env status
==> In environment myproject
If you want to leave this environment and go back to normal Spack,
you can use spack env deactivate
. We like to use the
despacktivate
alias (which Spack sets up automatically) for short:
$ despacktivate # short alias for `spack env deactivate`
$ spack env status
==> No active environment
$ spack find
==> 63 installed packages
-- linux-ubuntu18.04-x86_64 / clang@6.0.0 -----------------------
zlib@1.2.11
-- linux-ubuntu18.04-x86_64 / gcc@7.5.0 -------------------------
autoconf@2.69 hdf5@1.10.7 libxml2@2.9.10 netcdf-c@4.7.4 suite-sparse@5.8.1
autoconf-archive@2019.01.06 hdf5@1.10.7 m4@1.4.18 netlib-scalapack@2.1.0 tcl@8.6.10
automake@1.16.2 hdf5@1.10.7 matio@1.5.17 netlib-scalapack@2.1.0 tcl@8.6.10
berkeley-db@18.1.40 hdf5@1.10.7 matio@1.5.17 numactl@2.0.14 texinfo@6.5
boost@1.74.0 hwloc@1.11.11 metis@5.1.0 openblas@0.3.12 trilinos@13.0.1
bzip2@1.0.8 hwloc@2.2.0 mpc@1.1.0 openmpi@3.1.6 util-macros@1.19.1
cmake@3.18.4 hypre@2.20.0 mpfr@3.1.6 openssl@1.1.1h xz@5.2.5
diffutils@3.7 hypre@2.20.0 mpfr@4.0.2 parmetis@4.0.3 zlib@1.2.8
findutils@4.6.0 isl@0.18 mpich@3.3.2 parmetis@4.0.3 zlib@1.2.8
gcc@8.3.0 libiconv@1.16 mumps@5.3.3 patchelf@0.10 zlib@1.2.11
gdbm@1.18.1 libpciaccess@0.16 mumps@5.3.3 perl@5.32.0
glm@0.9.7.1 libsigsegv@2.12 ncurses@6.2 pkgconf@1.7.3
gmp@6.1.2 libtool@2.4.6 netcdf-c@4.7.4 readline@8.0
Phew – all of our packages are still installed.
Installing packages¶
Ok, now that we understand how creation and activation work, let’s go
back to myproject
and install a few packages:
$ spack env activate myproject
$ spack install tcl
==> All of the packages are already installed
$ spack install trilinos
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/libiconv-1.16-jearpk4xci4zc7dkrza4fufaqfkq7rfl
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/zlib-1.2.11-smoyzzo2qhzpn6mg6rd3l2p7b23enshg
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/pkgconf-1.7.3-4sh6pymrm2ms4auu3ajbjjr6fiuhz5g7
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/berkeley-db-18.1.40-4ihuiazsglf22f3pntq5hc4kyszqzexn
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/libsigsegv-2.12-lbrx7lnfz46ukewxbhxnucmx76g23c6q
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/util-macros-1.19.1-gs6ag7ktdoiirb62t7bcagjw62szrrg2
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/xz-5.2.5-komekkmyciga3kl24edjmredhj3uyt7v
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/openblas-0.3.12-te35tjxctgjpeeuk357xghoinvwf2k7u
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/autoconf-archive-2019.01.06-bdyarrkdgse5adcjr4kxqc4jns74k7yn
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/diffutils-3.7-otkktenrrvcaf37a7zyeytmjuwsalvtr
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/ncurses-6.2-crhlefo3dv7lmsv5pf4icsy4gepkdorm
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/m4-1.4.18-mkc3u4x2p2wie6jfhuku7g5rkovcrxps
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/libxml2-2.9.10-yn2r3wfhiilelyulh5toteicdtxjhw7d
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/bzip2-1.0.8-fvfpt26p5divvq6344vojyn64enojlui
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/readline-8.0-t54jzdy2jj4snltjazlm3br2urcilc6v
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/libtool-2.4.6-jdxbjftheiotj6solpomva7dowrhlerl
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/boost-1.74.0-hc5atqce4emwrrkxuyfayf5k43r5cst2
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/gdbm-1.18.1-4av4gywgpaspkhy3dvbb62nulqogtzbb
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/libpciaccess-0.16-bob4o5m3uku6vtdil5imasprgy775zg7
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/perl-5.32.0-zfdvt2jjuaees43ffrrtphqs2ky3o22t
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/autoconf-2.69-mm33a3ocsv3jsh2tfxc4mlab4xsurtdd
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/openssl-1.1.1h-es377uqsqougfc67jyg7yfjyyuukin52
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/automake-1.16.2-d2krmb5gweivlnztcymhklzsqbrpatt6
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/cmake-3.18.4-bltycqwh5oofai4f6o42q4uuj4w5zb3j
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/gmp-6.1.2-3ol3tldmm3oqflzrqonh6p2jmhgwsr4t
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/numactl-2.0.14-wbqbc5vw5sxzwhvu56p6x5nd5n4abrvh
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/metis-5.1.0-edcrft4l4szpqji2kowdkzq2ofueh3dr
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/glm-0.9.7.1-n4xti2gmzctc2oskcjphmruidrna4vfg
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/mpfr-4.0.2-mpv2v7v7dasis5nbrrnmci5tc2rrca2y
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/hwloc-1.11.11-zqwfzhw5k2ollygh6nrjpsi7u4d4g6lu
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/suite-sparse-5.8.1-rdfmg5b5lobcgmho4yqzd7zq7wibbelk
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/openmpi-3.1.6-pmsyupw6w3gql4loaor25gfumlmvkl25
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/hdf5-1.10.7-ejanivxt2gsv2qd3so7fil7izp4m3bcd
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/hypre-2.20.0-nndket2abxbsbuoocew5m643e36waxj3
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/parmetis-4.0.3-htdsyvt5yp6vl3nxbxp3627gohe7ldup
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/netlib-scalapack-2.1.0-wsiydak4jm7sms57zcu4rmzmdmgxqb5j
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/matio-1.5.17-zoqjxgkw7h3yoxbagripnd6muniihwno
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/netcdf-c-4.7.4-yixejlymvzs5aduzazrbs37hugw7sl7g
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/mumps-5.3.3-3jmjrzw7kx44fyoltswqx7uhu33zikhm
==> Installing trilinos-13.0.1-qmjbxf2kamskoplhqlejy3esksrquni2
==> Extracting trilinos-13.0.1-qmjbxf2kamskoplhqlejy3esksrquni2 from binary cache
gpgconf: socketdir is '/home/spack/.gnupg'
gpgconf: no /run/user dir
gpgconf: using homedir as fallback
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/trilinos-13.0.1-qmjbxf2kamskoplhqlejy3esksrquni2
==> Updating view at /home/spack/spack/var/spack/environments/myproject/.spack-env/view
$ spack find
==> In environment myproject
==> Root specs
tcl trilinos
==> 41 installed packages
-- linux-ubuntu18.04-x86_64 / gcc@7.5.0 -------------------------
autoconf@2.69 glm@0.9.7.1 libxml2@2.9.10 numactl@2.0.14 tcl@8.6.10
autoconf-archive@2019.01.06 gmp@6.1.2 m4@1.4.18 openblas@0.3.12 trilinos@13.0.1
automake@1.16.2 hdf5@1.10.7 matio@1.5.17 openmpi@3.1.6 util-macros@1.19.1
berkeley-db@18.1.40 hwloc@1.11.11 metis@5.1.0 openssl@1.1.1h xz@5.2.5
boost@1.74.0 hypre@2.20.0 mpfr@4.0.2 parmetis@4.0.3 zlib@1.2.11
bzip2@1.0.8 libiconv@1.16 mumps@5.3.3 perl@5.32.0
cmake@3.18.4 libpciaccess@0.16 ncurses@6.2 pkgconf@1.7.3
diffutils@3.7 libsigsegv@2.12 netcdf-c@4.7.4 readline@8.0
gdbm@1.18.1 libtool@2.4.6 netlib-scalapack@2.1.0 suite-sparse@5.8.1
We’ve installed tcl
and trilinos
in our environment, along with
all of their dependencies. We call tcl
and trilinos
the
roots because we asked for them explicitly. The other 20 packages
listed under “installed packages” are present because they were needed as
dependencies. So, these are the roots of the packages’ dependency graph.
Using packages¶
When you install packages into an environment, they are linked into a
single prefix, or a view. When you activate the environment with
spack env activate
, Spack adds subdirectories from the view to
PATH
, LD_LIBRARY_PATH
, CMAKE_PREFIX_PATH
and other
environment variables. This makes the environment easier to use.
Without environments, you need to spack load
or module load
a
package in order to use it. With environments, you can simply run
spack env activate
to get everything in the environment on your
PATH
.
Let’s try it out. myproject
is still the active environment, and we
just installed tcl
. You can see tclsh
in your PATH
immediately:
$ which tclsh
/home/spack/spack/var/spack/environments/myproject/.spack-env/view/bin/tclsh
And you can run it like you would any other program:
$ tclsh
% echo "hello world!"
hello world!
% exit
Likewise, we installed Trilinos, and you can run some of its sub-programs as well:
$ which algebra
/home/spack/spack/var/spack/environments/myproject/.spack-env/view/bin/algebra
$ algebra
AAAAA LL GGGGG EEEEEEE BBBBBB RRRRRR AAAAA
AA AA LL GG GG EE BB BB RR RR AA AA
AA AA LL GG EE BB BB RR RR AA AA
AAAAAAA LL GG EEEEE BBBBBB RRRRRR AAAAAAA
AA AA LL GG GGG EE BB BB RRRRR AA AA
AA AA LL GG GG EE BB BB RR RR AA AA
AA AA LLLLLLL GGGGG EEEEEEE BBBBBB RR RR AA AA
*** algebra Version 1.47 ***
Revised 2019/01/25
AN ALGEBRAIC MANIPULATION PROGRAM
FOR POST-PROCESSING OF FINITE ELEMENT ANALYSES
EXODUS II VERSION
Run on 2021-04-16 at 07:13:10
==== Email gdsjaar@sandia.gov for support ====
+++ Copyright 2008 NTESS +++
+++ Under the terms of Contract +++
+++ DE-NA0003525 with NTESS, the +++
+++ U.S. Government retains certain rights in this software. +++
FATAL ERROR - Filenames not specified.
FATAL ERROR - Syntax is: "algebra file_in file_out"
PROGRAM ERROR - Dynamic allocation problem - email code sponsor
0LAST ERROR/RETURN CODE: 1
0* * * * * E R R O R C O D E S * * * * *
0OCCURANCES FLAG MESSAGE TEXT
7 1 SUCCESSFUL COMPLETION
0 2 UNABLE TO GET REQUESTED SPACE FROM SYSTEM
0 3 DATA MANAGER NOT INITIALIZED
0 4 DATA MANAGER WAS PREVIOUSLY INITIALIZED
10 5 NAME NOT FOUND IN DICTIONARY
0 6 NAME ALREADY EXISTS IN DICTIONARY
0 7 ILLEGAL LENGTH REQUEST
0 8 UNKNOWN DATA TYPE
0 9 DICTIONARY IS FULL
0 10 VOID TABLE IS FULL
0 11 MEMORY BLOCK TABLE IS FULL
0 12 OVERLAPPING VOIDS - INTERNAL ERROR
0 13 OVERLAPPING MEMORY BLOCKS - INTERNAL ERROR
0 14 INVALID MEMORY BLOCK - EXTENSION LIBRARY ERROR
0 15 INVALID ERROR CODE
0 16 INVALID INPUT NAME
0 17 ILLEGAL CALL WHILE IN DEFERRED MODE
0 18 NAME IS OF WRONG TYPE FOR OPERATION
algebra used 0.01 seconds of CPU time
Uninstalling packages¶
Now let’s create another project. We’ll call this one myproject2
:
$ spack env create myproject2
==> Updating view at /home/spack/spack/var/spack/environments/myproject2/.spack-env/view
==> Created environment 'myproject2' in /home/spack/spack/var/spack/environments/myproject2
==> You can activate this environment with:
==> spack env activate myproject2
$ spack env activate myproject2
$ spack install hdf5+hl
==> All of the packages are already installed
$ spack install trilinos
==> All of the packages are already installed
$ spack find
==> In environment myproject2
==> Root specs
hdf5 +hl trilinos
==> 40 installed packages
-- linux-ubuntu18.04-x86_64 / gcc@7.5.0 -------------------------
autoconf@2.69 gdbm@1.18.1 libsigsegv@2.12 ncurses@6.2 perl@5.32.0
autoconf-archive@2019.01.06 glm@0.9.7.1 libtool@2.4.6 netcdf-c@4.7.4 pkgconf@1.7.3
automake@1.16.2 gmp@6.1.2 libxml2@2.9.10 netlib-scalapack@2.1.0 readline@8.0
berkeley-db@18.1.40 hdf5@1.10.7 m4@1.4.18 numactl@2.0.14 suite-sparse@5.8.1
boost@1.74.0 hwloc@1.11.11 matio@1.5.17 openblas@0.3.12 trilinos@13.0.1
bzip2@1.0.8 hypre@2.20.0 metis@5.1.0 openmpi@3.1.6 util-macros@1.19.1
cmake@3.18.4 libiconv@1.16 mpfr@4.0.2 openssl@1.1.1h xz@5.2.5
diffutils@3.7 libpciaccess@0.16 mumps@5.3.3 parmetis@4.0.3 zlib@1.2.11
Now we have two environments: one with tcl
and trilinos
, and
another with hdf5 +hl
and trilinos
. Notice that the roots display exactly as
we asked for them on the command line – the hdf5
for this environemnt has an
+hl
requirement.
We can uninstall trilinos from myproject2
as you would expect:
$ spack uninstall trilinos
y
==> The following packages will be uninstalled:
-- linux-ubuntu18.04-x86_64 / gcc@7.5.0 -------------------------
qmjbxf2 trilinos@13.0.1
==> Do you want to proceed? [y/N] ==> Updating view at /home/spack/spack/var/spack/environments/myproject2/.spack-env/view
$ spack find
==> In environment myproject2
==> Root specs
hdf5 +hl
==> 21 installed packages
-- linux-ubuntu18.04-x86_64 / gcc@7.5.0 -------------------------
autoconf@2.69 hdf5@1.10.7 libsigsegv@2.12 ncurses@6.2 pkgconf@1.7.3 zlib@1.2.11
automake@1.16.2 hwloc@1.11.11 libtool@2.4.6 numactl@2.0.14 readline@8.0
berkeley-db@18.1.40 libiconv@1.16 libxml2@2.9.10 openmpi@3.1.6 util-macros@1.19.1
gdbm@1.18.1 libpciaccess@0.16 m4@1.4.18 perl@5.32.0 xz@5.2.5
Now there is only one root spec, hdf5 +hl
, which requires fewer
additional dependencies.
However, we still needed trilinos
for the myproject
environment!
What happened to it? Let’s switch back and see.
$ despacktivate
$ spack env activate myproject
$ spack find
==> In environment myproject
==> Root specs
tcl trilinos
==> 41 installed packages
-- linux-ubuntu18.04-x86_64 / gcc@7.5.0 -------------------------
autoconf@2.69 glm@0.9.7.1 libxml2@2.9.10 numactl@2.0.14 tcl@8.6.10
autoconf-archive@2019.01.06 gmp@6.1.2 m4@1.4.18 openblas@0.3.12 trilinos@13.0.1
automake@1.16.2 hdf5@1.10.7 matio@1.5.17 openmpi@3.1.6 util-macros@1.19.1
berkeley-db@18.1.40 hwloc@1.11.11 metis@5.1.0 openssl@1.1.1h xz@5.2.5
boost@1.74.0 hypre@2.20.0 mpfr@4.0.2 parmetis@4.0.3 zlib@1.2.11
bzip2@1.0.8 libiconv@1.16 mumps@5.3.3 perl@5.32.0
cmake@3.18.4 libpciaccess@0.16 ncurses@6.2 pkgconf@1.7.3
diffutils@3.7 libsigsegv@2.12 netcdf-c@4.7.4 readline@8.0
gdbm@1.18.1 libtool@2.4.6 netlib-scalapack@2.1.0 suite-sparse@5.8.1
Spack is smart enough to realize that trilinos
is still present in
the other environment. Trilinos won’t actually be uninstalled unless
it is no longer needed by any environments or packages. If it is still
needed, it is only removed from the environment.
Dealing with Many Specs at Once¶
In the above examples, we just used install
and uninstall
. There
are other ways to deal with groups of packages, as well.
Adding specs¶
While we’re still in myproject
, let’s add a few specs instead of installing them:
$ spack add hdf5+hl
==> Adding hdf5+hl to environment myproject
==> Updating view at /home/spack/spack/var/spack/environments/myproject/.spack-env/view
$ spack add gmp
==> Adding gmp to environment myproject
==> Updating view at /home/spack/spack/var/spack/environments/myproject/.spack-env/view
$ spack find
==> In environment myproject
==> Root specs
gmp hdf5 +hl tcl trilinos
==> 41 installed packages
-- linux-ubuntu18.04-x86_64 / gcc@7.5.0 -------------------------
autoconf@2.69 glm@0.9.7.1 libxml2@2.9.10 numactl@2.0.14 tcl@8.6.10
autoconf-archive@2019.01.06 gmp@6.1.2 m4@1.4.18 openblas@0.3.12 trilinos@13.0.1
automake@1.16.2 hdf5@1.10.7 matio@1.5.17 openmpi@3.1.6 util-macros@1.19.1
berkeley-db@18.1.40 hwloc@1.11.11 metis@5.1.0 openssl@1.1.1h xz@5.2.5
boost@1.74.0 hypre@2.20.0 mpfr@4.0.2 parmetis@4.0.3 zlib@1.2.11
bzip2@1.0.8 libiconv@1.16 mumps@5.3.3 perl@5.32.0
cmake@3.18.4 libpciaccess@0.16 ncurses@6.2 pkgconf@1.7.3
diffutils@3.7 libsigsegv@2.12 netcdf-c@4.7.4 readline@8.0
gdbm@1.18.1 libtool@2.4.6 netlib-scalapack@2.1.0 suite-sparse@5.8.1
Let’s take a close look at what happened. The two requirements we added,
hdf5 +hl
and gmp
, are present, but they’re not installed in the
environment yet. spack add
just adds roots to the environment, but
it does not automatically install them.
We can install all the as-yet uninstalled packages in an environment by
simply running spack install
with no arguments:
$ spack install
==> Concretized hdf5+hl
[+] ejanivx hdf5@1.10.7%gcc@7.5.0~cxx~debug~fortran+hl~java+mpi+pic+shared~szip~threadsafe api=none arch=linux-ubuntu18.04-x86_64
[+] pmsyupw ^openmpi@3.1.6%gcc@7.5.0~atomics~cuda~cxx~cxx_exceptions+gpfs~java~legacylaunchers~lustre~memchecker~pmi~singularity~sqlite3+static~thread_multiple+vt+wrapper-rpath fabrics=none schedulers=none arch=linux-ubuntu18.04-x86_64
[+] zqwfzhw ^hwloc@1.11.11%gcc@7.5.0~cairo~cuda~gl~libudev+libxml2~netloc~nvml+pci+shared arch=linux-ubuntu18.04-x86_64
[+] bob4o5m ^libpciaccess@0.16%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] jdxbjft ^libtool@2.4.6%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] mkc3u4x ^m4@1.4.18%gcc@7.5.0+sigsegv patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 arch=linux-ubuntu18.04-x86_64
[+] lbrx7ln ^libsigsegv@2.12%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] 4sh6pym ^pkgconf@1.7.3%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] gs6ag7k ^util-macros@1.19.1%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] yn2r3wf ^libxml2@2.9.10%gcc@7.5.0~python arch=linux-ubuntu18.04-x86_64
[+] jearpk4 ^libiconv@1.16%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] komekkm ^xz@5.2.5%gcc@7.5.0~pic arch=linux-ubuntu18.04-x86_64
[+] smoyzzo ^zlib@1.2.11%gcc@7.5.0+optimize+pic+shared arch=linux-ubuntu18.04-x86_64
[+] wbqbc5v ^numactl@2.0.14%gcc@7.5.0 patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94 arch=linux-ubuntu18.04-x86_64
[+] mm33a3o ^autoconf@2.69%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] zfdvt2j ^perl@5.32.0%gcc@7.5.0+cpanm+shared+threads arch=linux-ubuntu18.04-x86_64
[+] 4ihuiaz ^berkeley-db@18.1.40%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] 4av4gyw ^gdbm@1.18.1%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] t54jzdy ^readline@8.0%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] crhlefo ^ncurses@6.2%gcc@7.5.0~symlinks+termlib arch=linux-ubuntu18.04-x86_64
[+] d2krmb5 ^automake@1.16.2%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
==> Concretized gmp
[+] 3ol3tld gmp@6.1.2%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] mm33a3o ^autoconf@2.69%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] mkc3u4x ^m4@1.4.18%gcc@7.5.0+sigsegv patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 arch=linux-ubuntu18.04-x86_64
[+] lbrx7ln ^libsigsegv@2.12%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] zfdvt2j ^perl@5.32.0%gcc@7.5.0+cpanm+shared+threads arch=linux-ubuntu18.04-x86_64
[+] 4ihuiaz ^berkeley-db@18.1.40%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] 4av4gyw ^gdbm@1.18.1%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] t54jzdy ^readline@8.0%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] crhlefo ^ncurses@6.2%gcc@7.5.0~symlinks+termlib arch=linux-ubuntu18.04-x86_64
[+] 4sh6pym ^pkgconf@1.7.3%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] d2krmb5 ^automake@1.16.2%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] jdxbjft ^libtool@2.4.6%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
==> Installing environment myproject
==> All of the packages are already installed
==> Updating view at /home/spack/spack/var/spack/environments/myproject/.spack-env/view
Spack will concretize the new roots, and install everything you added to
the environment. Now we can see the installed roots in the output of
spack find
:
$ spack find
==> In environment myproject
==> Root specs
gmp hdf5 +hl tcl trilinos
==> 41 installed packages
-- linux-ubuntu18.04-x86_64 / gcc@7.5.0 -------------------------
autoconf@2.69 glm@0.9.7.1 libxml2@2.9.10 numactl@2.0.14 tcl@8.6.10
autoconf-archive@2019.01.06 gmp@6.1.2 m4@1.4.18 openblas@0.3.12 trilinos@13.0.1
automake@1.16.2 hdf5@1.10.7 matio@1.5.17 openmpi@3.1.6 util-macros@1.19.1
berkeley-db@18.1.40 hwloc@1.11.11 metis@5.1.0 openssl@1.1.1h xz@5.2.5
boost@1.74.0 hypre@2.20.0 mpfr@4.0.2 parmetis@4.0.3 zlib@1.2.11
bzip2@1.0.8 libiconv@1.16 mumps@5.3.3 perl@5.32.0
cmake@3.18.4 libpciaccess@0.16 ncurses@6.2 pkgconf@1.7.3
diffutils@3.7 libsigsegv@2.12 netcdf-c@4.7.4 readline@8.0
gdbm@1.18.1 libtool@2.4.6 netlib-scalapack@2.1.0 suite-sparse@5.8.1
We can build whole environments this way, by adding specs and installing
all at once, or we can install them with the usual install
and
uninstall
portions. The advantage to doing them all at once is that
we don’t have to write a script outside of Spack to automate this, and we
can kick off a large build of many packages easily.
Configuration¶
An environment is more than just a list of root specs. It can also include
configuration that changes the way Spack behaves when the environment is
activated. So far, myproject
does not have any special configuration
associated with it. If you run spack spec
, you can see that concretization
looks the same as it does outside the environment:
$ spack spec hypre
Input spec
--------------------------------
hypre
Concretized
--------------------------------
hypre@2.20.0%gcc@7.5.0~complex~debug~int64~internal-superlu~mixedint+mpi~openmp+shared~superlu-dist patches=6e3336b1d62155f6350dfe42b0f9ea25d4fa0af60c7e540959139deb93a26059 arch=linux-ubuntu18.04-x86_64
^openblas@0.3.12%gcc@7.5.0~consistent_fpcsr~ilp64+pic+shared threads=none arch=linux-ubuntu18.04-x86_64
^openmpi@3.1.6%gcc@7.5.0~atomics~cuda~cxx~cxx_exceptions+gpfs~java~legacylaunchers~lustre~memchecker~pmi~singularity~sqlite3+static~thread_multiple+vt+wrapper-rpath fabrics=none schedulers=none arch=linux-ubuntu18.04-x86_64
^hwloc@1.11.11%gcc@7.5.0~cairo~cuda~gl~libudev+libxml2~netloc~nvml+pci+shared arch=linux-ubuntu18.04-x86_64
^libpciaccess@0.16%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
^libtool@2.4.6%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
^m4@1.4.18%gcc@7.5.0+sigsegv patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 arch=linux-ubuntu18.04-x86_64
^libsigsegv@2.12%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
^pkgconf@1.7.3%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
^util-macros@1.19.1%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
^libxml2@2.9.10%gcc@7.5.0~python arch=linux-ubuntu18.04-x86_64
^libiconv@1.16%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
^xz@5.2.5%gcc@7.5.0~pic arch=linux-ubuntu18.04-x86_64
^zlib@1.2.11%gcc@7.5.0+optimize+pic+shared arch=linux-ubuntu18.04-x86_64
^numactl@2.0.14%gcc@7.5.0 patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94 arch=linux-ubuntu18.04-x86_64
^autoconf@2.69%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
^perl@5.32.0%gcc@7.5.0+cpanm+shared+threads arch=linux-ubuntu18.04-x86_64
^berkeley-db@18.1.40%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
^gdbm@1.18.1%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
^readline@8.0%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
^ncurses@6.2%gcc@7.5.0~symlinks+termlib arch=linux-ubuntu18.04-x86_64
^automake@1.16.2%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
We can customize this using concretization preferences – special configuration that changes the behavior of the concretizer.
Let’s start by looking at the configuration of your environment. You can see
how your environment is configured using spack config get
:
$ spack config get
# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
# add package specs to the `specs` list
specs: [tcl, trilinos, hdf5+hl, gmp]
view: true
The output shows the special YAML configuration format that Spack uses to store
the state of your environment. Currently, the file is just a spack:
header
and a list of specs
. These are the roots.
You can edit this file to add your own custom configuration. Spack provides a shortcut to do that:
spack config edit
You should now see the same file in your editor. Change it to look like this:
# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
packages:
all:
providers:
mpi: [mpich]
# add package specs to the `specs` list
specs: [tcl, trilinos, hdf5, gmp]
We will learn much more in the configuration section,
but for now, all you need to know is that this changes the default mpi
provider. That is, if a package depends on mpi
, Spack will now satisfy that
dependency with mpich
instead of the default openmpi
.
To see what this looks like, run spack spec
again in the environment. You
can see that the spec concretizes with mpich
as the MPI implementation:
$ spack spec hypre
Input spec
--------------------------------
hypre
Concretized
--------------------------------
hypre@2.20.0%gcc@7.5.0~complex~debug~int64~internal-superlu~mixedint+mpi~openmp+shared~superlu-dist patches=6e3336b1d62155f6350dfe42b0f9ea25d4fa0af60c7e540959139deb93a26059 arch=linux-ubuntu18.04-x86_64
^mpich@3.3.2%gcc@7.5.0~argobots+fortran+hwloc+hydra+libxml2+pci+romio~slurm~verbs+wrapperrpath device=ch3 netmod=tcp patches=eb982de3366d48cbc55eb5e0df43373a45d9f51df208abf0835a72dc6c0b4774 pmi=pmi arch=linux-ubuntu18.04-x86_64
^autoconf@2.69%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
^m4@1.4.18%gcc@7.5.0+sigsegv patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 arch=linux-ubuntu18.04-x86_64
^libsigsegv@2.12%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
^perl@5.32.0%gcc@7.5.0+cpanm+shared+threads arch=linux-ubuntu18.04-x86_64
^berkeley-db@18.1.40%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
^gdbm@1.18.1%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
^readline@8.0%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
^ncurses@6.2%gcc@7.5.0~symlinks+termlib arch=linux-ubuntu18.04-x86_64
^pkgconf@1.7.3%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
^automake@1.16.2%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
^findutils@4.6.0%gcc@7.5.0 patches=84b916c0bf8c51b7e7b28417692f0ad3e7030d1f3c248ba77c42ede5c1c5d11e,bd9e4e5cc280f9753ae14956c4e4aa17fe7a210f55dd6c84aa60b12d106d47a2 arch=linux-ubuntu18.04-x86_64
^libtool@2.4.6%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
^texinfo@6.5%gcc@7.5.0 patches=12f6edb0c6b270b8c8dba2ce17998c580db01182d871ee32b7b6e4129bd1d23a,1732115f651cff98989cb0215d8f64da5e0f7911ebf0c13b064920f088f2ffe1 arch=linux-ubuntu18.04-x86_64
^hwloc@2.2.0%gcc@7.5.0~cairo~cuda~gl~libudev+libxml2~netloc~nvml+pci+shared arch=linux-ubuntu18.04-x86_64
^libpciaccess@0.16%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
^util-macros@1.19.1%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
^libxml2@2.9.10%gcc@7.5.0~python arch=linux-ubuntu18.04-x86_64
^libiconv@1.16%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
^xz@5.2.5%gcc@7.5.0~pic arch=linux-ubuntu18.04-x86_64
^zlib@1.2.11%gcc@7.5.0+optimize+pic+shared arch=linux-ubuntu18.04-x86_64
^openblas@0.3.12%gcc@7.5.0~consistent_fpcsr~ilp64+pic+shared threads=none arch=linux-ubuntu18.04-x86_64
In addition to the packages
section, an environment can contain many other
types of configuration. You can add custom package repositories, a custom
install location, custom compilers, custom external packages, and more. There
are also ways to build workflows for CI and for software development using
environment configuration, which we’ll learn later in the tutorial.
Right now, though, we have a problem. We already installed part of this
environment with openmpi, but now we want to install everything with mpich
.
You can run spack concretize --force
inside of an environment to concretize
all of its specs. We can run it here:
$ spack concretize --force
==> Concretized tcl
[+] rtz7664 tcl@8.6.10%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] smoyzzo ^zlib@1.2.11%gcc@7.5.0+optimize+pic+shared arch=linux-ubuntu18.04-x86_64
==> Concretized trilinos
[+] xvzpvxd trilinos@13.0.1%gcc@7.5.0~adios2~alloptpkgs+amesos+amesos2+anasazi+aztec+belos+boost~cgns~chaco~complex~cuda~debug~dtk+epetra+epetraext+exodus+explicit_template_instantiation~float+fortran+glm+gtest+hdf5~hwloc+hypre+ifpack+ifpack2~intrepid~intrepid2~ipo~isorropia+kokkos+matio~mesquite+metis~minitensor+ml+mpi+muelu+mumps+netcdf~nox~openmp~phalanx~piro~pnetcdf~python~rol~rythmos+sacado~shards+shared~shylu~stk~stratimikos~strumpack+suite-sparse~superlu~superlu-dist~teko~tempus+teuchos+tpetra~wrapper~x11~xsdkflags~zlib+zoltan+zoltan2 build_type=RelWithDebInfo cuda_arch=none cxxstd=11 gotype=long arch=linux-ubuntu18.04-x86_64
[+] hc5atqc ^boost@1.74.0%gcc@7.5.0+atomic+chrono~clanglibcpp~container~context~coroutine+date_time~debug+exception~fiber+filesystem+graph~icu+iostreams+locale+log+math~mpi+multithreaded~numpy~pic+program_options~python+random+regex+serialization+shared+signals~singlethreaded+system~taggedlayout+test+thread+timer~versionedlayout+wave cxxstd=98 visibility=hidden arch=linux-ubuntu18.04-x86_64
[+] fvfpt26 ^bzip2@1.0.8%gcc@7.5.0+shared arch=linux-ubuntu18.04-x86_64
[+] otkkten ^diffutils@3.7%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] jearpk4 ^libiconv@1.16%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] smoyzzo ^zlib@1.2.11%gcc@7.5.0+optimize+pic+shared arch=linux-ubuntu18.04-x86_64
[+] bltycqw ^cmake@3.18.4%gcc@7.5.0~doc+ncurses+openssl+ownlibs~qt patches=bf695e3febb222da2ed94b3beea600650e4318975da90e4a71d6f31a6d5d8c3d arch=linux-ubuntu18.04-x86_64
[+] crhlefo ^ncurses@6.2%gcc@7.5.0~symlinks+termlib arch=linux-ubuntu18.04-x86_64
[+] 4sh6pym ^pkgconf@1.7.3%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] es377uq ^openssl@1.1.1h%gcc@7.5.0+systemcerts arch=linux-ubuntu18.04-x86_64
[+] zfdvt2j ^perl@5.32.0%gcc@7.5.0+cpanm+shared+threads arch=linux-ubuntu18.04-x86_64
[+] 4ihuiaz ^berkeley-db@18.1.40%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] 4av4gyw ^gdbm@1.18.1%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] t54jzdy ^readline@8.0%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] n4xti2g ^glm@0.9.7.1%gcc@7.5.0~ipo build_type=RelWithDebInfo arch=linux-ubuntu18.04-x86_64
[+] cq4k4cv ^hdf5@1.10.7%gcc@7.5.0~cxx~debug~fortran+hl~java+mpi+pic+shared~szip~threadsafe api=none arch=linux-ubuntu18.04-x86_64
[+] vbelg5s ^mpich@3.3.2%gcc@7.5.0~argobots+fortran+hwloc+hydra+libxml2+pci+romio~slurm~verbs+wrapperrpath device=ch3 netmod=tcp patches=eb982de3366d48cbc55eb5e0df43373a45d9f51df208abf0835a72dc6c0b4774 pmi=pmi arch=linux-ubuntu18.04-x86_64
[+] mm33a3o ^autoconf@2.69%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] mkc3u4x ^m4@1.4.18%gcc@7.5.0+sigsegv patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 arch=linux-ubuntu18.04-x86_64
[+] lbrx7ln ^libsigsegv@2.12%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] d2krmb5 ^automake@1.16.2%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] hrv4dx5 ^findutils@4.6.0%gcc@7.5.0 patches=84b916c0bf8c51b7e7b28417692f0ad3e7030d1f3c248ba77c42ede5c1c5d11e,bd9e4e5cc280f9753ae14956c4e4aa17fe7a210f55dd6c84aa60b12d106d47a2 arch=linux-ubuntu18.04-x86_64
[+] jdxbjft ^libtool@2.4.6%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] p46ba5q ^texinfo@6.5%gcc@7.5.0 patches=12f6edb0c6b270b8c8dba2ce17998c580db01182d871ee32b7b6e4129bd1d23a,1732115f651cff98989cb0215d8f64da5e0f7911ebf0c13b064920f088f2ffe1 arch=linux-ubuntu18.04-x86_64
[+] a4cxlu7 ^hwloc@2.2.0%gcc@7.5.0~cairo~cuda~gl~libudev+libxml2~netloc~nvml+pci+shared arch=linux-ubuntu18.04-x86_64
[+] bob4o5m ^libpciaccess@0.16%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] gs6ag7k ^util-macros@1.19.1%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] yn2r3wf ^libxml2@2.9.10%gcc@7.5.0~python arch=linux-ubuntu18.04-x86_64
[+] komekkm ^xz@5.2.5%gcc@7.5.0~pic arch=linux-ubuntu18.04-x86_64
[+] us6giii ^hypre@2.20.0%gcc@7.5.0~complex~debug~int64~internal-superlu~mixedint+mpi~openmp+shared~superlu-dist patches=6e3336b1d62155f6350dfe42b0f9ea25d4fa0af60c7e540959139deb93a26059 arch=linux-ubuntu18.04-x86_64
[+] te35tjx ^openblas@0.3.12%gcc@7.5.0~consistent_fpcsr~ilp64+pic+shared threads=none arch=linux-ubuntu18.04-x86_64
[+] usmrkqu ^matio@1.5.17%gcc@7.5.0+hdf5+shared+zlib arch=linux-ubuntu18.04-x86_64
[+] edcrft4 ^metis@5.1.0%gcc@7.5.0~gdb~int64~real64+shared build_type=Release patches=4991da938c1d3a1d3dea78e49bbebecba00273f98df2a656e38b83d55b281da1,b1225da886605ea558db7ac08dd8054742ea5afe5ed61ad4d0fe7a495b1270d2 arch=linux-ubuntu18.04-x86_64
[+] rmlof2f ^mumps@5.3.3%gcc@7.5.0+complex+double+float~int64~metis+mpi~parmetis~ptscotch~scotch+shared arch=linux-ubuntu18.04-x86_64
[+] h7suxc3 ^netlib-scalapack@2.1.0%gcc@7.5.0~ipo~pic+shared build_type=Release patches=1c9ce5fee1451a08c2de3cc87f446aeda0b818ebbce4ad0d980ddf2f2a0b2dc4,f2baedde688ffe4c20943c334f580eb298e04d6f35c86b90a1f4e8cb7ae344a2 arch=linux-ubuntu18.04-x86_64
[+] spkgtxp ^netcdf-c@4.7.4%gcc@7.5.0~dap~hdf4~jna+mpi~parallel-netcdf+pic+shared arch=linux-ubuntu18.04-x86_64
[+] i7otoak ^parmetis@4.0.3%gcc@7.5.0~gdb~int64~ipo+shared build_type=RelWithDebInfo patches=4f892531eb0a807eb1b82e683a416d3e35154a455274cf9b162fb02054d11a5b,50ed2081bc939269689789942067c58b3e522c269269a430d5d34c00edbc5870,704b84f7c7444d4372cb59cca6e1209df4ef3b033bc4ee3cf50f369bce972a9d arch=linux-ubuntu18.04-x86_64
[+] rdfmg5b ^suite-sparse@5.8.1%gcc@7.5.0~cuda~openmp+pic~tbb arch=linux-ubuntu18.04-x86_64
[+] 3ol3tld ^gmp@6.1.2%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] mpv2v7v ^mpfr@4.0.2%gcc@7.5.0 patches=3f80b836948aa96f8d1cb9cc7f3f55973f19285482a96f9a4e1623d460bcccf0 arch=linux-ubuntu18.04-x86_64
[+] bdyarrk ^autoconf-archive@2019.01.06%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
==> Concretized hdf5+hl
[+] cq4k4cv hdf5@1.10.7%gcc@7.5.0~cxx~debug~fortran+hl~java+mpi+pic+shared~szip~threadsafe api=none arch=linux-ubuntu18.04-x86_64
[+] vbelg5s ^mpich@3.3.2%gcc@7.5.0~argobots+fortran+hwloc+hydra+libxml2+pci+romio~slurm~verbs+wrapperrpath device=ch3 netmod=tcp patches=eb982de3366d48cbc55eb5e0df43373a45d9f51df208abf0835a72dc6c0b4774 pmi=pmi arch=linux-ubuntu18.04-x86_64
[+] mm33a3o ^autoconf@2.69%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] mkc3u4x ^m4@1.4.18%gcc@7.5.0+sigsegv patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 arch=linux-ubuntu18.04-x86_64
[+] lbrx7ln ^libsigsegv@2.12%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] zfdvt2j ^perl@5.32.0%gcc@7.5.0+cpanm+shared+threads arch=linux-ubuntu18.04-x86_64
[+] 4ihuiaz ^berkeley-db@18.1.40%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] 4av4gyw ^gdbm@1.18.1%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] t54jzdy ^readline@8.0%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] crhlefo ^ncurses@6.2%gcc@7.5.0~symlinks+termlib arch=linux-ubuntu18.04-x86_64
[+] 4sh6pym ^pkgconf@1.7.3%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] d2krmb5 ^automake@1.16.2%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] hrv4dx5 ^findutils@4.6.0%gcc@7.5.0 patches=84b916c0bf8c51b7e7b28417692f0ad3e7030d1f3c248ba77c42ede5c1c5d11e,bd9e4e5cc280f9753ae14956c4e4aa17fe7a210f55dd6c84aa60b12d106d47a2 arch=linux-ubuntu18.04-x86_64
[+] jdxbjft ^libtool@2.4.6%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] p46ba5q ^texinfo@6.5%gcc@7.5.0 patches=12f6edb0c6b270b8c8dba2ce17998c580db01182d871ee32b7b6e4129bd1d23a,1732115f651cff98989cb0215d8f64da5e0f7911ebf0c13b064920f088f2ffe1 arch=linux-ubuntu18.04-x86_64
[+] a4cxlu7 ^hwloc@2.2.0%gcc@7.5.0~cairo~cuda~gl~libudev+libxml2~netloc~nvml+pci+shared arch=linux-ubuntu18.04-x86_64
[+] bob4o5m ^libpciaccess@0.16%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] gs6ag7k ^util-macros@1.19.1%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] yn2r3wf ^libxml2@2.9.10%gcc@7.5.0~python arch=linux-ubuntu18.04-x86_64
[+] jearpk4 ^libiconv@1.16%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] komekkm ^xz@5.2.5%gcc@7.5.0~pic arch=linux-ubuntu18.04-x86_64
[+] smoyzzo ^zlib@1.2.11%gcc@7.5.0+optimize+pic+shared arch=linux-ubuntu18.04-x86_64
==> Concretized gmp
[+] 3ol3tld gmp@6.1.2%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] mm33a3o ^autoconf@2.69%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] mkc3u4x ^m4@1.4.18%gcc@7.5.0+sigsegv patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 arch=linux-ubuntu18.04-x86_64
[+] lbrx7ln ^libsigsegv@2.12%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] zfdvt2j ^perl@5.32.0%gcc@7.5.0+cpanm+shared+threads arch=linux-ubuntu18.04-x86_64
[+] 4ihuiaz ^berkeley-db@18.1.40%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] 4av4gyw ^gdbm@1.18.1%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] t54jzdy ^readline@8.0%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] crhlefo ^ncurses@6.2%gcc@7.5.0~symlinks+termlib arch=linux-ubuntu18.04-x86_64
[+] 4sh6pym ^pkgconf@1.7.3%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] d2krmb5 ^automake@1.16.2%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] jdxbjft ^libtool@2.4.6%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
==> Updating view at /home/spack/spack/var/spack/environments/myproject/.spack-env/view
Now, all the specs in the environment are concrete and ready to be
installed with mpich
as the MPI implementation.
Normally, we could just run spack config edit
, edit the environment
configuration, spack add
some specs, and spack install
.
But, when we already have installed packages in the environment, we have
to force everything in the environment to be re-concretized using spack
concretize --force
. Then we can re-run spack install
.
Building in environments¶
You can use environments to set up a development environment. With the
environment activated, you can invoke any programs installed in the
environment. Suppose you wanted to compile some MPI programs. This environment
happens to have mpicc
installed:
$ spack env status
==> In environment myproject
$ which mpicc
/home/spack/spack/var/spack/environments/myproject/.spack-env/view/bin/mpicc
In addition, activating the environment has set variables like CPATH
,
LIBRARY_PATH
, and LD_LIBRARY_PATH
, so that you can easily find headers
and libraries from programs in the environemnt.
$ env | grep PATH=
LIBRARY_PATH=/home/spack/spack/var/spack/environments/myproject/.spack-env/view/lib:/home/spack/spack/var/spack/environments/myproject/.spack-env/view/lib64
C_INCLUDE_PATH=/home/spack/spack/var/spack/environments/myproject/.spack-env/view/include
LD_LIBRARY_PATH=/home/spack/spack/var/spack/environments/myproject/.spack-env/view/lib:/home/spack/spack/var/spack/environments/myproject/.spack-env/view/lib64
CPLUS_INCLUDE_PATH=/home/spack/spack/var/spack/environments/myproject/.spack-env/view/include
PKG_CONFIG_PATH=/home/spack/spack/var/spack/environments/myproject/.spack-env/view/lib/pkgconfig:/home/spack/spack/var/spack/environments/myproject/.spack-env/view/share/pkgconfig:/home/spack/spack/var/spack/environments/myproject/.spack-env/view/lib64/pkgconfig
ACLOCAL_PATH=/home/spack/spack/var/spack/environments/myproject/.spack-env/view/share/aclocal
PATH=/home/spack/spack/var/spack/environments/myproject/.spack-env/view/bin:/home/spack/spack/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PYTHONPATH=/home/spack/spack/var/spack/environments/myproject/.spack-env/view/lib
SPACK_LD_LIBRARY_PATH=/home/spack/spack/var/spack/environments/myproject/.spack-env/view/lib:/home/spack/spack/var/spack/environments/myproject/.spack-env/view/lib64
MANPATH=/home/spack/spack/var/spack/environments/myproject/.spack-env/view/share/man:/home/spack/spack/var/spack/environments/myproject/.spack-env/view/man
MODULEPATH=/home/spack/spack/share/spack/modules/linux-ubuntu18.04-x86_64
CMAKE_PREFIX_PATH=/home/spack/spack/var/spack/environments/myproject/.spack-env/view
Let’s use this to build a really simple MPI program. Make a simple test program
like this one. Call it mpi-hello.c
.
#include <stdio.h>
#include <mpi.h>
#include <zlib.h>
int main(int argc, char **argv) {
int rank;
MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
printf("Hello world from rank %d\n", rank);
if (rank == 0) {
printf("zlib version: %s\n", ZLIB_VERSION);
}
MPI_Finalize();
}
This program includes a header from zlib, and prints out a message from each MPI rank. It also prints the zlib version.
All you need to do is build and run it:
$ mpicc ./mpi-hello.c
$ mpirun -n 4 ./a.out
Hello world from rank 3
Hello world from rank 0
zlib version: 1.2.11
Hello world from rank 1
Hello world from rank 2
Note that we did not need to pass any special arguments to the compiler; just the source file. This simple example only scratches the surface, but you can use environments to set up dependencies for a project, set up a run environment for a user, support your usual development environment, and many other use cases. We’ll revisit some advanced use cases later, in the developer workflows tutorial.
spack.yaml
and spack.lock
¶
We’ve shown you how to interact with environments from the command line, but they also have a file-based interface that can be used by developers and admins to manage projects in a reproducible way.
In this section, we’ll dive a little deeper to see how environments are implemented, and how you could use this in your day-to-day development.
spack.yaml
¶
Earlier, we changed an environment’s configuration using spack config
edit
. We were actually editing a special file called spack.yaml
.
Let’s take a look.
We can get directly to the current environment’s location using spack cd
:
$ spack cd -e myproject
$ pwd
/home/spack/spack/var/spack/environments/myproject
$ ls
spack.lock spack.yaml
You should notice two things here. First, the environment is just a directory
inside of var/spack/environments
within the Spack installation. Second, it
contains two important files: spack.yaml
and spack.lock
.
spack.yaml
is the configuration file for environments that we’ve
already seen, but it does not have to live inside Spack. If you create
an environment using spack env create
, it is managed by
Spack in the var/spack/environments
directory, and you can refer to
it by name.
You can actually put a spack.yaml
file anywhere, and you can use it
to bundle an environment, or a list of dependencies to install, with your
project. Let’s make a simple project:
$ cd
$ mkdir code
$ cd code
$ spack env create -d .
==> Updating view at /home/spack/code/.spack-env/view
==> Created environment in /home/spack/code
==> You can activate this environment with:
==> spack env activate /home/spack/code
Here, we made a new directory called code, and we used the -d
option to create an environment in it.
What really happened?
$ ls
spack.yaml
$ cat spack.yaml
# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
# add package specs to the `specs` list
specs: []
view: true
Spack just created a spack.yaml
file in the code directory, with an
empty list of root specs. Now we have a Spack environment, in a
directory, that we can use to manage dependencies. Suppose your project
depends on boost
, trilinos
, and openmpi
. You can add these
to your spec list:
# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
# add package specs to the `specs` list
specs:
- boost
- trilinos
- openmpi
And now anyone who uses the code repository can use this format to
install the project’s dependencies. They need only clone the repository,
cd
into it, and type spack install
:
$ spack env activate .
$ spack install
==> Concretized boost
[+] hc5atqc boost@1.74.0%gcc@7.5.0+atomic+chrono~clanglibcpp~container~context~coroutine+date_time~debug+exception~fiber+filesystem+graph~icu+iostreams+locale+log+math~mpi+multithreaded~numpy~pic+program_options~python+random+regex+serialization+shared+signals~singlethreaded+system~taggedlayout+test+thread+timer~versionedlayout+wave cxxstd=98 visibility=hidden arch=linux-ubuntu18.04-x86_64
[+] fvfpt26 ^bzip2@1.0.8%gcc@7.5.0+shared arch=linux-ubuntu18.04-x86_64
[+] otkkten ^diffutils@3.7%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] jearpk4 ^libiconv@1.16%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] smoyzzo ^zlib@1.2.11%gcc@7.5.0+optimize+pic+shared arch=linux-ubuntu18.04-x86_64
==> Concretized trilinos
[+] qmjbxf2 trilinos@13.0.1%gcc@7.5.0~adios2~alloptpkgs+amesos+amesos2+anasazi+aztec+belos+boost~cgns~chaco~complex~cuda~debug~dtk+epetra+epetraext+exodus+explicit_template_instantiation~float+fortran+glm+gtest+hdf5~hwloc+hypre+ifpack+ifpack2~intrepid~intrepid2~ipo~isorropia+kokkos+matio~mesquite+metis~minitensor+ml+mpi+muelu+mumps+netcdf~nox~openmp~phalanx~piro~pnetcdf~python~rol~rythmos+sacado~shards+shared~shylu~stk~stratimikos~strumpack+suite-sparse~superlu~superlu-dist~teko~tempus+teuchos+tpetra~wrapper~x11~xsdkflags~zlib+zoltan+zoltan2 build_type=RelWithDebInfo cuda_arch=none cxxstd=11 gotype=long arch=linux-ubuntu18.04-x86_64
[+] hc5atqc ^boost@1.74.0%gcc@7.5.0+atomic+chrono~clanglibcpp~container~context~coroutine+date_time~debug+exception~fiber+filesystem+graph~icu+iostreams+locale+log+math~mpi+multithreaded~numpy~pic+program_options~python+random+regex+serialization+shared+signals~singlethreaded+system~taggedlayout+test+thread+timer~versionedlayout+wave cxxstd=98 visibility=hidden arch=linux-ubuntu18.04-x86_64
[+] fvfpt26 ^bzip2@1.0.8%gcc@7.5.0+shared arch=linux-ubuntu18.04-x86_64
[+] otkkten ^diffutils@3.7%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] jearpk4 ^libiconv@1.16%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] smoyzzo ^zlib@1.2.11%gcc@7.5.0+optimize+pic+shared arch=linux-ubuntu18.04-x86_64
[+] bltycqw ^cmake@3.18.4%gcc@7.5.0~doc+ncurses+openssl+ownlibs~qt patches=bf695e3febb222da2ed94b3beea600650e4318975da90e4a71d6f31a6d5d8c3d arch=linux-ubuntu18.04-x86_64
[+] crhlefo ^ncurses@6.2%gcc@7.5.0~symlinks+termlib arch=linux-ubuntu18.04-x86_64
[+] 4sh6pym ^pkgconf@1.7.3%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] es377uq ^openssl@1.1.1h%gcc@7.5.0+systemcerts arch=linux-ubuntu18.04-x86_64
[+] zfdvt2j ^perl@5.32.0%gcc@7.5.0+cpanm+shared+threads arch=linux-ubuntu18.04-x86_64
[+] 4ihuiaz ^berkeley-db@18.1.40%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] 4av4gyw ^gdbm@1.18.1%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] t54jzdy ^readline@8.0%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] n4xti2g ^glm@0.9.7.1%gcc@7.5.0~ipo build_type=RelWithDebInfo arch=linux-ubuntu18.04-x86_64
[+] ejanivx ^hdf5@1.10.7%gcc@7.5.0~cxx~debug~fortran+hl~java+mpi+pic+shared~szip~threadsafe api=none arch=linux-ubuntu18.04-x86_64
[+] pmsyupw ^openmpi@3.1.6%gcc@7.5.0~atomics~cuda~cxx~cxx_exceptions+gpfs~java~legacylaunchers~lustre~memchecker~pmi~singularity~sqlite3+static~thread_multiple+vt+wrapper-rpath fabrics=none schedulers=none arch=linux-ubuntu18.04-x86_64
[+] zqwfzhw ^hwloc@1.11.11%gcc@7.5.0~cairo~cuda~gl~libudev+libxml2~netloc~nvml+pci+shared arch=linux-ubuntu18.04-x86_64
[+] bob4o5m ^libpciaccess@0.16%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] jdxbjft ^libtool@2.4.6%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] mkc3u4x ^m4@1.4.18%gcc@7.5.0+sigsegv patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 arch=linux-ubuntu18.04-x86_64
[+] lbrx7ln ^libsigsegv@2.12%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] gs6ag7k ^util-macros@1.19.1%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] yn2r3wf ^libxml2@2.9.10%gcc@7.5.0~python arch=linux-ubuntu18.04-x86_64
[+] komekkm ^xz@5.2.5%gcc@7.5.0~pic arch=linux-ubuntu18.04-x86_64
[+] wbqbc5v ^numactl@2.0.14%gcc@7.5.0 patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94 arch=linux-ubuntu18.04-x86_64
[+] mm33a3o ^autoconf@2.69%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] d2krmb5 ^automake@1.16.2%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] nndket2 ^hypre@2.20.0%gcc@7.5.0~complex~debug~int64~internal-superlu~mixedint+mpi~openmp+shared~superlu-dist patches=6e3336b1d62155f6350dfe42b0f9ea25d4fa0af60c7e540959139deb93a26059 arch=linux-ubuntu18.04-x86_64
[+] te35tjx ^openblas@0.3.12%gcc@7.5.0~consistent_fpcsr~ilp64+pic+shared threads=none arch=linux-ubuntu18.04-x86_64
[+] zoqjxgk ^matio@1.5.17%gcc@7.5.0+hdf5+shared+zlib arch=linux-ubuntu18.04-x86_64
[+] edcrft4 ^metis@5.1.0%gcc@7.5.0~gdb~int64~real64+shared build_type=Release patches=4991da938c1d3a1d3dea78e49bbebecba00273f98df2a656e38b83d55b281da1,b1225da886605ea558db7ac08dd8054742ea5afe5ed61ad4d0fe7a495b1270d2 arch=linux-ubuntu18.04-x86_64
[+] 3jmjrzw ^mumps@5.3.3%gcc@7.5.0+complex+double+float~int64~metis+mpi~parmetis~ptscotch~scotch+shared arch=linux-ubuntu18.04-x86_64
[+] wsiydak ^netlib-scalapack@2.1.0%gcc@7.5.0~ipo~pic+shared build_type=Release patches=1c9ce5fee1451a08c2de3cc87f446aeda0b818ebbce4ad0d980ddf2f2a0b2dc4,f2baedde688ffe4c20943c334f580eb298e04d6f35c86b90a1f4e8cb7ae344a2 arch=linux-ubuntu18.04-x86_64
[+] yixejly ^netcdf-c@4.7.4%gcc@7.5.0~dap~hdf4~jna+mpi~parallel-netcdf+pic+shared arch=linux-ubuntu18.04-x86_64
[+] htdsyvt ^parmetis@4.0.3%gcc@7.5.0~gdb~int64~ipo+shared build_type=RelWithDebInfo patches=4f892531eb0a807eb1b82e683a416d3e35154a455274cf9b162fb02054d11a5b,50ed2081bc939269689789942067c58b3e522c269269a430d5d34c00edbc5870,704b84f7c7444d4372cb59cca6e1209df4ef3b033bc4ee3cf50f369bce972a9d arch=linux-ubuntu18.04-x86_64
[+] rdfmg5b ^suite-sparse@5.8.1%gcc@7.5.0~cuda~openmp+pic~tbb arch=linux-ubuntu18.04-x86_64
[+] 3ol3tld ^gmp@6.1.2%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] mpv2v7v ^mpfr@4.0.2%gcc@7.5.0 patches=3f80b836948aa96f8d1cb9cc7f3f55973f19285482a96f9a4e1623d460bcccf0 arch=linux-ubuntu18.04-x86_64
[+] bdyarrk ^autoconf-archive@2019.01.06%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
==> Concretized openmpi
[+] pmsyupw openmpi@3.1.6%gcc@7.5.0~atomics~cuda~cxx~cxx_exceptions+gpfs~java~legacylaunchers~lustre~memchecker~pmi~singularity~sqlite3+static~thread_multiple+vt+wrapper-rpath fabrics=none schedulers=none arch=linux-ubuntu18.04-x86_64
[+] zqwfzhw ^hwloc@1.11.11%gcc@7.5.0~cairo~cuda~gl~libudev+libxml2~netloc~nvml+pci+shared arch=linux-ubuntu18.04-x86_64
[+] bob4o5m ^libpciaccess@0.16%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] jdxbjft ^libtool@2.4.6%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] mkc3u4x ^m4@1.4.18%gcc@7.5.0+sigsegv patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 arch=linux-ubuntu18.04-x86_64
[+] lbrx7ln ^libsigsegv@2.12%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] 4sh6pym ^pkgconf@1.7.3%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] gs6ag7k ^util-macros@1.19.1%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] yn2r3wf ^libxml2@2.9.10%gcc@7.5.0~python arch=linux-ubuntu18.04-x86_64
[+] jearpk4 ^libiconv@1.16%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] komekkm ^xz@5.2.5%gcc@7.5.0~pic arch=linux-ubuntu18.04-x86_64
[+] smoyzzo ^zlib@1.2.11%gcc@7.5.0+optimize+pic+shared arch=linux-ubuntu18.04-x86_64
[+] wbqbc5v ^numactl@2.0.14%gcc@7.5.0 patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94 arch=linux-ubuntu18.04-x86_64
[+] mm33a3o ^autoconf@2.69%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] zfdvt2j ^perl@5.32.0%gcc@7.5.0+cpanm+shared+threads arch=linux-ubuntu18.04-x86_64
[+] 4ihuiaz ^berkeley-db@18.1.40%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] 4av4gyw ^gdbm@1.18.1%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] t54jzdy ^readline@8.0%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
[+] crhlefo ^ncurses@6.2%gcc@7.5.0~symlinks+termlib arch=linux-ubuntu18.04-x86_64
[+] d2krmb5 ^automake@1.16.2%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64
==> Installing environment /home/spack/code
==> All of the packages are already installed
==> Updating view at /home/spack/code/.spack-env/view
Spack concretizes the specs in the spack.yaml
file and installs them.
So, from ~/code
, we can actually manipulate spack.yaml
using
spack add
and spack remove
(just like managed environments):
$ spack add hdf5@5.5.1
==> Adding hdf5@5.5.1 to environment /home/spack/code
==> Updating view at /home/spack/code/.spack-env/view
$ cat spack.yaml
# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
# add package specs to the `specs` list
specs: [boost, trilinos, openmpi, hdf5@5.5.1]
view: true
$ spack remove hdf5
==> Removing hdf5 from environment /home/spack/code
==> Updating view at /home/spack/code/.spack-env/view
spack.lock
¶
Okay, we’ve covered managed environments, environments in directories, and
the last thing we’ll cover is spack.lock
. You may remember that when
we ran spack install
, Spack concretized all the specs in the
spack.yaml
file and installed them.
Whenever we concretize Specs in an environment, all concrete specs in the
environment are written out to a spack.lock
file alongside
spack.yaml
. The spack.lock
file is not really human-readable
like the spack.yaml
file. It is a json
format that contains all
the information that we need to reproduce the build of an
environment:
$ head -30 spack.lock
{
"_meta": {
"file-type": "spack-lockfile",
"lockfile-version": 2
},
"roots": [
{
"hash": "iwji2vty6m7hp27zbzufewdhq6pty3fx",
"spec": "boost"
},
{
"hash": "246z4kkzk6vsg2ot3rzlz5jwwimxj63g",
"spec": "trilinos"
},
{
"hash": "7o7eidaryeh72cizcbpvvjnq7ojjya6f",
"spec": "openmpi"
}
],
"concrete_specs": {
"iwji2vty6m7hp27zbzufewdhq6pty3fx": {
"boost": {
"version": "1.74.0",
"arch": {
"platform": "linux",
"platform_os": "ubuntu18.04",
"target": "x86_64"
},
"compiler": {
"name": "gcc",
spack.yaml
and spack.lock
correspond to two fundamental concepts
in Spack:
spack.yaml
is the set of abstract specs and configuration that you want to install.
spack.lock
is the set of all fully concrete specs generated from concretizingspack.yaml
In this sense, you can think of environments as generalizations of specs, but for sets of packages.
spack.yaml
and spack.lock
both allow you to recreate an environment
that someone else built. You can pass either as an argument to spack env
create
:
$ spack env create abstract spack.yaml
==> Updating view at /home/spack/spack/var/spack/environments/abstract/.spack-env/view
==> Created environment 'abstract' in /home/spack/spack/var/spack/environments/abstract
==> You can activate this environment with:
==> spack env activate abstract
$ spack env create concrete spack.lock
==> Updating view at /home/spack/spack/var/spack/environments/concrete/.spack-env/view
==> Created environment 'concrete' in /home/spack/spack/var/spack/environments/concrete
==> You can activate this environment with:
==> spack env activate concrete
Both of these create a new environment from the old one, but which one you choose depends on your needs:
abstract
: copying thespack.yaml
file allows someone else to build your requirements, potentially a different way.concrete
: copying thespack.lock
file allows someone else to rebuild your installation exactly as you built it.
The first use case can re-concretize the same specs on new platforms in order to build, but it will preserve the abstract requirements. The second use case (currently) requires you to be on the same type of machine, but it retains all decisions made during concretization and is faithful to a prior install.