Tutorial setup
If you have not done the prior sections, you’ll need to set Spack up like this:
git clone https://github.com/spack/spack
. spack/share/spack/setup-env.sh
spack tutorial
See the Basic Installation Tutorial for full details on setup. For more
help join us in the #tutorial channel on Slack – get an
invitation at spackpm.herokuapp.com
Scripting with Spack¶
This tutorial introduces advanced Spack features related to scripting.
Specifically, we will show you how to write scripts using spack find
and spack python.
Earlier sections of the tutorial demonstrated using spack find to
list and search installed packages.
The spack python command gives you access to all of Spack’s internal
APIs, allowing
you to write more complex queries, for example.
Since Spack has an extensive API, we’ll only scratch the surface here. We’ll give you enough information to start writing your own scripts and to find what you need, with a little digging.
Setting up the tutorial¶
Before proceeding, let’s ensure the outputs for this segment are reasonable. Since you may have a lot of packages installed from earlier sections of the tutorial, we want to perform a little cleanup.
Let’s remove gcc@8.3.0 and re-install hdf5 and zlib@clang
using the following commands:
$ spack uninstall -ay
==> Successfully uninstalled gawk@5.1.1%gcc@7.5.0~nls arch=linux-ubuntu18.04-x86_64/ejl2btv
==> Successfully uninstalled diffutils@3.8%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64/k22pe4r
==> Successfully uninstalled autoconf@2.69%gcc@7.5.0 patches=35c4492,7793209,a49dd5b arch=linux-ubuntu18.04-x86_64/hlj3ujm
==> Successfully uninstalled util-macros@1.19.3%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64/vkk73ww
==> Successfully uninstalled hdf5@1.12.2%gcc@7.5.0~cxx~fortran~hl~ipo~java+mpi+shared~szip~threadsafe+tools api=default build_type=RelWithDebInfo arch=linux-ubuntu18.04-x86_64/rh2gilz
==> Successfully uninstalled zlib@1.2.12%clang@7.0.0+optimize+pic+shared patches=0d38234 arch=linux-ubuntu18.04-x86_64/3svypja
==> Successfully uninstalled cmake@3.23.1%gcc@7.5.0~doc+ncurses+ownlibs~qt build_type=Release arch=linux-ubuntu18.04-x86_64/tulqz4n
==> Successfully uninstalled automake@1.16.5%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64/6vk5ehd
==> Successfully uninstalled libtool@2.4.7%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64/7fhdrxx
==> Successfully uninstalled mpc@1.1.0%gcc@7.5.0 libs=shared,static arch=linux-ubuntu18.04-x86_64/lmcyfxq
==> Successfully uninstalled texinfo@6.5%gcc@7.5.0 patches=12f6edb,1732115 arch=linux-ubuntu18.04-x86_64/ot5b2ak
==> Successfully uninstalled perl@5.34.1%gcc@7.5.0+cpanm+shared+threads arch=linux-ubuntu18.04-x86_64/pn5mtes
==> Successfully uninstalled openmpi@4.1.3%gcc@7.5.0~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~java~legacylaunchers~lustre~memchecker~pmi+romio+rsh~singularity+static+vt+wrapper-rpath fabrics=none schedulers=none arch=linux-ubuntu18.04-x86_64/jfxctqw
==> Successfully uninstalled mpfr@3.1.6%gcc@7.5.0 libs=shared,static patches=7a6dd71 arch=linux-ubuntu18.04-x86_64/vmql5cb
==> Successfully uninstalled m4@1.4.19%gcc@7.5.0+sigsegv patches=9dc5fbd,bfdffa7 arch=linux-ubuntu18.04-x86_64/uadbn2a
==> Successfully uninstalled pkgconf@1.8.0%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64/i5po27g
==> Successfully uninstalled gdbm@1.19%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64/xub4jsy
==> Successfully uninstalled berkeley-db@18.1.40%gcc@7.5.0+cxx~docs+stl patches=b231fcc arch=linux-ubuntu18.04-x86_64/q6tqmmm
==> Successfully uninstalled bzip2@1.0.8%gcc@7.5.0~debug~pic+shared arch=linux-ubuntu18.04-x86_64/cyn2yam
==> Successfully uninstalled openssh@9.0p1%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64/3ez3d3j
==> Successfully uninstalled pmix@4.1.2%gcc@7.5.0~docs+pmi_backwards_compatibility~restful arch=linux-ubuntu18.04-x86_64/e4fax47
==> Successfully uninstalled gmp@6.2.1%gcc@7.5.0 libs=shared,static arch=linux-ubuntu18.04-x86_64/lf7yoxu
==> Successfully uninstalled numactl@2.0.14%gcc@7.5.0 patches=4e1d78c,62fc8a8,ff37630 arch=linux-ubuntu18.04-x86_64/wfnw532
==> Successfully uninstalled libsigsegv@2.13%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64/ubezeyc
==> Successfully uninstalled readline@8.1%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64/ls6erxw
==> Successfully uninstalled libevent@2.1.12%gcc@7.5.0+openssl arch=linux-ubuntu18.04-x86_64/hlnkgmr
==> Successfully uninstalled libedit@3.1-20210216%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64/zc56lv6
==> Successfully uninstalled hwloc@2.7.1%gcc@7.5.0~cairo~cuda~gl~libudev+libxml2~netloc~nvml~opencl+pci~rocm+shared arch=linux-ubuntu18.04-x86_64/oixja6d
==> Successfully uninstalled openssl@1.1.1o%gcc@7.5.0~docs~shared certs=system arch=linux-ubuntu18.04-x86_64/hqlqpsn
==> Successfully uninstalled libpciaccess@0.16%gcc@7.5.0 arch=linux-ubuntu18.04-x86_64/csp5gvq
==> Successfully uninstalled libxml2@2.9.13%gcc@7.5.0~python arch=linux-ubuntu18.04-x86_64/4run532
==> Successfully uninstalled ncurses@6.2%gcc@7.5.0~symlinks+termlib abi=none arch=linux-ubuntu18.04-x86_64/frwzsc4
==> Successfully uninstalled xz@5.2.5%gcc@7.5.0~pic libs=shared,static arch=linux-ubuntu18.04-x86_64/olptpbs
==> Successfully uninstalled libiconv@1.16%gcc@7.5.0 libs=shared,static arch=linux-ubuntu18.04-x86_64/dr56jt4
==> Successfully uninstalled zlib@1.2.12%gcc@7.5.0+optimize+pic+shared patches=0d38234 arch=linux-ubuntu18.04-x86_64/fntvsj6
$ spack compiler rm gcc@8.4.0
==> Removed compiler gcc@8.4.0
$ spack install hdf5
==> Waiting for pkgconf-1.8.0-i5po27g2n6zrzxnhxdy2fnpaai7xsdaw
==> Installing pkgconf-1.8.0-i5po27g2n6zrzxnhxdy2fnpaai7xsdaw
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-pkgconf-1.8.0-i5po27g2n6zrzxnhxdy2fnpaai7xsdaw.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/pkgconf-1.8.0/linux-ubuntu18.04-x86_64-gcc-7.5.0-pkgconf-1.8.0-i5po27g2n6zrzxnhxdy2fnpaai7xsdaw.spack
==> Extracting pkgconf-1.8.0-i5po27g2n6zrzxnhxdy2fnpaai7xsdaw from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/pkgconf-1.8.0-i5po27g2n6zrzxnhxdy2fnpaai7xsdaw
==> Waiting for berkeley-db-18.1.40-q6tqmmmitkatscuadcwn6ea32iscs55k
==> Installing berkeley-db-18.1.40-q6tqmmmitkatscuadcwn6ea32iscs55k
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-berkeley-db-18.1.40-q6tqmmmitkatscuadcwn6ea32iscs55k.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/berkeley-db-18.1.40/linux-ubuntu18.04-x86_64-gcc-7.5.0-berkeley-db-18.1.40-q6tqmmmitkatscuadcwn6ea32iscs55k.spack
==> Extracting berkeley-db-18.1.40-q6tqmmmitkatscuadcwn6ea32iscs55k from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/berkeley-db-18.1.40-q6tqmmmitkatscuadcwn6ea32iscs55k
==> Waiting for libiconv-1.16-dr56jt456uiebij2mkueldmejdxftigc
==> Installing libiconv-1.16-dr56jt456uiebij2mkueldmejdxftigc
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-libiconv-1.16-dr56jt456uiebij2mkueldmejdxftigc.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/libiconv-1.16/linux-ubuntu18.04-x86_64-gcc-7.5.0-libiconv-1.16-dr56jt456uiebij2mkueldmejdxftigc.spack
==> Extracting libiconv-1.16-dr56jt456uiebij2mkueldmejdxftigc from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/libiconv-1.16-dr56jt456uiebij2mkueldmejdxftigc
==> Waiting for zlib-1.2.12-fntvsj6xevbz5gyq7kfa4xg7oxnaolxs
==> Installing zlib-1.2.12-fntvsj6xevbz5gyq7kfa4xg7oxnaolxs
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-zlib-1.2.12-fntvsj6xevbz5gyq7kfa4xg7oxnaolxs.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/zlib-1.2.12/linux-ubuntu18.04-x86_64-gcc-7.5.0-zlib-1.2.12-fntvsj6xevbz5gyq7kfa4xg7oxnaolxs.spack
==> Extracting zlib-1.2.12-fntvsj6xevbz5gyq7kfa4xg7oxnaolxs from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/zlib-1.2.12-fntvsj6xevbz5gyq7kfa4xg7oxnaolxs
==> Waiting for libsigsegv-2.13-ubezeychw4iojr3tg6wbsnvp6s3s2qwd
==> Installing libsigsegv-2.13-ubezeychw4iojr3tg6wbsnvp6s3s2qwd
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-libsigsegv-2.13-ubezeychw4iojr3tg6wbsnvp6s3s2qwd.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/libsigsegv-2.13/linux-ubuntu18.04-x86_64-gcc-7.5.0-libsigsegv-2.13-ubezeychw4iojr3tg6wbsnvp6s3s2qwd.spack
==> Extracting libsigsegv-2.13-ubezeychw4iojr3tg6wbsnvp6s3s2qwd from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/libsigsegv-2.13-ubezeychw4iojr3tg6wbsnvp6s3s2qwd
==> Waiting for util-macros-1.19.3-vkk73ww5p7g4u6lpnay5hcclsckcihtt
==> Installing util-macros-1.19.3-vkk73ww5p7g4u6lpnay5hcclsckcihtt
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-util-macros-1.19.3-vkk73ww5p7g4u6lpnay5hcclsckcihtt.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/util-macros-1.19.3/linux-ubuntu18.04-x86_64-gcc-7.5.0-util-macros-1.19.3-vkk73ww5p7g4u6lpnay5hcclsckcihtt.spack
==> Extracting util-macros-1.19.3-vkk73ww5p7g4u6lpnay5hcclsckcihtt from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/util-macros-1.19.3-vkk73ww5p7g4u6lpnay5hcclsckcihtt
==> Waiting for xz-5.2.5-olptpbsse74hpreilbfo4qmlqexihzye
==> Installing xz-5.2.5-olptpbsse74hpreilbfo4qmlqexihzye
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-xz-5.2.5-olptpbsse74hpreilbfo4qmlqexihzye.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/xz-5.2.5/linux-ubuntu18.04-x86_64-gcc-7.5.0-xz-5.2.5-olptpbsse74hpreilbfo4qmlqexihzye.spack
==> Extracting xz-5.2.5-olptpbsse74hpreilbfo4qmlqexihzye from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/xz-5.2.5-olptpbsse74hpreilbfo4qmlqexihzye
==> Waiting for ncurses-6.2-frwzsc4jrou7efe7huyzeownqfjvsrfd
==> Installing ncurses-6.2-frwzsc4jrou7efe7huyzeownqfjvsrfd
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-ncurses-6.2-frwzsc4jrou7efe7huyzeownqfjvsrfd.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/ncurses-6.2/linux-ubuntu18.04-x86_64-gcc-7.5.0-ncurses-6.2-frwzsc4jrou7efe7huyzeownqfjvsrfd.spack
==> Extracting ncurses-6.2-frwzsc4jrou7efe7huyzeownqfjvsrfd from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/ncurses-6.2-frwzsc4jrou7efe7huyzeownqfjvsrfd
==> Waiting for diffutils-3.8-k22pe4rhr7yllf5pojl75veh5gkjdglq
==> Installing diffutils-3.8-k22pe4rhr7yllf5pojl75veh5gkjdglq
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-diffutils-3.8-k22pe4rhr7yllf5pojl75veh5gkjdglq.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/diffutils-3.8/linux-ubuntu18.04-x86_64-gcc-7.5.0-diffutils-3.8-k22pe4rhr7yllf5pojl75veh5gkjdglq.spack
==> Extracting diffutils-3.8-k22pe4rhr7yllf5pojl75veh5gkjdglq from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/diffutils-3.8-k22pe4rhr7yllf5pojl75veh5gkjdglq
==> Waiting for m4-1.4.19-uadbn2a64h744jto2xqy6u2wmujcww6v
==> Installing m4-1.4.19-uadbn2a64h744jto2xqy6u2wmujcww6v
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-m4-1.4.19-uadbn2a64h744jto2xqy6u2wmujcww6v.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/m4-1.4.19/linux-ubuntu18.04-x86_64-gcc-7.5.0-m4-1.4.19-uadbn2a64h744jto2xqy6u2wmujcww6v.spack
==> Extracting m4-1.4.19-uadbn2a64h744jto2xqy6u2wmujcww6v from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/m4-1.4.19-uadbn2a64h744jto2xqy6u2wmujcww6v
==> Waiting for libxml2-2.9.13-4run532u4c6iy5th5gkpgm26tr4vajc4
==> Installing libxml2-2.9.13-4run532u4c6iy5th5gkpgm26tr4vajc4
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-libxml2-2.9.13-4run532u4c6iy5th5gkpgm26tr4vajc4.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/libxml2-2.9.13/linux-ubuntu18.04-x86_64-gcc-7.5.0-libxml2-2.9.13-4run532u4c6iy5th5gkpgm26tr4vajc4.spack
==> Extracting libxml2-2.9.13-4run532u4c6iy5th5gkpgm26tr4vajc4 from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/libxml2-2.9.13-4run532u4c6iy5th5gkpgm26tr4vajc4
==> Waiting for libedit-3.1-20210216-zc56lv6gyjpfvi4tpdtmx5yb7ljfnlrb
==> Installing libedit-3.1-20210216-zc56lv6gyjpfvi4tpdtmx5yb7ljfnlrb
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-libedit-3.1-20210216-zc56lv6gyjpfvi4tpdtmx5yb7ljfnlrb.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/libedit-3.1-20210216/linux-ubuntu18.04-x86_64-gcc-7.5.0-libedit-3.1-20210216-zc56lv6gyjpfvi4tpdtmx5yb7ljfnlrb.spack
==> Extracting libedit-3.1-20210216-zc56lv6gyjpfvi4tpdtmx5yb7ljfnlrb from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/libedit-3.1-20210216-zc56lv6gyjpfvi4tpdtmx5yb7ljfnlrb
==> Waiting for readline-8.1-ls6erxwsj2ubsfzt4x5mblj6nrab5jkp
==> Installing readline-8.1-ls6erxwsj2ubsfzt4x5mblj6nrab5jkp
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-readline-8.1-ls6erxwsj2ubsfzt4x5mblj6nrab5jkp.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/readline-8.1/linux-ubuntu18.04-x86_64-gcc-7.5.0-readline-8.1-ls6erxwsj2ubsfzt4x5mblj6nrab5jkp.spack
==> Extracting readline-8.1-ls6erxwsj2ubsfzt4x5mblj6nrab5jkp from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/readline-8.1-ls6erxwsj2ubsfzt4x5mblj6nrab5jkp
==> Waiting for bzip2-1.0.8-cyn2yamiu26ofxwgxzxbioreuwigzl6f
==> Installing bzip2-1.0.8-cyn2yamiu26ofxwgxzxbioreuwigzl6f
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-bzip2-1.0.8-cyn2yamiu26ofxwgxzxbioreuwigzl6f.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/bzip2-1.0.8/linux-ubuntu18.04-x86_64-gcc-7.5.0-bzip2-1.0.8-cyn2yamiu26ofxwgxzxbioreuwigzl6f.spack
==> Extracting bzip2-1.0.8-cyn2yamiu26ofxwgxzxbioreuwigzl6f from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/bzip2-1.0.8-cyn2yamiu26ofxwgxzxbioreuwigzl6f
==> Waiting for libtool-2.4.7-7fhdrxxizepvvkxrfaaizhzkwx2sjzbt
==> Installing libtool-2.4.7-7fhdrxxizepvvkxrfaaizhzkwx2sjzbt
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-libtool-2.4.7-7fhdrxxizepvvkxrfaaizhzkwx2sjzbt.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/libtool-2.4.7/linux-ubuntu18.04-x86_64-gcc-7.5.0-libtool-2.4.7-7fhdrxxizepvvkxrfaaizhzkwx2sjzbt.spack
==> Extracting libtool-2.4.7-7fhdrxxizepvvkxrfaaizhzkwx2sjzbt from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/libtool-2.4.7-7fhdrxxizepvvkxrfaaizhzkwx2sjzbt
==> Waiting for gdbm-1.19-xub4jsy5pz5qjf6vooropf2shkigy5pm
==> Installing gdbm-1.19-xub4jsy5pz5qjf6vooropf2shkigy5pm
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-gdbm-1.19-xub4jsy5pz5qjf6vooropf2shkigy5pm.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/gdbm-1.19/linux-ubuntu18.04-x86_64-gcc-7.5.0-gdbm-1.19-xub4jsy5pz5qjf6vooropf2shkigy5pm.spack
==> Extracting gdbm-1.19-xub4jsy5pz5qjf6vooropf2shkigy5pm from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/gdbm-1.19-xub4jsy5pz5qjf6vooropf2shkigy5pm
==> Waiting for libpciaccess-0.16-csp5gvqmwcmqsd4al5iafotfk5fqh3uf
==> Installing libpciaccess-0.16-csp5gvqmwcmqsd4al5iafotfk5fqh3uf
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-libpciaccess-0.16-csp5gvqmwcmqsd4al5iafotfk5fqh3uf.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/libpciaccess-0.16/linux-ubuntu18.04-x86_64-gcc-7.5.0-libpciaccess-0.16-csp5gvqmwcmqsd4al5iafotfk5fqh3uf.spack
==> Extracting libpciaccess-0.16-csp5gvqmwcmqsd4al5iafotfk5fqh3uf from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/libpciaccess-0.16-csp5gvqmwcmqsd4al5iafotfk5fqh3uf
==> Waiting for perl-5.34.1-pn5mteswujdkuong6mbqaush54rdas7c
==> Installing perl-5.34.1-pn5mteswujdkuong6mbqaush54rdas7c
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-perl-5.34.1-pn5mteswujdkuong6mbqaush54rdas7c.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/perl-5.34.1/linux-ubuntu18.04-x86_64-gcc-7.5.0-perl-5.34.1-pn5mteswujdkuong6mbqaush54rdas7c.spack
==> Extracting perl-5.34.1-pn5mteswujdkuong6mbqaush54rdas7c from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/perl-5.34.1-pn5mteswujdkuong6mbqaush54rdas7c
==> Waiting for hwloc-2.7.1-oixja6dnsnc4mydqswdya2nyt4rlcxpa
==> Installing hwloc-2.7.1-oixja6dnsnc4mydqswdya2nyt4rlcxpa
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-hwloc-2.7.1-oixja6dnsnc4mydqswdya2nyt4rlcxpa.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/hwloc-2.7.1/linux-ubuntu18.04-x86_64-gcc-7.5.0-hwloc-2.7.1-oixja6dnsnc4mydqswdya2nyt4rlcxpa.spack
==> Extracting hwloc-2.7.1-oixja6dnsnc4mydqswdya2nyt4rlcxpa from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/hwloc-2.7.1-oixja6dnsnc4mydqswdya2nyt4rlcxpa
==> Waiting for autoconf-2.69-hlj3ujmrsu5gs7uhfdotozzg6ipys45b
==> Installing autoconf-2.69-hlj3ujmrsu5gs7uhfdotozzg6ipys45b
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-autoconf-2.69-hlj3ujmrsu5gs7uhfdotozzg6ipys45b.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/autoconf-2.69/linux-ubuntu18.04-x86_64-gcc-7.5.0-autoconf-2.69-hlj3ujmrsu5gs7uhfdotozzg6ipys45b.spack
==> Extracting autoconf-2.69-hlj3ujmrsu5gs7uhfdotozzg6ipys45b from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/autoconf-2.69-hlj3ujmrsu5gs7uhfdotozzg6ipys45b
==> Waiting for openssl-1.1.1o-hqlqpsnqoifap6mlkbr5fyxoaq3gynbk
==> Installing openssl-1.1.1o-hqlqpsnqoifap6mlkbr5fyxoaq3gynbk
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-openssl-1.1.1o-hqlqpsnqoifap6mlkbr5fyxoaq3gynbk.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/openssl-1.1.1o/linux-ubuntu18.04-x86_64-gcc-7.5.0-openssl-1.1.1o-hqlqpsnqoifap6mlkbr5fyxoaq3gynbk.spack
==> Extracting openssl-1.1.1o-hqlqpsnqoifap6mlkbr5fyxoaq3gynbk from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/openssl-1.1.1o-hqlqpsnqoifap6mlkbr5fyxoaq3gynbk
==> Waiting for automake-1.16.5-6vk5ehdtjzgcvug6vuwh2mmkjhg2pn2z
==> Installing automake-1.16.5-6vk5ehdtjzgcvug6vuwh2mmkjhg2pn2z
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-automake-1.16.5-6vk5ehdtjzgcvug6vuwh2mmkjhg2pn2z.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/automake-1.16.5/linux-ubuntu18.04-x86_64-gcc-7.5.0-automake-1.16.5-6vk5ehdtjzgcvug6vuwh2mmkjhg2pn2z.spack
==> Extracting automake-1.16.5-6vk5ehdtjzgcvug6vuwh2mmkjhg2pn2z from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/automake-1.16.5-6vk5ehdtjzgcvug6vuwh2mmkjhg2pn2z
==> Waiting for libevent-2.1.12-hlnkgmrdcycqf65n6vrvoraq4dhddagv
==> Installing libevent-2.1.12-hlnkgmrdcycqf65n6vrvoraq4dhddagv
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-libevent-2.1.12-hlnkgmrdcycqf65n6vrvoraq4dhddagv.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/libevent-2.1.12/linux-ubuntu18.04-x86_64-gcc-7.5.0-libevent-2.1.12-hlnkgmrdcycqf65n6vrvoraq4dhddagv.spack
==> Extracting libevent-2.1.12-hlnkgmrdcycqf65n6vrvoraq4dhddagv from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/libevent-2.1.12-hlnkgmrdcycqf65n6vrvoraq4dhddagv
==> Waiting for openssh-9.0p1-3ez3d3jtc4f7bph5suuzqebujp3gppwv
==> Installing openssh-9.0p1-3ez3d3jtc4f7bph5suuzqebujp3gppwv
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-openssh-9.0p1-3ez3d3jtc4f7bph5suuzqebujp3gppwv.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/openssh-9.0p1/linux-ubuntu18.04-x86_64-gcc-7.5.0-openssh-9.0p1-3ez3d3jtc4f7bph5suuzqebujp3gppwv.spack
==> Extracting openssh-9.0p1-3ez3d3jtc4f7bph5suuzqebujp3gppwv from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/openssh-9.0p1-3ez3d3jtc4f7bph5suuzqebujp3gppwv
==> Waiting for cmake-3.23.1-tulqz4nxor764kgeij23tfap4ddgpgvh
==> Installing cmake-3.23.1-tulqz4nxor764kgeij23tfap4ddgpgvh
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-cmake-3.23.1-tulqz4nxor764kgeij23tfap4ddgpgvh.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/cmake-3.23.1/linux-ubuntu18.04-x86_64-gcc-7.5.0-cmake-3.23.1-tulqz4nxor764kgeij23tfap4ddgpgvh.spack
==> Extracting cmake-3.23.1-tulqz4nxor764kgeij23tfap4ddgpgvh from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/cmake-3.23.1-tulqz4nxor764kgeij23tfap4ddgpgvh
==> Waiting for numactl-2.0.14-wfnw532l74emh6rwx4mrvot3huyiq53t
==> Installing numactl-2.0.14-wfnw532l74emh6rwx4mrvot3huyiq53t
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-numactl-2.0.14-wfnw532l74emh6rwx4mrvot3huyiq53t.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/numactl-2.0.14/linux-ubuntu18.04-x86_64-gcc-7.5.0-numactl-2.0.14-wfnw532l74emh6rwx4mrvot3huyiq53t.spack
==> Extracting numactl-2.0.14-wfnw532l74emh6rwx4mrvot3huyiq53t from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/numactl-2.0.14-wfnw532l74emh6rwx4mrvot3huyiq53t
==> Waiting for pmix-4.1.2-e4fax4744gqg222snd6da55c3xq3zu34
==> Installing pmix-4.1.2-e4fax4744gqg222snd6da55c3xq3zu34
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-pmix-4.1.2-e4fax4744gqg222snd6da55c3xq3zu34.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/pmix-4.1.2/linux-ubuntu18.04-x86_64-gcc-7.5.0-pmix-4.1.2-e4fax4744gqg222snd6da55c3xq3zu34.spack
==> Extracting pmix-4.1.2-e4fax4744gqg222snd6da55c3xq3zu34 from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/pmix-4.1.2-e4fax4744gqg222snd6da55c3xq3zu34
==> Waiting for openmpi-4.1.3-jfxctqwar7wb65rn7wf5mot7m4jxmsey
==> Installing openmpi-4.1.3-jfxctqwar7wb65rn7wf5mot7m4jxmsey
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-openmpi-4.1.3-jfxctqwar7wb65rn7wf5mot7m4jxmsey.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/openmpi-4.1.3/linux-ubuntu18.04-x86_64-gcc-7.5.0-openmpi-4.1.3-jfxctqwar7wb65rn7wf5mot7m4jxmsey.spack
==> Extracting openmpi-4.1.3-jfxctqwar7wb65rn7wf5mot7m4jxmsey from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/openmpi-4.1.3-jfxctqwar7wb65rn7wf5mot7m4jxmsey
==> Waiting for hdf5-1.12.2-rh2gilz6xlosoeuldxtgeyz65wsualsj
==> Installing hdf5-1.12.2-rh2gilz6xlosoeuldxtgeyz65wsualsj
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-gcc-7.5.0-hdf5-1.12.2-rh2gilz6xlosoeuldxtgeyz65wsualsj.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/gcc-7.5.0/hdf5-1.12.2/linux-ubuntu18.04-x86_64-gcc-7.5.0-hdf5-1.12.2-rh2gilz6xlosoeuldxtgeyz65wsualsj.spack
==> Extracting hdf5-1.12.2-rh2gilz6xlosoeuldxtgeyz65wsualsj from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/hdf5-1.12.2-rh2gilz6xlosoeuldxtgeyz65wsualsj
$ spack install zlib%clang
==> Waiting for zlib-1.2.12-3svypjaie5farii5zcgm6yfu43la2ixs
==> Installing zlib-1.2.12-3svypjaie5farii5zcgm6yfu43la2ixs
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64-clang-7.0.0-zlib-1.2.12-3svypjaie5farii5zcgm6yfu43la2ixs.spec.json.sig
==> Fetching file:///mirror/build_cache/linux-ubuntu18.04-x86_64/clang-7.0.0/zlib-1.2.12/linux-ubuntu18.04-x86_64-clang-7.0.0-zlib-1.2.12-3svypjaie5farii5zcgm6yfu43la2ixs.spack
==> Extracting zlib-1.2.12-3svypjaie5farii5zcgm6yfu43la2ixs from binary cache
[+] /home/spack/spack/opt/spack/linux-ubuntu18.04-x86_64/clang-7.0.0/zlib-1.2.12-3svypjaie5farii5zcgm6yfu43la2ixs
Now we are ready to use the Spack’s find and python subcommands
to query the installed packages.
Scripting with spack find¶
So far, the output we’ve seen from spack find has been for human
consumption. But you can take advantage of some advanced options of
the command to generate machine-readable output suitable for piping
to a script.
spack find --format¶
The main job of spack find is to show the user a bunch of concrete
specs that correspond to installed packages. By default, we display them
with some default attributes, like the @version suffix you’re used to
seeing in the output.
The --format argument allows you to display the specs however you
choose, using custom format strings. Format strings let you specify the
names of particular parts of the specs you want displayed. Let’s see
the first option in action.
Suppose you only want to display the name, version, and first ten (10) characters of the hash for every package installed in your Spack instance. You can generate that output with the following command:
$ spack find --format "{name} {version} {hash:10}"
autoconf 2.69 hlj3ujmrsu diffutils 3.8 k22pe4rhr7 libevent 2.1.12 hlnkgmrdcy libxml2 2.9.13 4run532u4c openssh 9.0p1 3ez3d3jtc4 readline 8.1 ls6erxwsj2
automake 1.16.5 6vk5ehdtjz gdbm 1.19 xub4jsy5pz libiconv 1.16 dr56jt456u m4 1.4.19 uadbn2a64h openssl 1.1.1o hqlqpsnqoi util-macros 1.19.3 vkk73ww5p7
berkeley-db 18.1.40 q6tqmmmitk hdf5 1.12.2 rh2gilz6xl libpciaccess 0.16 csp5gvqmwc ncurses 6.2 frwzsc4jro perl 5.34.1 pn5mteswuj xz 5.2.5 olptpbsse7
bzip2 1.0.8 cyn2yamiu2 hwloc 2.7.1 oixja6dnsn libsigsegv 2.13 ubezeychw4 numactl 2.0.14 wfnw532l74 pkgconf 1.8.0 i5po27g2n6 zlib 1.2.12 3svypjaie5
cmake 3.23.1 tulqz4nxor libedit 3.1-20210216 zc56lv6gyj libtool 2.4.7 7fhdrxxize openmpi 4.1.3 jfxctqwar7 pmix 4.1.2 e4fax4744g zlib 1.2.12 fntvsj6xev
Note that name, version, and hash are attributes of Spack’s
internal Spec object and enclosing them in braces ensures they are
output according to your format string.
Using spack find --format allows you to retrieve just the information
you need to do things like pipe the output to typical UNIX command-line
tools like sort or uniq.
spack find --json¶
Alternatively, you can get a serialized version of Spec objects in
the JSON format using the --json option. For example, you can
get attributes for all installations of zlib by entering:
$ spack find --json zlib
[
{
"name": "zlib",
"version": "1.2.12",
"arch": {
"platform": "linux",
"platform_os": "ubuntu18.04",
"target": "x86_64"
},
"compiler": {
"name": "clang",
"version": "7.0.0"
},
"namespace": "builtin",
"parameters": {
"optimize": true,
"patches": [
"0d38234384870bfd34dfcb738a9083952656f0c766a0f5990b1893076b084b76"
],
"pic": true,
"shared": true,
"cflags": [],
"cppflags": [],
"cxxflags": [],
"fflags": [],
"ldflags": [],
"ldlibs": []
},
"patches": [
"0d38234384870bfd34dfcb738a9083952656f0c766a0f5990b1893076b084b76"
],
"hash": "3svypjaie5farii5zcgm6yfu43la2ixs"
},
{
"name": "zlib",
"version": "1.2.12",
"arch": {
"platform": "linux",
"platform_os": "ubuntu18.04",
"target": "x86_64"
},
"compiler": {
"name": "gcc",
"version": "7.5.0"
},
"namespace": "builtin",
"parameters": {
"optimize": true,
"patches": [
"0d38234384870bfd34dfcb738a9083952656f0c766a0f5990b1893076b084b76"
],
"pic": true,
"shared": true,
"cflags": [],
"cppflags": [],
"cxxflags": [],
"fflags": [],
"ldflags": [],
"ldlibs": []
},
"patches": [
"0d38234384870bfd34dfcb738a9083952656f0c766a0f5990b1893076b084b76"
],
"hash": "fntvsj6xevbz5gyq7kfa4xg7oxnaolxs"
}
The spack find --json command gives you everything we know about
the specs in a structured format. You can pipe its output to
JSON filtering tools like jq to extract just the parts you want.
Check out the basic usage docs for more examples.
Introducing the spack python command¶
What if we need to perform more advanced queries?
Spack provides the spack python command to launch a python interpreter
with Spack’s python modules available to import. It uses the underlying
python for the rest of its commands. So you can write scripts to:
- run Spack commands;
- explore abstract and concretized specs; and
- directly access other internal components of Spack.
Let’s launch a Spack-aware python interpreter by entering:
$ spack python
exit()
Spack version 0.18.0
Python 3.6.9, Linux x86_64
>>> exit()
Since we are in a python interpreter, use exit() to end
the session and return to the terminal.
Accessing the Spec object¶
Now let’s take a look at the internal representation of the Spack Spec.
As you already know, specs can be either abstract or concrete. The
specs you’ve seen in package.py files (e.g., in the install()
method) have been concrete, or fully specified. The specs you’ve typed
on the command line have been abstract. Understanding the differences
between the two types is key to using Spack’s internal API.
Let’s open another python interpreter with spack python, instantiate
the zlib spec, and check a few properties of an abstract spec:
>>> from spack.spec import Spec
>>> s = Spec('zlib target=ivybridge')
>>> s.concrete
False
>>> s.version
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/home/spack/spack/lib/spack/spack/spec.py", line 3166, in version
raise SpecError("Spec version is not concrete: " + str(self))
SpecError: Spec version is not concrete: zlib arch=linux-None-ivybridge
>>> s.versions
[:]
>>> str(s.architecture)
linux-None-ivybridge
Notice that there are Spec properties and methods that are not
accessible to abstract specs; specifically:
- an exception –
SpecError– is raised if we try to access itsversion; - there are no associated
versions; and - the spec’s operating system is
None.
Now, without exiting the interpreter, let’s concretize the spec and try again:
>>> s.concretize()
>>> s.concrete
True
>>> s.version
Version('1.2.11')
>>> s.versions
[Version('1.2.11')]
>>> str(s.architecture)
linux-ubuntu18.04-ivybridge
Notice that the concretized spec now:
- has a
version; - has a single entry in its
versionslist; and - the operating system is now
ubuntu18.04.
It is not necessary to store the intermediate abstract spec – you can
use the .concretized() method as shorthand:
>>> t = Spec('zlib target=ivybridge').concretized()
>>> s == t
True
Querying the Spack database¶
Even more powerful queries are available when we look at the information
stored in the Spack database. The Database object in Spack is in the
spack.store.db variable. We’ll interact with it mainly through the
query() method. Let’s see the documentation available for query()
using python’s built-in help() function:
>>> import spack.store
>>> help(spack.store.db.query)
Help on method query in module spack.database:
query(*args, **kwargs) method of spack.database.Database instance
Query the Spack database including all upstream databases.
Args:
query_spec: queries iterate through specs in the database and
return those that satisfy the supplied ``query_spec``. If
query_spec is `any`, This will match all specs in the
database. If it is a spec, we'll evaluate
``spec.satisfies(query_spec)``
known (bool or any, optional): Specs that are "known" are those
for which Spack can locate a ``package.py`` file -- i.e.,
Spack "knows" how to install them. Specs that are unknown may
represent packages that existed in a previous version of
Spack, but have since either changed their name or
been removed
installed (bool or any, or InstallStatus or iterable of
InstallStatus, optional): if ``True``, includes only installed
specs in the search; if ``False`` only missing specs, and if
``any``, all specs in database. If an InstallStatus or iterable
of InstallStatus, returns specs whose install status
(installed, deprecated, or missing) matches (one of) the
InstallStatus. (default: True)
explicit (bool or any, optional): A spec that was installed
following a specific user request is marked as explicit. If
instead it was pulled-in as a dependency of a user requested
spec it's considered implicit.
start_date (datetime, optional): filters the query discarding
specs that have been installed before ``start_date``.
end_date (datetime, optional): filters the query discarding
specs that have been installed after ``end_date``.
hashes (container): list or set of hashes that we can use to
restrict the search
Returns:
list of specs that match the query
(END)
We will primarily make use of the query_spec argument.
Recall that queries using the spack find command are limited to
queries of attributes with matching values, not values they do not
have. In other words, we cannot use the spack find command for
all packages that do not satisfy a certain criterion.
We can use the python interface to write these types of queries.
For example, let’s find all packages that were compiled with gcc but
do not depend on mpich. We can do this by using custom python code
and Spack database queries. We will use the spack.cmd.display_specs
for output to achieve the same printing functionality as the spack
find command:
>>> gcc_query_spec = Spec('%gcc')
>>> gcc_specs = spack.store.db.query(gcc_query_spec)
>>> result = filter(lambda spec: not spec.satisfies('^mpich'), gcc_specs)
>>> import spack.cmd
>>> spack.cmd.display_specs(result)
-- linux-ubuntu18.04-x86_64 / gcc@7.5.0 -------------------------
autoconf@2.69 libiconv@1.16 m4@1.4.18 perl@5.30.3 zlib@1.2.11
automake@1.16.2 libpciaccess@0.13.5 ncurses@6.2 pkgconf@1.7.3
gdbm@1.18.1 libsigsegv@2.12 numactl@2.0.12 readline@8.0
hdf5@1.10.6 libtool@2.4.6 openmpi@3.1.6 util-macros@1.19.1
hwloc@1.11.11 libxml2@2.9.10 patchelf@0.10 xz@5.2.5
Now we have a powerful query not available through spack find.
Let’s exit the interpreter to take us back to the command line:
>>> exit()
before generalizing the functionality for re-use.
Using scripts¶
Now let’s parameterize our script to accept arguments on the command line. With a few generalizations to use the include and exclude specs as arguments, we can create a powerful, general-purpose query script.
Open a file called find_exclude.py in your preferred editor
and add the following code:
from spack.spec import Spec
import spack.store
import spack.cmd
import sys
include_spec = Spec(sys.argv[1])
exclude_spec = Spec(sys.argv[2])
all_included = spack.store.db.query(include_spec)
result = filter(lambda spec: not spec.satisfies(exclude_spec), all_included)
spack.cmd.display_specs(result)
Notice we added importing and using the system package (sys)
to access the first and second command line arguments.
Now we can run our new script by entering the following:
$ spack python find_exclude.py %gcc ^mpich
-- linux-ubuntu18.04-x86_64 / gcc@7.5.0 -------------------------
autoconf@2.69 bzip2@1.0.8 gdbm@1.19 libedit@3.1-20210216 libpciaccess@0.16 libxml2@2.9.13 numactl@2.0.14 openssl@1.1.1o pmix@4.1.2 xz@5.2.5
automake@1.16.5 cmake@3.23.1 hdf5@1.12.2 libevent@2.1.12 libsigsegv@2.13 m4@1.4.19 openmpi@4.1.3 perl@5.34.1 readline@8.1 zlib@1.2.12
berkeley-db@18.1.40 diffutils@3.8 hwloc@2.7.1 libiconv@1.16 libtool@2.4.7 ncurses@6.2 openssh@9.0p1 pkgconf@1.8.0 util-macros@1.19.3
This is great for us, as long as we remember to use Spack’s
python command to run it.
Using the spack-python executable¶
What if we want to make our script available for others to use without
the hassle of having to remember to use spack python?
We can take advantage of the shebang line typically added as the first line of python executable files. But there is a catch, as we will soon see.
Open the find_exclude.py script we created above in your preferred
editor and add the shebang line with spack python as the arguments
to env:
#!/usr/bin/env spack python
from spack.spec import Spec
import spack.store
import spack.cmd
import sys
include_spec = Spec(sys.argv[1])
exclude_spec = Spec(sys.argv[2])
all_included = spack.store.db.query(include_spec)
result = filter(lambda spec: not spec.satisfies(exclude_spec), all_included)
spack.cmd.display_specs(result)
Then exit our editor and add execute permissions to the script before running it as follows:
$ chmod u+x find_exclude.py
$ ./find_exclude.py %gcc ^mpich
/usr/bin/env: 'spack python': No such file or directory
If you are lucky, it worked on your system, but there is no guarantee.
Some systems only support a single argument on the shebang line (see
here).
spack-python, which is a wrapper script for spack python, solves
this issue.
Bring up the file in your editor again and change the env argument
to spack-python as follows:
#!/usr/bin/env spack-python
from spack.spec import Spec
import spack.store
import spack.cmd
import sys
include_spec = Spec(sys.argv[1])
exclude_spec = Spec(sys.argv[2])
all_included = spack.store.db.query(include_spec)
result = filter(lambda spec: not spec.satisfies(exclude_spec), all_included)
spack.cmd.display_specs(result)
Exit your editor and let’s run the script again:
$ ./find_exclude.py %gcc ^mpich
-- linux-ubuntu18.04-x86_64 / gcc@7.5.0 -------------------------
autoconf@2.69 bzip2@1.0.8 gdbm@1.19 libedit@3.1-20210216 libpciaccess@0.16 libxml2@2.9.13 numactl@2.0.14 openssl@1.1.1o pmix@4.1.2 xz@5.2.5
automake@1.16.5 cmake@3.23.1 hdf5@1.12.2 libevent@2.1.12 libsigsegv@2.13 m4@1.4.19 openmpi@4.1.3 perl@5.34.1 readline@8.1 zlib@1.2.12
berkeley-db@18.1.40 diffutils@3.8 hwloc@2.7.1 libiconv@1.16 libtool@2.4.7 ncurses@6.2 openssh@9.0p1 pkgconf@1.8.0 util-macros@1.19.3
Congratulations! It will now work on any system with Spack installed.
You now have the basic tools to create your own custom Spack queries and prototype ideas. We hope one day you’ll contribute them back to Spack.