Tutorial: Spack 101

This is an introduction to Spack with lectures and live demos. It was last presented at the HPCIC 2024 HPC Tutorials, August 6th, 2024. The event was two online half-day tutorials.

You can use these materials to teach a course on Spack at your own site, or you can just skip ahead and read the live demo scripts to see how Spack is used in practice.

Slides

Download Slides.

Full citation: Todd Gamblin, Gregory Becker, Alec Scott. Managing HPC Software Complexity with Spack. HPCIC Tutorials 2024, Livermore, California. July 22, 2024.

Video

The recording of the virtual event is available in two parts:

• Part 1
  
• Part 2
  

Live Demos

We provide scripts that take you step-by-step through basic Spack tasks. They correspond to sections in the slides above.

To run through the scripts, we provide the spack/tutorial container image. You can invoke

$ docker pull ghcr.io/spack/tutorial:hpcic24
$ docker run -it ghcr.io/spack/tutorial:hpcic24

to start using the container. You should now be ready to run through our demo scripts:

1. Basic Installation Tutorial

2. Environments Tutorial

3. Configuration Tutorial

4. Package Creation Tutorial

5. Stacks Tutorial

6. Developer Workflows Tutorial

7. Binary Caches Tutorial

8. Scripting with Spack

Other sections from past tutorials are also available, although they may not be kept up-to-date as frequently:

1. Module Files Tutorial

2. Spack Package Build Systems

3. Advanced Topics in Packaging

Full contents:

Links

• Main Spack Documentation

Tutorial

• Basic Installation Tutorial
  • Installing Spack
  • What is in Spack?
  • Installing Packages
  • Uninstalling Packages
  • Advanced spack find Usage
  • Customizing Compilers
• Environments Tutorial
  • Environment Basics
    • Creating and activating environments
    • Installing packages
    • Creating an environment incrementally
    • Using packages
    • Uninstalling packages
  • The spack.yaml file
    • Editing environment configuration
    • Re-concretizing the environment
  • Building in environments
  • Reproducing builds
    • Environment files
    • Managed versus independent environments
    • Reviewing a managed environment
    • Creating an independent environment
    • Updating an installed environment
    • Reviewing spack.lock
    • Reproducing an environment
  • More information
    • Setting up and building environments
    • Using environments
    • Finding examples of environments
• Configuration Tutorial
  • Configuration from the command line
  • Configuration Scopes
    • Platform-specific scopes
  • YAML Format
  • Compiler Configuration
    • Compiler flags
    • Advanced compiler configuration
  • Configuring Package Preferences
    • Variant preferences
    • External packages
    • Installation permissions
  • High-level Config
  • Conclusion
• Package Creation Tutorial
  • What is a Spack Package?
  • Getting Started
  • Creating the Package File
  • Adding Package Documentation
  • Adding Dependencies
  • Debugging Package Builds
    • Reviewing the Build Log
    • Building Manually
  • Specifying Configure Arguments
  • Adding Variants
  • Adding Tests
  • Querying the Spec Object
    • Querying Spec Versions
    • Querying Spec Names
    • Querying Variants
  • Multiple Build Systems
  • Cleaning Up
  • More information
    • Testing an installation
    • Customizing package-related environments
    • Using other build systems
    • Making a package externally detectable
• Stacks Tutorial
  • Setup the compiler
  • Install a software stack
    • Tuning concretizer options for a stack
    • Spec matrices
    • Reusable definitions
    • Conditional definitions
    • Other useful features
  • Make the software stack easy to use
    • View descriptors
    • Module files
  • Summary
• Developer Workflows Tutorial
  • Installing from local source
  • Development iteration cycles
  • Workflow Summary
• Binary Caches Tutorial
  • Setting up an OCI build cache on GitHub Packages
  • Installing from the build cache
  • Reuse of binaries from a build cache
  • Creating runnable container images
  • Spack environments as container images
  • Do I need docker or buildah?
  • Relocation
  • Using build caches in CI
  • Summary
• Scripting with Spack
  • Setting up the tutorial
  • Scripting with spack find
    • spack find --format
    • spack find --json
  • Introducing the spack python command
    • Accessing the Spec object
    • Querying the Spack database
    • Using scripts
  • Using the spack-python executable

Additional sections

• Module Files Tutorial
  • Setup for the Tutorial
    • Build a module tool
    • Add a new compiler
    • Build the software that will be used in the tutorial
  • What are Module Files?
    • Module Systems
    • How does Spack generate module files?
    • Modules vs spack load
  • Non-hierarchical Module Files
    • Filter unwanted modifications to the environment
    • Prevent some module files from being generated
    • Change module file naming
    • Add custom environment modifications
    • Autoload dependencies
  • Hierarchical Module Files
    • Core/Compiler/MPI
    • Add LAPACK to the hierarchy
  • Working with Templates
    • Module file templates
    • Extend the default templates
    • Restore settings for future sections
• Spack Package Build Systems
  • Package Class Hierarchy
  • Package
  • Autotools
  • Makefile
  • CMake
  • PythonPackage
  • Other Build Systems
• Advanced Topics in Packaging
  • Setup for the Tutorial
  • Modifying a Package’s Build Environment
    • Set environment variables in dependent packages at build-time
    • Set environment variables in your own package
  • Retrieving Library Information
    • Accessing dependency libraries
    • Providing libraries to dependents
  • Other Packaging Topics
    • Attach attributes to other packages
    • Extra query parameters