Tutorial: Spack 101

This is an introduction to Spack with lectures and live demos. It was last presented at the ISC-HPC 2024 Conference, May 12, 2023. The event was a full-day tutorial.

You can use these materials to teach a course on Spack at your own site, or you can just skip ahead and read the live demo scripts to see how Spack is used in practice.

Slides

Download Slides.

Full citation: Todd Gamblin, Gregory Becker, Massimiliano Culpo, Michael Kuhn, Adam Stewart, and Harmen Stoppels. Managing HPC Software Complexity with Spack. ISC-HPC 2024 (ISC24), Hamburg, Germany. May 12, 2024.

Video

For the last video of a Spack tutorial like this one, see the AWS/RADIUSS 2023 version

Live Demos

We provide scripts that take you step-by-step through basic Spack tasks. They correspond to sections in the slides above.

To run through the scripts, we provide the spack/tutorial container image. You can invoke

$ docker pull ghcr.io/spack/tutorial:isc24
$ docker run -it ghcr.io/spack/tutorial:isc24

to start using the container. You should now be ready to run through our demo scripts:

1. Basic Installation Tutorial

2. Environments Tutorial

3. Binary Caches Tutorial

4. Stacks Tutorial

5. Package Creation Tutorial

Other sections from past tutorials are also available, although they may not be kept up-to-date as frequently:

1. Configuration Tutorial

2. Developer Workflows Tutorial

3. Scripting with Spack

4. Module Files Tutorial

5. Spack Package Build Systems

6. Advanced Topics in Packaging

Full contents:

Links

• Main Spack Documentation

Tutorial

• Basic Installation Tutorial
  • Installing Spack
  • What is in Spack?
  • Installing Packages
  • Uninstalling Packages
  • Advanced spack find Usage
  • Customizing Compilers
• Environments Tutorial
  • Environment Basics
    • Creating and activating environments
    • Installing packages
    • Creating an environment incrementally
    • Using packages
    • Uninstalling packages
  • The spack.yaml file
    • Editing environment configuration
    • Re-concretizing the environment
  • Building in environments
  • Reproducing builds
    • Environment files
    • Managed versus independent environments
    • Reviewing a managed environment
    • Creating an independent environment
    • Updating an installed environment
    • Reviewing spack.lock
    • Reproducing an environment
  • More information
    • Setting up and building environments
    • Using environments
    • Finding examples of environments
• Binary Caches Tutorial
  • Setting up an OCI build cache on GitHub Packages
  • Installing from the build cache
  • Reuse of binaries from a build cache
  • Creating runnable container images
  • Spack environments as container images
  • Do I need docker or buildah?
  • Relocation
  • Using build caches in CI
  • Summary
• Stacks Tutorial
  • Setup the compiler
  • Install a software stack
    • Tuning concretizer options for a stack
    • Spec matrices
    • Reusable definitions
    • Conditional definitions
    • Other useful features
  • Make the software stack easy to use
    • View descriptors
    • Module files
  • Summary
• Package Creation Tutorial
  • What is a Spack Package?
  • Getting Started
  • Creating the Package File
  • Adding Package Documentation
  • Adding Dependencies
  • Debugging Package Builds
    • Reviewing the Build Log
    • Building Manually
  • Specifying Configure Arguments
  • Adding Variants
  • Adding Tests
  • Querying the Spec Object
    • Querying Spec Versions
    • Querying Spec Names
    • Querying Variants
  • Multiple Build Systems
  • Cleaning Up
  • More information
    • Testing an installation
    • Customizing package-related environments
    • Using other build systems
    • Making a package externally detectable

Additional sections

• Configuration Tutorial
  • Configuration from the command line
  • Configuration Scopes
    • Platform-specific scopes
  • YAML Format
  • Compiler Configuration
    • Compiler flags
    • Advanced compiler configuration
  • Configuring Package Preferences
    • Variant preferences
    • External packages
    • Installation permissions
  • High-level Config
  • Conclusion
• Developer Workflows Tutorial
  • Installing from local source
  • Development iteration cycles
  • Workflow Summary
• Scripting with Spack
  • Setting up the tutorial
  • Scripting with spack find
    • spack find --format
    • spack find --json
  • Introducing the spack python command
    • Accessing the Spec object
    • Querying the Spack database
    • Using scripts
  • Using the spack-python executable
• Module Files Tutorial
  • Setup for the Tutorial
    • Build a module tool
    • Add a new compiler
    • Build the software that will be used in the tutorial
  • What are Module Files?
    • Module Systems
    • How does Spack generate module files?
    • Modules vs spack load
  • Non-hierarchical Module Files
    • Filter unwanted modifications to the environment
    • Prevent some module files from being generated
    • Change module file naming
    • Add custom environment modifications
    • Autoload dependencies
  • Hierarchical Module Files
    • Core/Compiler/MPI
    • Add LAPACK to the hierarchy
  • Working with Templates
    • Module file templates
    • Extend the default templates
    • Restore settings for future sections
• Spack Package Build Systems
  • Package Class Hierarchy
  • Package
  • Autotools
  • Makefile
  • CMake
  • PythonPackage
  • Other Build Systems
• Advanced Topics in Packaging
  • Setup for the Tutorial
  • Modifying a Package’s Build Environment
    • Set environment variables in dependent packages at build-time
    • Set environment variables in your own package
  • Retrieving Library Information
    • Accessing dependency libraries
    • Providing libraries to dependents
  • Other Packaging Topics
    • Attach attributes to other packages
    • Extra query parameters