Skip to content

Commit 0e09d60

Browse files
committed
Update Spack slides and demo notes
1 parent acd08a7 commit 0e09d60

File tree

2 files changed

+32
-67
lines changed

2 files changed

+32
-67
lines changed

03_building_and_packaging/spack_demo.md

Lines changed: 17 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -1,28 +1,24 @@
11
# Packaging for High-Performance Computing (Notes)
22

3-
**Note**: It is recommended to try out Spack in a fresh Docker container. To understand how Spack itself is installed, follow Step 1 in a fresh Ubuntu container. To make things simpler from Step 2 onwards, create a container from the [spack/ubuntu-jammy](https://hub.docker.com/r/spack/ubuntu-jammy) image, so that Spack is preinstalled.
3+
**Note**: It is recommended to try out Spack in a fresh Docker container. To understand how Spack itself is installed, follow Step 1 in a fresh Ubuntu Noble container (`docker run --rm -it ubuntu:noble`). To make things simpler from Step 2 onwards, create a container from the [spack/ubuntu-noble](https://hub.docker.com/r/spack/ubuntu-noble) image, so that Spack is preinstalled.
44

55
## 1. Spack Setup/Installation
66

7-
- Git repository of Python scripts
7+
- This demo (and Spack) needs Python, Git, a C/C++ compiler, patch, make, tar, and a few more tools. Basically `build-essential`, `git`, and `python` on Ubuntu Noble.
8+
9+
- Get the Spack repository
810

911
```bash
10-
git clone -b v0.23.0 -c feature.manyFiles=true --depth=2 https://github.com/spack/spack.git
12+
git clone -b v1.1.0 --depth=2 https://github.com/spack/spack.git
1113
```
1214

13-
- `v0.23.0` is currently the latest major release
14-
15-
- **Note:** Install `curl`, `libcurl4-openssl-dev`, and `vim` to ensure that Spack v0.23.0 works in a fresh Ubuntu Jammy container.
16-
1715
- Initializing Spack with
1816

1917
```bash
2018
. <spack_prefix>/share/spack/setup-env.sh
2119
```
2220

23-
will set `SPACK_ROOT` and also add `spack` to `PATH`.
24-
25-
Note that the `.` operator will run the commands in the supplied script as if we would type the commands supplied by the script in the shell ourselves. In bash the `.` operator is equivalent to `source`. However, `source` is not specified in POSIX and thus using `.` is likely to work on more platforms.
21+
will set `SPACK_ROOT` and also add `spack` to `PATH`. Note that the `.` operator will run the commands in the supplied script as if we would type the commands supplied by the script in the shell ourselves. In bash the `.` operator is equivalent to `source`. However, `source` is not specified in POSIX and thus using `.` is likely to work on more platforms.
2622

2723
- Finish Spack setup
2824

@@ -40,25 +36,23 @@
4036

4137
prints the list of compilers that Spack has added.
4238

43-
- Find external packages (optional)
39+
- Find external packages
4440

4541
```bash
4642
spack external find
4743
```
4844

4945
This command tries to find preinstalled software packages which are also in Spack's package database. This way we do not have to recompile everything from scratch.
5046

51-
**Note** `spack external find` is an experimental feature and might fail. System packages [can be defined manually](https://spack.readthedocs.io/en/latest/getting_started.html#system-packages).
47+
Found packages (including version, configuration etc.) are stored in `~/.spack/packages.yaml`. There one can add further packages manually. Look at contents of this file.
5248

53-
- Found packages (including version, configuration etc.) are stored in `~/.spack/packages.yaml`. There one can add further packages manually. **Note**: Maybe show content of this file.
54-
55-
- Concretize a spec to trigger bootstrap process (optional)
49+
- Concretize a spec to trigger bootstrap process
5650

5751
```bash
5852
spack spec zlib
5953
```
6054

61-
This will try to concretize the package and its dependencies. `clingo`, Spack's concretizer, is needed for this. If one calls this command the very first time, a bootstrapping process is triggered that installs `clingo`.
55+
This will try to concretize the package and its dependencies. `clingo`, Spack's concretizer, is needed for this. Running `spack spec` for the first time triggers a bootstrapping process that installs `clingo`.
6256

6357
## 2. Package Installation/Management with Spack
6458

@@ -68,15 +62,7 @@
6862
spack info zlib
6963
```
7064

71-
- Install a simple example package
72-
73-
```bash
74-
spack spec zlib
75-
```
76-
77-
This will trigger the bootstrap process of Spack. It installs `clingo`, the dependency resolver, if not done before.
78-
79-
- Install `zlib`
65+
- Install the package
8066

8167
```bash
8268
spack install --reuse zlib
@@ -102,7 +88,7 @@
10288

10389
Spack will fetch the archive and try to deduce some basic settings. It will also check for other releases and asks whether to add checksums. We want all checksums.
10490

105-
- Afterwards Spack drops us in the boilerplate package.
91+
- Spack immediately drops you in a boilerplate package file.
10692

10793
```Python
10894
class Helloworld(CMakePackage):
@@ -133,7 +119,7 @@
133119

134120
At least the `FIXME` statements should be fixed before releasing the package. The Spack package recipe is written in Python so we can also do common Python magic here.
135121

136-
- Spack has deduced a good number of information already.
122+
- Spack has deduced a good amount of information already.
137123
- We work with CMake
138124
- There are 3 releases of the software
139125
- The URL to download packages
@@ -259,16 +245,16 @@
259245

260246
### References
261247

262-
- [EasyBuild](https://github.com/easybuilders/easybuild)
263-
- [EasyConfigs](https://github.com/easybuilders/easybuild-easyconfigs)
264248
- [Spack](https://spack.io/)
265249
- [Spack docs](https://spack.readthedocs.io/en/latest/)
266250
- [Spack 101 tutorials](https://spack-tutorial.readthedocs.io/en/latest/)
251+
- [EasyBuild](https://github.com/easybuilders/easybuild)
252+
- [EasyConfigs](https://github.com/easybuilders/easybuild-easyconfigs)
267253
- [archspec project](https://github.com/archspec/)
268254

269-
### Talks
255+
### Interesting Talks On This Topic
270256

271-
Talks at FOSDEM
257+
FOSDEM talks:
272258

273259
- 2020: [Spack's new Concretizer](https://archive.fosdem.org/2020/schedule/event/dependency_solving_not_just_sat/)
274260
- 2020: [Build for your microarchitecture: experiences with Spack and archspec](https://archive.fosdem.org/2020/schedule/event/archspec/)

03_building_and_packaging/spack_slides.md

Lines changed: 15 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ slideOptions:
3838

3939
## Learning goals
4040

41-
- Understand the challenges when bringing (your) software to supercomputers.
41+
- Understand the challenges of deploying software to supercomputers.
4242
- Use Spack to install software.
4343
- Create a Spack package for your own software.
4444

@@ -47,20 +47,21 @@ slideOptions:
4747
## Introduction 1/2
4848

4949
- Scientific software installation on supercomputers faces challenges:
50-
- Parallel architectures (MPI, OpenMP, HPX...)
51-
- Special hardware (accelerators, network...)
52-
- Greatly varying architectures (AMD, Intel, ARM...)
53-
- Missing dependencies (too new, too old...)
54-
- Admins will not/cannot install software for you, you do not have superuser rights
50+
- Parallel architectures (MPI, OpenMP, HPX, ...)
51+
- Special hardware (accelerators, network, ...)
52+
- Greatly varying architectures (AMD, Intel, ARM, ...)
53+
- Missing dependencies (too new, too old)
54+
- Admins will not/cannot install software for you
55+
- You do not have superuser rights
5556

5657
---
5758

5859
## Introduction 2/2
5960

60-
- Software should have best possible performance
61+
- Performance is critical on supercomputers
6162
- Software needs to be **compiled** to get best performance (special libraries, compilers...)
6263
- Optimized compilation settings (best settings not always obvious)
63-
- Optimal settings depend on platform (supercomputers might be heterogeneous, different generation of CPUs e.g.)
64+
- Optimal settings depend on platform (supercomputers might be heterogeneous, for e.g., different generation of CPUs)
6465
- **HPC-friendly package managers** can help with
6566
- organizing software installations
6667
- compilation of software
@@ -87,14 +88,14 @@ slideOptions:
8788
- Common scientific software included
8889
- No superuser rights needed
8990
- Spack's [dependencies](https://spack.readthedocs.io/en/latest/getting_started.html#system-prerequisites) commonly available on supercomputers
90-
- Deals with dependency resolution, compilation flags and **compilation**
91+
- Deals with dependency resolution, compilation flags and compilation
9192
- [Open-source project on GitHub](https://github.com/spack/spack)
9293
- Valuable resources:
9394
- [Good starter guides](https://spack-tutorial.readthedocs.io/en/latest/)
9495
- [Extensive documentation](https://spack.readthedocs.io)
9596
- [Spack Slack](https://slack.spack.io/) for communication with community
96-
- Heavily relies on concretizer [`clingo`](https://spack.readthedocs.io/en/latest/getting_started.html#bootstrapping-clingo)
97-
- Prefers to install newest version of packages
97+
- Spack [bootstraps](https://spack.readthedocs.io/en/latest/bootstrapping.html) its own dependencies the first time it is used
98+
- Prefers to install newest version of packages
9899

99100
---
100101

@@ -109,28 +110,13 @@ slideOptions:
109110

110111
---
111112

112-
## Spack Installation
113-
114-
- Dependencies: Python, Git, C/C++ compiler, patch, make, tar...
115-
- Basically `build-essential`, `git`, and `python` on Ubuntu
116-
- May be old, install newer versions with Spack if needed
117-
- Installation in user-writable location
118-
119-
```bash
120-
git clone -b v0.23.0 -c feature.manyFiles=true --depth=2 https://github.com/spack/spack.git
121-
```
122-
123-
- `v0.23.0` is currently the latest major release
124-
125-
---
126-
127113
## Demo 1: Spack Installation
128114

129115
---
130116

131117
## Spack Configuration and Caches
132118

133-
- Permanently set up Spack
119+
- Set up Spack
134120

135121
```bash
136122
. <spack_prefix>/share/spack/setup-env.sh
@@ -150,7 +136,7 @@ slideOptions:
150136

151137
- Spack works with `spec`s (`spec`=specification)
152138

153-
```
139+
```bash
154140
spack install [email protected]+mpi -shared~suite-sparse
155141
```
156142

@@ -166,12 +152,6 @@ slideOptions:
166152
spack unload [email protected]
167153
```
168154

169-
- [General syntax](https://spack.readthedocs.io/en/latest/command_index.html)
170-
171-
```bash
172-
spack COMMAND OPTION
173-
```
174-
175155
---
176156

177157
## Demo 2: Package Installation and Management
@@ -345,7 +325,7 @@ slideOptions:
345325

346326
## Demo 3: Package Creation
347327

348-
- Create package for our [HelloWorld example](https://github.com/Simulation-Software-Engineering/HelloWorld)
328+
- Create package for our [HelloWorld](https://github.com/Simulation-Software-Engineering/HelloWorld) code
349329
1. Create boilerplate package
350330
2. Add package details
351331
3. Verify package
@@ -361,7 +341,6 @@ slideOptions:
361341
- [Chained installation](https://spack.readthedocs.io/en/latest/chain.html)
362342
- [Stacks (build matrices)](https://spack-tutorial.readthedocs.io/en/latest/tutorial_stacks.html)
363343
- [Software management (modules)](https://spack.readthedocs.io/en/latest/module_file_support.html)
364-
- ...
365344

366345
---
367346

0 commit comments

Comments
 (0)