|
1 | | -EpiModelHPC |
2 | | -================ |
3 | | - <!-- badges: start --> |
4 | | - [](https://github.com/EpiModel/EpiModelHPC/actions) |
5 | | - <!-- badges: end --> |
| 1 | +# EpiModelHPC |
6 | 2 |
|
7 | | -EpiModelHPC is an R package that provides extensions for simulating stochastic network models in EpiModel on high-performance computing (HPC) systems. Functionality provided to simulate models in parallel, with checkpointing functions to save and restore simulation work. |
| 3 | +<!-- badges: start --> |
| 4 | +[](https://github.com/EpiModel/EpiModelHPC/actions) |
| 5 | +<!-- badges: end --> |
8 | 6 |
|
9 | | -While there are many potential HPCs systems, this software is developed with the standard within large-scale scientific computing: linux-based clusters that operate job scheduling software running Slurm. These types of system are not necessary for running EpiModelHPC: the functionality of this package may be useful in any system that supports parallelization, including desktop computers with multiple cores. |
| 7 | +EpiModelHPC extends the [EpiModel](https://github.com/EpiModel/EpiModel) R package for running stochastic network epidemic models on high-performance computing (HPC) systems. If you are already using EpiModel's `netsim` function to simulate epidemic dynamics on local hardware and need to scale up -- running hundreds of simulations across parameter scenarios on a multi-node cluster -- EpiModelHPC provides the tools to get there. |
| 8 | + |
| 9 | +## How EpiModelHPC Relates to EpiModel |
| 10 | + |
| 11 | +[EpiModel](https://www.epimodel.org/) is the core R package for simulating mathematical models of infectious disease dynamics using stochastic, individual-based network models based on exponential-family random graph models (ERGMs). EpiModel handles network estimation (`netest`), epidemic simulation (`netsim`), and analysis of results on a single machine. |
| 12 | + |
| 13 | +EpiModelHPC does not replace any of this. Instead, it wraps and extends EpiModel's simulation engine with functionality needed when models become too computationally intensive for a single workstation: |
| 14 | + |
| 15 | +- **Parallelization**: Distributes simulation replicates across multiple cores on an HPC node. |
| 16 | +- **Checkpointing**: Automatically saves intermediate simulation state at configurable intervals, so that long-running jobs can resume from where they left off if interrupted (e.g., by a wall-time limit). |
| 17 | +- **Scenario Batching**: Runs multiple parameter scenarios (defined via `EpiModel::create_scenario_list`) in batched parallel jobs, with deterministic file naming for downstream merging. |
| 18 | +- **Result Merging**: Merges outputs from distributed batch jobs back into single simulation objects or tidy data frames for analysis. |
| 19 | + |
| 20 | +If your simulations complete in reasonable time on a laptop, you likely do not need this package. EpiModelHPC is designed for the point at which you need to run large numbers of replicates, sweep across many scenarios, or your model's per-replicate runtime exceeds what is practical without job scheduling and checkpointing. |
| 21 | + |
| 22 | +## How slurmworkflow Fits In |
| 23 | + |
| 24 | +[slurmworkflow](https://github.com/EpiModel/slurmworkflow) is a companion R package that provides a general-purpose framework for defining and submitting multi-step Slurm job workflows from R. It handles the mechanics of writing sbatch scripts, managing job dependencies, and organizing output directories. |
| 25 | + |
| 26 | +EpiModelHPC builds directly on slurmworkflow by providing **step templates** -- pre-built workflow steps tailored to common EpiModel tasks: |
| 27 | + |
| 28 | +| EpiModelHPC Step Template | Purpose | |
| 29 | +|---|---| |
| 30 | +| `step_tmpl_netsim_scenarios()` | Submit scenario-based `netsim` simulations as a Slurm array job | |
| 31 | +| `step_tmpl_merge_netsim_scenarios()` | Merge batched simulation files into one file per scenario | |
| 32 | +| `step_tmpl_merge_netsim_scenarios_tibble()` | Convert merged results to tidy tibble format | |
| 33 | +| `step_tmpl_netsim_swfcalib_output()` | Run simulations using calibrated parameters from [swfcalib](https://github.com/EpiModel/swfcalib) | |
| 34 | +| `step_tmpl_renv_restore()` | Ensure the HPC project environment is up to date via `renv` | |
| 35 | + |
| 36 | +Each step template has a corresponding standalone function (e.g., `netsim_scenarios()`) that runs the same logic locally for testing before submitting to the cluster. |
| 37 | + |
| 38 | +A typical applied workflow looks like: |
| 39 | + |
| 40 | +1. **Estimate networks** locally with `EpiModel::netest`. |
| 41 | +2. **Test simulations** locally with `netsim_scenarios()` on a small number of replicates. |
| 42 | +3. **Define a slurmworkflow** using `step_tmpl_netsim_scenarios()` and `step_tmpl_merge_netsim_scenarios()` to run at scale on the cluster. |
| 43 | +4. **Merge and analyze** results locally or on the cluster. |
| 44 | + |
| 45 | +EpiModelHPC also provides pre-configured cluster settings for specific HPC environments (`swf_configs_hyak()` for the University of Washington HYAK cluster, `swf_configs_rsph()` for the Emory RSPH cluster) that supply sensible default sbatch options and R module-loading commands. |
| 46 | + |
| 47 | +## Installation |
| 48 | + |
| 49 | +EpiModelHPC and its companion packages are hosted on GitHub. Install with: |
10 | 50 |
|
11 | | -### Installation |
12 | | -This software is currently hosted on Github only. Install it using the `remotes` package: |
13 | 51 | ```r |
14 | 52 | if (!require("remotes")) install.packages("remotes") |
15 | 53 | remotes::install_github("EpiModel/EpiModelHPC") |
16 | 54 | ``` |
| 55 | + |
| 56 | +This will also install `slurmworkflow` and `swfcalib` from their GitHub repositories. |
| 57 | + |
| 58 | +## Key Functions |
| 59 | + |
| 60 | +### Simulation |
| 61 | +- **`netsim_hpc()`** -- Run `netsim` in parallel with automatic checkpointing. Best for single-scenario runs where checkpoint/resume is the primary need. |
| 62 | +- **`netsim_scenarios()`** -- Run multiple scenarios locally with batched parallelization. Mirrors `step_tmpl_netsim_scenarios()` for local testing. |
| 63 | + |
| 64 | +### Checkpointing |
| 65 | +- **`check_cp()`** / **`initialize_cp()`** / **`save_cpdata()`** -- Low-level checkpointing utilities used internally by `netsim_hpc()`. Checkpoint data are saved to `data/sim<N>/` directories and cleaned up on successful completion. |
| 66 | + |
| 67 | +### File Management |
| 68 | +- **`merge_netsim_scenarios()`** -- Merge per-batch simulation files into one `netsim` object per scenario. |
| 69 | +- **`merge_netsim_scenarios_tibble()`** -- Convert scenario results to a single tidy tibble per scenario with configurable column selection and time-step truncation. |
| 70 | +- **`get_scenarios_batches_infos()`** / **`get_scenarios_tibble_infos()`** -- Inspect output directories to list available simulation files and their associated scenarios. |
| 71 | + |
| 72 | +### HPC Configuration |
| 73 | +- **`swf_configs_hyak()`** / **`swf_configs_rsph()`** -- Return lists of sbatch options, renv build settings, and R module-loading commands for supported clusters. |
| 74 | +- **`pull_env_vars()`** -- Extract Slurm environment variables (e.g., `SLURM_ARRAY_TASK_ID`) into R's global environment. |
| 75 | + |
| 76 | +## System Requirements |
| 77 | + |
| 78 | +While developed for Linux-based HPC clusters running the [Slurm](https://slurm.schedmd.com/) workload manager, the core parallelization and checkpointing functionality works on any system with multiple cores, including macOS and Windows workstations. The slurmworkflow integration and step templates are specific to Slurm-managed clusters. |
| 79 | + |
| 80 | +## Resources |
| 81 | + |
| 82 | +- **EpiModel website**: <https://www.epimodel.org/> |
| 83 | +- **EpiModelHPC documentation**: <https://epimodel.github.io/EpiModelHPC/> |
| 84 | +- **slurmworkflow**: <https://github.com/EpiModel/slurmworkflow> |
| 85 | +- **Bug reports**: <https://github.com/EpiModel/EpiModelHPC/issues> |
0 commit comments