fix: per-CPU memory accounting and --ntasks in SLURM executor (#352)

[mpvanderschelling]

Summary

Fixes #352. The SLURM executor hard-coded per-node --mem + --nodes and never emitted --ntasks, so every job (orchestrator and every step) was rejected at submission time on sites whose job_submit filter mandates per-CPU memory accounting — notably TU Delft's DelftBlue. Because --mem and --mem-per-cpu are mutually exclusive (fatal) at every SLURM site, extra_sbatch could not work around it — the fix has to omit --mem, which only the renderer can do.

Changes

• SlurmResources gains two fields: ntasks: int = 1 and mem_per_cpu: str | None = None. When mem_per_cpu is set, mem is ignored (documented, not validated).
• Both header renderers (render_sbatch_script, render_orchestrator_script) now share one _render_resource_directives(res) helper so they cannot drift:
  • --ntasks={res.ntasks} is emitted unconditionally on every cluster (confirmed harmless on plain-SLURM sites like Oscar).
  • --mem-per-cpu replaces --mem when mem_per_cpu is set.
  • --nodes is omitted only when mem_per_cpu is set and nodes == 1; an explicit nodes > 1 is always emitted.
• ADR 0003 records the rationale, including the caveat that a DelftBlue pipeline must also override Pipeline.orchestrator_resources with a mem_per_cpu (the built-in orchestrator default stays per-node by design).
• Regenerated all 6 tests/pipeline/golden/*.sh (one --ntasks=1 line each); extended test_resources.py and test_slurm.py (new TestPerCpuMemory); bumped version to 2.4.0.

Backward compatibility

mem_per_cpu defaults to None, so per-node --mem output is unchanged wherever it is not opted into. The only behavior change for existing configs is the added --ntasks=1 line.

Verification

• uv run pytest (full suite, [tests,all] extras + coverage gate): 1133 passed, coverage 91.84% (gate 85%).
• pre-commit run --all-files: passing (ruff format/check, toml-sort).
• Live on DelftBlue via sbatch --test-only: new step + orchestrator scripts accepted on compute; the old-style header rejected with the exact error from the issue.

🤖 Generated with Claude Code