User Story
As the WMS consumer,
I want an ARCBackend that submits and monitors jobs on an ARC 7 CE via REST with token auth,
So that the primary grid-CE path (DIRAC's AREX CE) exists on the new contract, validated against
the arc-slurm stack.
Feature Description
Essential half of the AREX port (IC-ADR-001 migration map):
ARCBackend: implements Submitter + Monitor against the ARC REST
API (/arex/rest/1.0) using pyarcrest (evaluate fitness; fall back to httpx against the REST
spec if it can't do async/bulk cleanly — record the choice).
- Job description generation from
SubmissionSpec (XRSL or ADL — pick one, document why;
DIRAC builds XRSL), incl. input-sandbox declaration and queue targeting.
- Bulk semantics: ARC REST supports multi-job operations — submit
count copies and poll many
ids per call; per-id failures ride the outcome maps.
- State normalisation: ARC state map →
JobStatus (port DIRAC's STATES_MAP, incl. the
auto-kill-on-Hold question — surface as policy, don't hide it in the backend).
- Token auth per IC-ADR-003: bearer header, audience exposure. Proxy/delegation
is issue-14.
OutputRetriever (destructive=False): list the ARC session dir, stream every member into
dest file-by-file, return the JobOutput manifest (.stdout/.stderr/.files/.log).
Bounded materialisation enforced inline: byte ceiling mid-stream, file-count ceiling,
per-transfer timeout, path containment (../absolute members rejected) — IC-ADR-001 §4.
Cancellable.kill / Purgeable.purge (DIRAC killJob/cleanJob): bulk, per-id outcomes.
LoadReporter.counts() from the CE info endpoint.
SupportsLiveDiagnostics.diagnostics(): the diagnose/errors endpoint — the one genuinely
independent, non-destructive pre-completion fetch.
- Proxy/delegation auth: create delegation (CSR → sign with the
proxy chain → upload), renew, AlwaysIncludeProxy parity where tokens+proxy coexist. Add the
config/proxy/ configuration axis to the arc-slurm stack (ADR-002 §7) to test it.
- Fetch-twice regression test proving re-fetchability (
destructive=False) — the counterpart
of HTCondor's one-shot test.
- Phase 2 wiring: contract suite (submit/status subset) against the
arc-slurm stack
(markers remote and arc).
Definition of Done
Alternatives Considered
- Wrapping the
arc CLI tools — spawns processes, brittle parsing; REST is first-class in ARC 7
and DIRAC's AREX already proves it.
Related Issues
Additional Context
ARC floor is 7.x (IC-ADR-002 §8). DIRAC reference: `AREXComputingElement.py
User Story
As the WMS consumer,
I want an
ARCBackendthat submits and monitors jobs on an ARC 7 CE via REST with token auth,So that the primary grid-CE path (DIRAC's AREX CE) exists on the new contract, validated against
the
arc-slurmstack.Feature Description
Essential half of the AREX port (IC-ADR-001 migration map):
ARCBackend: implementsSubmitter+Monitoragainst the ARC RESTAPI (
/arex/rest/1.0) usingpyarcrest(evaluate fitness; fall back to httpx against the RESTspec if it can't do async/bulk cleanly — record the choice).
SubmissionSpec(XRSL or ADL — pick one, document why;DIRAC builds XRSL), incl. input-sandbox declaration and queue targeting.
countcopies and poll manyids per call; per-id failures ride the outcome maps.
JobStatus(port DIRAC'sSTATES_MAP, incl. theauto-kill-on-
Holdquestion — surface as policy, don't hide it in the backend).is issue-14.
OutputRetriever(destructive=False): list the ARC session dir, stream every member intodestfile-by-file, return theJobOutputmanifest (.stdout/.stderr/.files/.log).Bounded materialisation enforced inline: byte ceiling mid-stream, file-count ceiling,
per-transfer timeout, path containment (
../absolute members rejected) — IC-ADR-001 §4.Cancellable.kill/Purgeable.purge(DIRACkillJob/cleanJob): bulk, per-id outcomes.LoadReporter.counts()from the CE info endpoint.SupportsLiveDiagnostics.diagnostics(): thediagnose/errorsendpoint — the one genuinelyindependent, non-destructive pre-completion fetch.
proxy chain → upload), renew,
AlwaysIncludeProxyparity where tokens+proxy coexist. Add theconfig/proxy/configuration axis to thearc-slurmstack (ADR-002 §7) to test it.destructive=False) — the counterpartof HTCondor's one-shot test.
arc-slurmstack(markers
remote and arc).Definition of Done
arc-slurm, both token and proxyconfigs
count > 1submission bundles oneSubmissionwith per-copy failure reportingJobStatus.UNKNOWN(never dropped)output)
arc-slurmstack gainsconfig/proxy/; manifest updatedintercede.backendsasarcAlternatives Considered
arcCLI tools — spawns processes, brittle parsing; REST is first-class in ARC 7and DIRAC's AREX already proves it.
Related Issues
Additional Context
ARC floor is 7.x (IC-ADR-002 §8). DIRAC reference: `AREXComputingElement.py