Skip to content

Storage GC, acceptable-use enforcement, and data residency #49

@jeremymanning

Description

@jeremymanning

Description

Per spec T130-T131 and T144a:

  1. Storage cap and GC (T144a): per-donor storage cap enforcement and content GC for expired/withdrawn data
  2. Acceptable-use filter (T130): refuse unauthorized scanning, malware, illegal content, surveillance, credential cracking at job submission
  3. Shard residency enforcement (T131): per-donor shard-category allowlist enforcement in data plane placement

Requirements

  • Storage GC: track per-donor storage usage, evict expired/orphaned data, enforce configurable cap
  • Acceptable-use: content classification at submission time, reject prohibited workloads
  • Shard residency: enforce data residency constraints (EU, US, UK, JP resident data placed only on nodes in matching jurisdiction)
  • Geographic shard placement: ≥3 continents, ≤2 shards/country per the erasure coding spec

Success Criteria

  • Per-donor storage usage tracked and capped
  • Expired data garbage collected automatically
  • Prohibited workload classes rejected at submission
  • Data residency constraints enforced in shard placement
  • Geographic diversity maintained in erasure-coded placement
  • Integration tests for each enforcement mechanism
  • cargo test passes

Testing (Principle V)

  • Fill donor storage to cap → verify new data rejected/old data evicted
  • Submit malware-class workload → verify rejected
  • Submit EU-resident data → verify shards placed only on EU nodes
  • Verify geographic diversity: remove nodes → verify placement adjusts

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions