Skip to content

Implement consume-path decryption with per-consumer authorization in kafka-cache-client #1689

@jfallows

Description

@jfallows

Describe the desired outcome from the user's perspective
As a Zilla operator, I want the kafka-cache-client to decrypt encrypted messages per consumer authorization on the consume path, so that each consumer receives exactly the fields they are permitted to see — with unauthorized fields either redacted to a default value or omitted entirely — and without any encryption metadata headers visible to the application.

Acceptance criteria

  • Call encryption.decryptField(fieldId, ...) per field using the model's FieldEncryption strategy
  • Embedded mode (EmbeddedFieldEncryption):
    • Reads zilla:key → vault key name; zilla:edek → base64 decode → EDEK bytes; calls vault to decrypt EDEK → DEK
    • Reads zilla:edata → block descriptors; zilla:efields → field-ids (one per block); correlates block[i] ↔ field-id[i]
  • Per-field authorization outcomes:
    • decrypt: seek to offset, decrypt in place → plaintext restored
    • redact: replace field at offset with a configured default value (null, empty string, 0) — schema compatibility preserved
    • omit: field absent from message entirely — field existence itself is sensitive
  • unauthorized: configured per tag or per named field in the model vault section
  • Strip zilla:key, zilla:edek, zilla:edata, zilla:efields from all messages delivered to consumers — these are internal infrastructure details
  • Same cached encrypted message served to multiple consumers with different decryption outcomes

Additional context

vault:
  my-kms:
    key: orders-key
    fields:
      - tagged: PII
        unauthorized: redact      # schema-compatible default value
      - tagged: SENSITIVE
        unauthorized: omit        # field absent for unauthorized consumers

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions