Ops Engine

Event-driven multi-org release orchestration for GitHub and Forgejo.

---

The Problem

When managing multiple organizations and repositories, relying purely on GitHub Actions for organizational hygiene quickly drains runner capacity and exhausts API rate limits. Beyond that, release automation, auto-merge, and cross-forge mirror verification require custom glue code scattered across repos.

The Solution

ops-engine is a pure-infrastructure Python framework that captures incoming webhooks from git forges, places them in a bounded async queue, and processes them through configurable handler modules. Zero business logic — all behavior is driven by YAML config.

Modules

Module	Description	Trigger
Rate Limit Queue	Bounded async queue with backpressure, retry, dead letter, and metrics	Always
Triage & Auto-Labeling	Labels PRs and issues based on title keywords	Webhook
Stale Cleanup	Marks and closes old issues across all orgs	Cron
Dependency Triggers	Fires repository_dispatch on downstream repos when upstream releases	Webhook
Cron Dispatcher	Centralized sequential cron (CodeQL, etc.)	Cron
Release Automation	Creates GitHub/Forgejo releases on tag push with CHANGELOG parsing	Webhook
Auto-Merge	Merges PRs when CI passes and trigger label is present	Webhook
Mirror Verification	Verifies cross-forge mirror sync (Forgejo ↔ GitHub)	Webhook
Notifications	Multi-channel alerts (webhook, Slack, Discord) with event filtering	Event
Event Deduplication	In-memory webhook dedup for GitHub, Forgejo, and Gitea delivery IDs	Always
Health Monitor	Scheduled HTTP probes with pluggable sinks (stdout / file / webhook / GitHub Issue). Replaces the anti-pattern of committing health logs back to the source repo.	Cron
Migration Runner	Forward-only SQL migration tracker with schema_migrations table, drift checks, lock-timeout backoff, and CREATE INDEX CONCURRENTLY handling. Optional [postgres] extra.	Cron + Manual

---

Architecture: The 3-Layer Model

Layer 1: ops-engine (this package — pip install)
  QueueManager, ForgeAdapter, Handler Modules, Config Models

Layer 2: xyz-ops (your org layover — private repo per org)
  FastAPI app, config.yml, webhook endpoints, cron loop

Layer 3: .ops.yaml (per-repo overrides — optional)
  Repo-specific config that overrides org defaults

ops-engine has no dependency on any specific CI system, AI tool, or orchestration framework. The interface is the webhook. A git push --tags produces the same result regardless of what triggered it.

---

Quickstart

pip install git+https://github.com/LangeVC/ops-engine.git

Layover Example

import asyncio
from fastapi import FastAPI, Request
from ops_engine import (
    QueueManager, QueueMetrics, EventDeduplicator,
    TriageHandler, ReleaseHandler, MergeHandler,
    OpsEngineConfig,
)
from ops_engine.adapters.github_adapter import GithubAdapter

app = FastAPI()
queue = QueueManager(rate_limit_delay_seconds=1.0, max_queue_size=1000)
dedup = EventDeduplicator()
adapter = GithubAdapter(token="...", webhook_secret="...")

config = OpsEngineConfig(...)  # loaded from config.yml

@app.post("/webhooks/github")
async def github_webhook(request: Request):
    headers = dict(request.headers)
    payload = await request.body()

    # Dedup
    delivery_id = dedup.extract_delivery_id(headers)
    if dedup.is_duplicate(delivery_id):
        return {"status": "duplicate"}

    # Verify signature
    if not adapter.verify_signature(payload, headers):
        return {"status": "invalid"}, 401

    event = await adapter.parse_webhook(headers, payload)
    repo_config = config.get_repo_config(org, repo)

    # Enqueue handlers
    if repo_config.release and repo_config.release.enabled:
        await queue.enqueue(adapter, event,
            lambda a, e: ReleaseHandler.process_event(a, e, repo_config.release))

    if repo_config.auto_merge and repo_config.auto_merge.enabled:
        await queue.enqueue(adapter, event,
            lambda a, e: MergeHandler.process_event(a, e, repo_config.auto_merge))

    return {"status": "queued"}

---

Config Reference

All modules are configured via YAML in your org layover's config.yml:

Release

release:
  enabled: true
  trigger: "tag_push"         # tag_push | merge_label | both
  tag_pattern: "v*"           # fnmatch pattern
  changelog_path: "CHANGELOG.md"
  draft: false

Auto-Merge

auto_merge:
  enabled: true
  trigger_label: "auto-merge"
  merge_method: "squash"      # squash | merge | rebase
  required_checks: ["test", "lint"]
  delete_branch: true

Mirror Verification

mirror:
  enabled: true
  primary_forge: "forgejo"    # forgejo | github
  mirror_url: "github.com/org/repo"
  verify_on_push: true
  max_drift_seconds: 300

Notifications

notifications:
  enabled: true
  channels:
    - type: "slack"           # webhook | slack | discord
      url: "${SLACK_WEBHOOK_URL}"
      events: ["release", "mirror_drift"]

Migration Runner

Forward-only SQL migration tracker. Install the optional Postgres driver (pip install 'ops-engine[postgres]') on layovers that use this module. One target per service; the layover wires check_pending() to its cron loop and apply_pending() to an admin endpoint.

migrations:
  <service-name>:
    db_url_env: SERVICE_DB_URL
    source:
      type: git              # git | local
      url: https://github.com/Org/service.git
      ref: main
      subpath: migrations
      token_env_var: GITHUB_APP_TOKEN
    table_name: schema_migrations
    lock_timeout: 5s
    max_retries: 5
    check_interval_seconds: 900
    mode: manual_apply       # manual_apply (safe) | auto_apply (ephemeral only)

The runner stores (version, applied_at, checksum, applied_by) in the configured table, detects file drift via sha256 checksums, retries on lock_timeout with backoff [2, 5, 10, 20, 30]s, and automatically pulls CREATE INDEX CONCURRENTLY out of the per-file transaction (Postgres requirement). The siblings lvc-ops and fusionaize-ops layovers can opt in by adding the same migrations: block — only capacium-ops wires it today.

Config Inheritance

Org-level defaults are inherited by all repos. Repo-level config overrides org defaults. Mirror config is repo-specific only (no org inheritance).

MyOrg:
  # Org defaults — inherited by all repos
  auto_triage:
    add_needs_triage_label: true
  stale_management:
    days_until_stale: 60

  repositories:
    my-repo:
      release:
        enabled: true
        trigger: "tag_push"
      # Inherits org-level auto_triage and stale_management

---

Development

git clone https://github.com/LangeVC/ops-engine.git
cd ops-engine
python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
pytest tests/ -v

---

Community & Support

• Contributing: See CONTRIBUTING.md
• Security: See SECURITY.md
• Code of Conduct: See CODE_OF_CONDUCT.md

License

Licensed under the Apache License 2.0. See LICENSE for details.