Skip to content

07-cicd-publishing.md

Latest commit

 

History

History
380 lines (287 loc) · 13 KB

07-cicd-publishing.md

File metadata and controls

380 lines (287 loc) · 13 KB
Raw

07 — CI/CD Publishing Pipeline

This document describes the release path for every EdgeParse distribution asset: Rust crates, Python wheels, Node.js packages, the WASM SDK, CLI archives, Homebrew, and Docker images.

Overview

Publishing is driven by six GitHub Actions workflows, all triggered by pushing a semantic version tag:

git tag v0.2.2
git push origin v0.2.2
vX.Y.Z tag
  ├─ release-rust.yml    -> crates.io         (pdf-cos, edgeparse-core, edgeparse-cli)
  ├─ release-python.yml  -> PyPI              (edgeparse wheels + sdist)
  ├─ release-node.yml    -> npm               (edgeparse + 5 platform packages)
  ├─ release-wasm.yml    -> npm + GitHub Packages + GitHub Releases  (edgeparse-wasm)
  ├─ release-cli.yml     -> GitHub Releases   (5 CLI archives) + Homebrew tap
  └─ release-docker.yml  -> GHCR + Docker Hub (linux/amd64, linux/arm64)

Shared verification happens in ci.yml on pushes and pull requests:

  • Rust build, test, clippy, and fmt checks
  • Python wheel build and SDK tests
  • Node.js addon build and SDK tests
  • WASM target compilation check
  • Docker image smoke build
  • Cargo audit and cargo-deny

Published Artifacts

Channel Artifact Registry / Location
crates.io pdf-cos https://crates.io/crates/pdf-cos
crates.io edgeparse-core https://crates.io/crates/edgeparse-core
crates.io edgeparse-cli https://crates.io/crates/edgeparse-cli
PyPI edgeparse https://pypi.org/project/edgeparse/
npm edgeparse https://www.npmjs.com/package/edgeparse
npm edgeparse-darwin-arm64 https://www.npmjs.com/package/edgeparse-darwin-arm64
npm edgeparse-darwin-x64 https://www.npmjs.com/package/edgeparse-darwin-x64
npm edgeparse-linux-arm64-gnu https://www.npmjs.com/package/edgeparse-linux-arm64-gnu
npm edgeparse-linux-x64-gnu https://www.npmjs.com/package/edgeparse-linux-x64-gnu
npm edgeparse-win32-x64-msvc https://www.npmjs.com/package/edgeparse-win32-x64-msvc
npm edgeparse-wasm https://www.npmjs.com/package/edgeparse-wasm
GitHub Packages @raphaelmansuy/edgeparse-wasm https://github.com/raphaelmansuy/edgeparse/pkgs/npm/edgeparse-wasm
GitHub Releases CLI archives + WASM npm tarball https://github.com/raphaelmansuy/edgeparse/releases
Homebrew raphaelmansuy/edgeparse tap https://github.com/raphaelmansuy/homebrew-edgeparse
GHCR ghcr.io/raphaelmansuy/edgeparse https://github.com/raphaelmansuy/edgeparse/pkgs/container/edgeparse
Docker Hub rmansuy/edgeparse https://hub.docker.com/r/rmansuy/edgeparse

CLI Release Targets

Each GitHub Release includes:

Archive Platform
edgeparse-X.Y.Z-aarch64-apple-darwin.tar.gz macOS Apple Silicon
edgeparse-X.Y.Z-x86_64-apple-darwin.tar.gz macOS Intel
edgeparse-X.Y.Z-x86_64-unknown-linux-gnu.tar.gz Linux x86_64 (glibc >= 2.17)
edgeparse-X.Y.Z-aarch64-unknown-linux-gnu.tar.gz Linux ARM64 (glibc >= 2.17)
edgeparse-X.Y.Z-x86_64-pc-windows-gnu.zip Windows x86_64

Python Wheel Coverage

Platform Python versions
Linux x86_64 cp310, cp311, cp312, cp313
Linux ARM64 cp310, cp311, cp312, cp313
macOS Intel cp310, cp311, cp312, cp313
macOS Apple Silicon cp310, cp311, cp312, cp313
Windows x86_64 cp310, cp311, cp312, cp313
Source distribution sdist

Secrets and Environments

Repository secrets

Secret Used by Purpose
CARGO_REGISTRY_TOKEN release-rust.yml Publish crates to crates.io
NPM_TOKEN release-node.yml, release-wasm.yml Fallback token — not needed when Trusted Publisher (OIDC) is configured for the package
DOCKERHUB_TOKEN release-docker.yml Push Docker images to Docker Hub
HOMEBREW_TAP_TOKEN release-cli.yml Push edgeparse.rb to the Homebrew tap

GitHub environments

Environment Used by Notes
npm release-node.yml, release-wasm.yml Optional protection rules for npm release jobs
pypi release-python.yml Required for PyPI Trusted Publishing

External setup

  • crates.io: create a token with publish-new and publish-update

  • npm (Trusted Publisher — recommended): configure via OIDC so CI publishes without any token:

    1. Go to the package settings page on npmjs.com
    2. Under Trusted Publisher, set Publisher = GitHub Actions, org/user = raphaelmansuy, repository = edgeparse, workflow filename = the publishing workflow (see table below), environment = (leave blank)
    3. Add id-token: write permission to the publishing job (already done in both release-node.yml and release-wasm.yml)
    4. Use npm publish --provenance --access public — no NODE_AUTH_TOKEN env var needed
    npm package Workflow filename
    edgeparse release-node.yml
    edgeparse-wasm release-wasm.yml

    For edgeparse-wasm: the package must exist on npm before the Trusted Publisher entry can be saved. Publish the first version manually with --otp, then configure the Trusted Publisher entry — all subsequent releases use OIDC.

  • npm (Classic token — fallback): if Trusted Publisher is not yet configured, create a Classic Automation token at https://www.npmjs.com/settings/raphaelmansuy/tokens and store it as NPM_TOKEN. Granular Access Tokens cannot create new packages and will fail with E404.

  • PyPI: configure Trusted Publishing for release-python.yml in environment pypi

  • Docker Hub: create a read/write access token for account rmansuy

  • Homebrew tap: create a PAT with contents: write on raphaelmansuy/homebrew-edgeparse

Release Checklist

  1. Ensure the working tree is clean.
  2. Update versioned manifests:
    • root Cargo.toml
    • crates/edgeparse-cli/Cargo.toml
    • sdks/node/package.json
    • sdks/node/package-lock.json
    • sdks/node/npm/*/package.json
    • crates/edgeparse-wasm/pkg/package.json
  3. Update release notes:
    • CHANGELOG.md
    • README.md
    • this document when the release surface changes
  4. Run local release-prep verification.
  5. Push the release branch and open a PR.
  6. Merge the PR.
  7. Tag the merge commit and push the tag.
  8. Watch all six release workflows complete.

Local Verification

Run the checks that correspond to shipped assets before tagging:

cargo test
cargo check -p edgeparse-wasm --target wasm32-unknown-unknown
docker build -f docker/Dockerfile .

cd sdks/node
npm ci
cargo build --manifest-path ../../crates/edgeparse-node/Cargo.toml --release
# Copy the host-specific addon into the matching local package before testing.
# Example shown here for Apple Silicon:
cp ../../target/release/libedgeparse_node.dylib npm/darwin-arm64/edgeparse-node.darwin-arm64.node
npm install --no-save file:./npm/darwin-arm64
npm run build:ts
npm test
cd ../..

cd benchmark
uv run python run.py --check-regression
cd ..

Optional dry runs:

make publish-rust-dry
make publish-python-dry
make publish-node-dry
make publish-wasm-dry
make publish-cli-dry
make publish-brew-dry

Tag Release Flow

# 1. Commit and push the release-prep branch
git add -A
git commit -m "chore: prepare 0.2.4 release"
git push origin <branch>

# 2. Open and squash-merge the PR
gh pr create --base main --head <branch> --title "chore: prepare 0.2.4 release"
gh pr merge <pr-number> --squash --delete-branch

# 3. Tag the merge commit on main
git checkout main
git pull --ff-only origin main
git tag v0.2.4
git push origin v0.2.4

The tag must match v[0-9]+.[0-9]+.[0-9]+. The Rust and WASM release workflows verify that the tag version matches the workspace version and fail fast on mismatches.

Workflow Reference

ci.yml

Job Coverage
rust cargo build, cargo test, cargo clippy, cargo fmt --check
python Build wheel with maturin and run SDK tests
node Build native addon, compile TypeScript, run SDK tests
wasm cargo check -p edgeparse-wasm --target wasm32-unknown-unknown
docker docker build -f docker/Dockerfile .
security cargo audit and cargo deny check

release-rust.yml

  • Verifies tag/version consistency
  • Publishes pdf-cos, then edgeparse-core, then edgeparse-cli
  • Waits for crates.io index propagation between dependent crates
  • Creates or updates the GitHub Release notes

release-python.yml

  • Builds wheel artifacts for Linux, macOS, and Windows
  • Builds an sdist
  • Publishes to PyPI via OIDC Trusted Publishing

release-node.yml

  • Builds native .node binaries for five targets
  • Syncs the package version from the tag
  • Publishes five platform packages and the main edgeparse package using npm Trusted Publisher (OIDC) — no NPM_TOKEN needed
  • Adds provenance attestation (--provenance) to every published package
  • Treats "already published" as idempotent rather than fatal

release-wasm.yml

  • Builds the browser-targeted WASM package with wasm-pack
  • Syncs the npm package metadata (version, exports, files, provenance) from the tag
  • Publishes edgeparse-wasm to npm using npm Trusted Publisher (OIDC) — no NPM_TOKEN needed
  • Adds provenance attestation (--provenance) so the package appears as verified on npmjs.com
  • Publishes @raphaelmansuy/edgeparse-wasm to GitHub Packages (secondary) using the built-in GITHUB_TOKEN — no extra secret required
  • Uploads the tarball to the GitHub Release (--clobber for idempotent re-runs)
  • Both publish steps treat "already published" as non-fatal

Required permissions (already in workflow)

Permission Purpose
id-token: write Mint OIDC token for npm Trusted Publisher
packages: write Push to GitHub Packages
contents: write Create / update GitHub Releases

Configuring npm Trusted Publisher for edgeparse-wasm

One-time setup — after the first manual publish creates the package on npm:

  1. Go to https://www.npmjs.com/package/edgeparse-wasm → Settings → Trusted Publisher
  2. Publisher: GitHub Actions
  3. Organization or user: raphaelmansuy
  4. Repository: edgeparse
  5. Workflow filename: release-wasm.yml
  6. Environment name: (leave blank)
  7. Click Save changes

All future tag releases will publish via OIDC with no token required.

After updating, re-trigger without retagging:

gh workflow run release-wasm.yml --repo raphaelmansuy/edgeparse \
  --field tag_name=v0.2.4

release-cli.yml

  • Builds five CLI archives
  • Uploads them to the GitHub Release
  • Regenerates and pushes the Homebrew formula

release-docker.yml

  • Builds and pushes a multi-arch container image
  • Publishes to GHCR and Docker Hub
  • Generates provenance and SBOM metadata
  • Runs a Trivy vulnerability scan

Local Publish Helpers

The Makefile mirrors the registry release flow for manual publishing:

make publish-rust
make publish-python
make publish-node
make publish-wasm
make publish-cli
make publish-brew
make publish-all

make publish-all covers crates, Python, Node.js, the WASM SDK, CLI archives, and Homebrew. Docker publishing remains CI-driven through release-docker.yml.

Troubleshooting

crates.io rejects a publish because the version already exists

Crates.io versions are immutable. Bump the version and retag.

npm publish fails on platform packages

Use a Classic Automation token for NPM_TOKEN. Granular tokens often miss one or more package names and produce E403 Forbidden.

edgeparse-wasm npm publish fails with E404 Not Found

This happens when either:

  1. Wrong token type — A Granular Access Token was used instead of a Classic Automation token. Granular tokens cannot create brand-new package names on npm. Solution: replace NPM_TOKEN with a Classic Automation token (see Configuring NPM_TOKEN above).

  2. Package name not yet claimed — edgeparse-wasm has never been published before, so npm has no record of the package. This is normal on first release; a Classic Automation token will create it automatically.

After fixing the secret, re-trigger the workflow without retagging:

gh workflow run release-wasm.yml --repo raphaelmansuy/edgeparse \
  --field tag_name=v0.2.4

The workflow treats "already published" as idempotent, so re-running is safe.

PyPI publish fails with invalid-publisher

The PyPI Trusted Publisher entry must match:

  • project: edgeparse
  • owner: raphaelmansuy
  • repository: edgeparse
  • workflow: release-python.yml
  • environment: pypi

The GitHub Release exists but some assets are missing

Re-run the specific workflow. release-cli.yml and release-wasm.yml upload assets with --clobber, so the release can be repaired without retagging.

Local Linux cross-builds fail on macOS

Use cargo-zigbuild plus zig. The release workflows already do this for the Linux ARM64 and Windows targets.