Release 0.4.0a2#39
Open
github-actions[bot] wants to merge 19 commits into
Open
Conversation
* ci: replace hand-rolled release jobs with shared template inputs The old publish_stable.yml and release_workflow.yml extracted the version by importing hivemind_sqlite_database, which triggered __init__.py and pulled in ovos_utils — not present in the bare workflow environment. Replace all duplicated publish_pypi, sync_dev, propose_release, and notify jobs with the equivalent inputs on the shared publish-stable.yml@dev and publish-alpha.yml@dev reusable workflows, which use get_version.py to read version.py directly without any package import. AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: fix version extraction failure in CI release workflows - Impact: replaced publish_stable.yml, release_workflow.yml - Verified via: reviewed diff against shared template inputs Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add optional SQLCipher encryption to SQLiteDB via password kwarg AI-Generated Change: - Model: Claude Sonnet 4.6 - Intent: allow operators to encrypt the client database at rest using SQLCipher AES-256 - Impact: SQLiteDB gains a password field; when set, sqlcipher3 is used as the sqlite3 backend and PRAGMA key is issued immediately after connect; when None (default) the existing stdlib sqlite3 path is unchanged; ImportError with install hint raised if sqlcipher3 is missing but a password is supplied - Verified via: python -m pytest tests/test_sqlitedb.py -q -p no:ovoscope (31 passed) * chore: add cipher optional-dependency extra for sqlcipher3 AI-Generated Change: - Model: Claude Sonnet 4.6 - Intent: let users opt into SQLCipher encryption via pip install hivemind-sqlite-database[cipher] - Impact: pyproject.toml gains [project.optional-dependencies] cipher = ["sqlcipher3"] - Verified via: python -m pytest tests/test_sqlitedb.py -q -p no:ovoscope (31 passed) * test: add SQLCipher encrypted-path tests (skipped without sqlcipher3) AI-Generated Change: - Model: Claude Sonnet 4.6 - Intent: verify encrypted path works end-to-end and existing CI remains green without sqlcipher3 - Impact: TestSQLiteDBEncrypted (3 tests, skipped when sqlcipher3 absent) and TestSQLiteDBMissingCipher (ImportError when sqlcipher3 not installed) added - Verified via: python -m pytest tests/test_sqlitedb.py -q -p no:ovoscope (35 passed with sqlcipher3; 32 passed + 3 skipped without) * docs: document SQLCipher encryption in README and add CI workflow AI-Generated Change: - Model: Claude Sonnet 4.6 - Intent: document the password kwarg, install steps, and data-loss warning; add CI that tests both plain and encrypted paths - Impact: README.md rewritten with usage examples, apt/pip install instructions, and data-loss warning; .github/workflows/tests.yml added with two jobs (plain sqlite3 and sqlcipher3) - Verified via: python -m pytest tests/test_sqlitedb.py -q -p no:ovoscope (35 passed) * fix(ci): remove libsqlcipher0 from apt install — not present on Ubuntu 24.04 Ubuntu Noble (24.04) replaced libsqlcipher0 with libsqlcipher1. Installing libsqlcipher-dev is sufficient as it declares a dependency on libsqlcipher1 and pulls it in automatically. AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: fix SQLite + SQLCipher CI job failing with exit code 100 - Impact: `apt-get install -y libsqlcipher0` → removed; libsqlcipher-dev alone satisfies the requirement - Verified via: gh run log inspection showing "Unable to locate package libsqlcipher0" Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address CodeRabbit Major issues in encryption implementation AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: address CodeRabbit Major review comments on PR #24 Changes: - __init__.py: gate on `password is not None` instead of truthiness so empty-string passwords don't silently downgrade to plaintext; raise ValueError for empty strings - __init__.py: escape single-quotes in passphrase before building PRAGMA key string (SQLite double-quote escaping) to avoid broken opens and eliminate an injection surface - tests/test_sqlitedb.py: rewrite _make_encrypted_db() to drive through __post_init__() (patching xdg_data_home) instead of reimplementing the SQLCipher setup, so regressions in the production path are caught - tests/test_sqlitedb.py: remove unused `importlib` import (Ruff F401 / CI lint failure) Verified via: python -m pytest tests/test_sqlitedb.py -q (35 passed) python -m ruff check (all checks passed) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(ci): address CodeRabbit workflow issues AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: address CodeRabbit Critical/Major review comments on CI workflows Changes: - tests.yml: add `sudo apt-get update` before apt install to prevent intermittent "Unable to locate package" failures on stale runner indexes - tests.yml: remove broken `|| pip install -e . && pip install sqlcipher3` fallback that masked a misconfigured cipher extra; now fails loudly if `.[cipher]` is broken (cipher extra is defined in pyproject.toml so this is the correct install path) - release_workflow.yml: add `if: github.event.pull_request.merged == true` guard to publish_alpha job so closing a PR without merging does not trigger PyPI publish, release proposal, or Matrix notification Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* Preserve client metadata
* refactor: require hivemind-plugin-manager>=0.5.0 and drop feature detection
hivemind-plugin-manager 0.5.0 ships Client.metadata, so the
CLIENT_SUPPORTS_METADATA runtime detection and dual code paths are
dead code. Bump the dependency floor and always read/write metadata.
The schema migration (ALTER TABLE ... ADD COLUMN metadata) is kept
for backwards compatibility with DB files created by older versions.
Tests: drop feature-detection skips, add nested-dict and non-ASCII
metadata round-trip coverage.
* test: add metadata helper and update-semantics coverage
- empty-dict default when no metadata kwarg
- INSERT OR REPLACE overwrites metadata on same client_id
- _metadata_to_json returns '{}' for non-dict inputs
- _metadata_from_row coerces NULL / malformed JSON / non-object JSON to {}
* docs: clarify _metadata_to_json/_metadata_from_row contracts
- _metadata_to_json: document that default=str makes datetime/UUID
insertable but they come back as strings (column is opaque JSON, not
a typed map)
- _metadata_from_row: document the swallow-garbage rationale (one bad
row mustn't break iteration)
No behavior change.
---------
Co-authored-by: Gaëtan Trellu <gaetan.trellu@gmail.com>
* ci: pass PYPI_TOKEN explicitly, drop secrets:inherit elsewhere secrets:inherit fails for the publish workflows (publish-stable, publish-alpha) — pass PYPI_TOKEN explicitly. For workflows that don't declare any secrets upstream, drop the secrets: line entirely. * ci: add workflow_dispatch to release_workflow so alphas can be triggered manually Other repos in the family (hivemind-plugin-manager, hivemind-ovos-agent-plugin, HiveMind-core, hivemind-redis-database) all have workflow_dispatch on this workflow. sqlite was missing it, blocking manual alpha-release triggering.
The previous tests.yml had two jobs: - test-plain: redundant with the shared build-tests.yml (which already runs the unencrypted path across Python 3.10-3.14). - test-cipher: unique — installs libsqlcipher-dev + the [cipher] extra and exercises the SQLCipher path on every PR. Keep the cipher job, drop the duplicate, rename the file so its purpose is obvious in the Actions UI. Also align the trigger surface with the rest of the workflows (pull_request on the named branches + workflow_dispatch), instead of the catch-all push/PR on every branch.
…tadata (#32) * feat(db): schema v2 migration — fold legacy blacklist columns into metadata Overrides AbstractDB.migrate() to merge intent_blacklist / skill_blacklist / message_blacklist column values into each row's metadata JSON (setdefault — never clobbers explicit metadata) and NULL the legacy columns. Migration is gated on PRAGMA user_version, idempotent, and crash-safe. Going forward add_item() writes NULL to the legacy columns and _row_to_client() reads exclusively from metadata, with a defensive fallback that re-folds legacy data on the fly if a v1-shape row slips through. Columns themselves stay in the schema for back-compat with older readers; can be removed in a future major. Defensive against older HPM (no SCHEMA_VERSION attr): falls back to v1 / no-op. * refactor(migrate): purge message_blacklist outright on v1->v2 HPM removed message_blacklist from the Client data model entirely (the field was a 2024-12-20 design mistake that contradicted the deny-by-default whitelist model and never functioned as a real gate). Update the v2 migration to match: - migrate(): NULL the legacy column without folding into metadata, and strip any residual metadata key from earlier migration runs. - _row_to_client(): pop metadata["message_blacklist"] defensively. Disk is now clean of message_blacklist after migration runs. * ci: drop HPM branch pin — exercise partial-upgrade path against released HPM The plugin guards on getattr(AbstractDB, 'SCHEMA_VERSION', 1) so it runs cleanly against any HPM version. Running CI against the PyPI HPM exercises the partial-upgrade scenario operators hit when they upgrade this plugin before HPM; if CI breaks here, the back-compat contract has regressed. * docs: drop history/date references in migrate() docstring * fix(migrate): bundle schema migration and user_version bump in one tx Migrate row rewrites and the user_version sentinel write now share a single sqlite transaction so a crash cannot leave the DB in a 'migrated rows but stale sentinel' (or vice versa) state. - factor out _migrate_locked() inner body that assumes the caller already holds the write lock and is in a transaction - _maybe_migrate opens one with self._write_lock, self.conn: block, calls _migrate_locked, then bumps PRAGMA user_version atomically - add empty-new-DB migration test asserting sentinel moves to SCHEMA_VERSION on a fresh open with zero rows * feat(db): targeted refresh, forward-compat schema rejection Override get_client_by_id with a direct primary-key SELECT — used by AbstractDB.refresh() on the per-message admission hot path. Call _check_forward_compat in _maybe_migrate so a DB whose user_version exceeds the backend's SCHEMA_VERSION raises RuntimeError instead of being silently downgraded. * ci: drop secrets: inherit, prune main branch, add release workflow_dispatch * ci: fold dedupe-tests into feat branch — rename tests.yml to cipher-tests.yml Drop the redundant test-plain job (build-tests.yml already covers the unencrypted path on Python 3.10-3.14) and rename to cipher-tests.yml so its purpose is clear. Narrow triggers from catch-all push/PR to pull_request:[dev,master,main] + workflow_dispatch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci(cipher-tests): drop test-plain job, narrow triggers The shared build-tests.yml already runs the unencrypted path on Python 3.10-3.14. This file becomes cipher-only: drop test-plain, narrow on.push/PR triggers to [dev, master, main] + workflow_dispatch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(sqlite): add v2 schema round-trip tests Cover the v2 field set explicitly: allowed_types round-trip, skill_blacklist/intent_blacklist in metadata survive add→search and add→refresh, message_blacklist absent from stored records, v1 row reads cleanly (forward-compat), and refresh returns v2 fields. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: restore version.py to dev baseline * ci: fix pre_install_pip quoting (literal backslashes broke pip) * ci(cipher): install hivemind-plugin-manager==0.6.0a1 (schema-v2: refresh, migrate, _check_forward_compat) * test(e2e): migration → policy → session — real SQLiteDB through a hivescope topology asserts a migrated skill_blacklist is injected into the OVOS session (skips until hivescope db= ships) * test(e2e): importorskip the cross-repo policy stack so the module skips cleanly in the DB plugin's own CI * ci: dedicated Policy Migration E2E job (pins published policy stack + hivescope db=) to activate the migration→policy→session test --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Human review requested!