ENG-3324: System data-stewards change hook for inheritance#8172
ENG-3324: System data-stewards change hook for inheritance#8172adamsachs wants to merge 7 commits into
Conversation
wip: manual type addition wip: refactoring chore: lint chore: remove commented out code feat: adding to website form chore: update changelog
|
The latest updates on your projects. Learn more about Vercel for GitHub. 2 Skipped Deployments
|
0012e92 to
b05cc79
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8172 +/- ##
==========================================
+ Coverage 84.97% 85.62% +0.65%
==========================================
Files 631 660 +29
Lines 41239 42901 +1662
Branches 4787 5022 +235
==========================================
+ Hits 35041 36734 +1693
+ Misses 5113 5063 -50
- Partials 1085 1104 +19 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
39effe9 to
b4c63fc
Compare
02164b0 to
d4e27f4
Compare
37f08c9 to
de998ba
Compare
Temporary single-purpose callback registry under ``fides.api`` that lets the fidesplus monitor-stewardship-inheritance feature react when a user is added/removed as a data steward (system manager) of a system. The three v1 system-manager mutation routes (``bulk_assign_steward``, ``update_managed_systems``, ``remove_user_as_system_manager``, and the approver-demotion branch of ``update_user_permissions``) gain a ``BackgroundTasks`` injection and call ``notify_system_stewards_changed(background_tasks, system.id)`` after each successful set/remove. The model methods themselves are unchanged so callers from tests/conftest don't trigger propagation. This is intentionally a tiny ~50-line shim. The module docstring documents the cutover: when the event framework lands (fides PR #8096), trigger sites swap to ``publish_after_commit(...)`` and this file is deleted. Do not generalize this into a ``hooks`` package. Pairs with fidesplus ``ENG-3324/monitor-steward-inheritance-propagation``, which registers the inheritance-reconcile callback at app startup. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
de998ba to
1620a2e
Compare
Adds a second temporary callback registry (sibling to ``system_steward_change_hooks``) so fidesplus's monitor-stewardship- inheritance propagation can react when the system↔connection-config link that backs a monitor changes. Inheritance walks ``monitor → connection_config → system → data_stewards``, so a link add/remove changes the desired inherited set even when the system's own steward roster is untouched. Same shim posture as the steward hook: single-purpose registry, loud docstring pointing at the event-framework cutover (#8096), explicit "do not generalize this into a ``hooks`` package" note. * ``src/fides/api/system_connection_config_link_change_hooks.py`` — registry with ``register_…`` + ``notify_system_connection_config_link_changed(background_tasks, connection_config_id)``. Failure-isolated dispatch (each hook wrapped in try/except). * ``src/fides/system_integration_link/service.py`` — ``set_links`` and ``delete_link`` accept an optional ``background_tasks``; when provided and a write actually happened, they fire the link-change hooks. * ``src/fides/system_integration_link/routes.py`` — ``PUT`` and ``DELETE`` system-links routes inject ``BackgroundTasks`` and pass it through to the service. * Tests: 5 unit tests for the registry itself (``tests/api/test_system_connection_config_link_change_hooks.py``) plus 4 service-level tests confirming dispatch fires on writes when ``background_tasks`` is supplied and stays silent otherwise / on no-op deletes. FK ``ondelete='CASCADE'`` on both join columns means deleting a ``System`` or ``ConnectionConfig`` outright bypasses Python and won't fire the hook — documented in the module docstring; consumers must rely on a periodic reconciler as the convergence safety net for those.
|
/code-review |
There was a problem hiding this comment.
Code Review — ENG-3324: System↔ConnectionConfig link change hooks
Overall the approach is clean and well-scoped. The two shim modules are ~50 LOC each, clearly temporary, loudly documented (especially the cascade-delete caveat and the "don't generalize this" warning), and come with solid unit tests. The failure-isolation pattern in notify_* is good defensive practice. A few things worth addressing before merge:
Substantive
set_links fires hook even on logical no-ops (see inline)
The docstring says "when a write actually happened," but the guard is just background_tasks is not None. Since the implementation always does delete_all + re-insert, a DB write always occurs when existing links are present or new ones are supplied. However, calling set_links([], background_tasks=bg) when no links existed fires the hook despite zero state change. A secondary case: set_links([same_link]) deletes and recreates the same row, fires the hook, and triggers unnecessary consumer reconciliation. Both are false positives that cause fidesplus to run inheritance reconciliation when nothing changed.
The fix is straightforward: track whether had_existing or results before calling notify_*. At minimum the docstring should accurately describe the actual firing condition.
Missing test for the set_links([])-with-no-prior-links edge case (see inline)
The new TestLinkChangeHookDispatch class doesn't cover this path.
Minor
No changelog fragment
changelog/8172-*.yaml doesn't exist. The checklist item is unchecked — if this is intentional (internal plumbing, no user-visible change), mark it explicitly; otherwise add a fragment under Added or Changed.
_HOOKS thread-safety (see inline)
Fine for startup-time registration, but worth a one-line comment that registration must complete before request handling begins.
update_user_permissions hook call is asymmetric (see inline)
Nit-level: the removal path is hooked but a future promotion path would need explicit instrumentation. A short comment would prevent the oversight.
Positive highlights
- The
autouseisolated_registryfixture pattern in tests is the right approach — avoids cross-test pollution without exposing an unregister API. test_delete_link_does_not_fire_when_link_missingcorrectly asserts the hook stays silent whenSystemIntegrationLinkNotFoundErroris raised (exception path exits before the hook call).background_tasksis optional in the service methods, keeping internal callers (tests, scripts) clean.- The cascade-delete caveat in the docstring is a valuable heads-up for future readers.
🔬 Codegraph: connected (50452 nodes)
💡 Write /code-review in a comment to re-run this review.
…24/monitor-steward-inheritance-propagation
Addresses code-review feedback on PR #8172. * ``set_links([])`` on a connection with no prior links is a true no-op — no row is deleted, none inserted. The hook now stays silent in that case even when ``background_tasks`` is supplied, instead of firing a wasteful consumer reconcile. Added a regression test alongside the existing ``TestLinkChangeHookDispatch`` cases. * Re-setting an identical link still fires the hook (the implementation does delete-and-recreate); documented as tolerable in the docstring alongside the cascade-delete caveat, since consumers are expected to be idempotent. * Added ``changelog/8172-monitor-steward-inheritance-hooks.yaml``.
Dropping the basic add/dispatch tests for both hook registries — they exercise list.append and empty-loop iteration, not anything specific to the registries' contract. The integration tests in ``tests/system_integration_link/test_service.py`` already cover register-then-dispatch end-to-end as a side effect of asserting that ``set_links`` / ``delete_link`` fire the hooks. What remains in each registry's test file: * ``test_register_is_idempotent`` — guards the ``if hook not in _HOOKS`` short-circuit; registering the same hook twice must not duplicate it. * ``test_notify_isolates_hook_failures`` — guards the failure-isolation contract; one raising hook must not stop later hooks from firing. Both behaviors are non-obvious safety properties of the shim modules and worth a dedicated test each. Everything else was effectively ``assert library == library``.
vcruces
left a comment
There was a problem hiding this comment.
Looks good! Clean, well-scoped temporary scaffolding
|
|
||
| DO NOT use this as a precedent for adding more cross-repo hooks. If you need | ||
| a similar shim before the framework lands, create a separate single-purpose | ||
| module — do not generalize this into a ``hooks`` package. |
Ticket ENG-3324
Description Of Changes
Adds two temporary single-purpose callback registries under
fides.apithat let fidesplus react when state backing monitor stewardship inheritance changes, plus the FE toggle that exposes theinherit_system_stewardsflag on the monitor configuration forms. Both registries follow the same shim posture, are loudly documented, and will be deleted when the event framework (#8096) lands.system_steward_change_hooks— fires when a user is added/removed as a data steward (system manager) of a system. Instrumented in the three v1 routes that mutate thesystemmanagerjoin.system_connection_config_link_change_hooks— fires when thesystem_integration_linkrow backing aConnectionConfigis added, replaced, or removed. Instrumented in theSystemIntegrationLinkServicemutators (set_links/delete_link), driven from the correspondingPUT/DELETEroutes under/connection/{key}/system-links.Inherit system stewardstoggle on bothConfigureMonitorForm(databases / detection-discovery monitors) andConfigureWebsiteMonitorForm. Defaultstrueon create, reflects the saved value on edit. Replaces the old "defaultstewardsto the linked system'sdata_stewards" behavior on the website monitor form — that auto-population now happens server-side via the inherit flag.Now that #7888 (the data-model PR) has merged, this PR is based directly on
main. Pairs with the fidesplus propagation PR (ENG-3324/monitor-steward-inheritance-propagation, fidesplus #3564), which registers the inheritance-reconcile callbacks at app startup.The hook modules are loudly documented as TEMPORARY scaffolding for the event framework (#8096), with cutover instructions in their docstrings and a "do not generalize this into a
hookspackage" note to keep us from accreting more shims around them. They also call out the FK-cascade caveat: deleting aSystemorConnectionConfigoutright bypasses Python and won't fire either hook — consumers must rely on a periodic reconciler as the convergence safety net for those.Code Changes
Steward-change registry
src/fides/api/system_steward_change_hooks.py— single-purpose registry exposingregister_system_steward_change_hook(callback)andnotify_system_stewards_changed(background_tasks, system_id). Failure-isolated (each hook wrapped in try/except, logged not raised). Idempotent registration. ~50 LOC.src/fides/api/v1/endpoints/system.py:bulk_assign_steward— injectsBackgroundTasks, callsnotify_system_stewards_changedafter each successfulset_as_system_manager.src/fides/api/v1/endpoints/user_endpoints.py:update_managed_systems— same pattern; fires on both adds and removes (it loops over a diff).src/fides/api/v1/endpoints/user_endpoints.py:remove_user_as_system_manager— fires after the remove.src/fides/api/v1/endpoints/user_permission_endpoints.py:update_user_permissions— when this route demotes a user to APPROVER and consequently removes all their system-manager assignments, fires once per affected system.FidesUser.set/remove_as_system_manager) are unchanged. Test/conftest callers don't trigger propagation by accident.Link-change registry
src/fides/api/system_connection_config_link_change_hooks.py— same shim shape as the steward registry; ~50 LOC.src/fides/system_integration_link/service.py—set_linksanddelete_linkaccept an optionalbackground_tasks; when provided and a write actually happened, they fire the link-change hooks.src/fides/system_integration_link/routes.py—PUTandDELETEsystem-links routes injectBackgroundTasksand pass it through to the service.Monitor configuration FE (from #8015)
clients/admin-ui/src/types/api/models/EditableMonitorConfig.ts— addsinherit_system_stewards?: boolean | nullto the type.clients/admin-ui/src/features/integrations/configure-monitor/ConfigureMonitorForm.tsx— newInherit system stewardsForm.Item (Switch), interface field, initial-value logic (defaultstrueon create), andinherit_system_stewardsthreaded into the upsert payload for both edit and create.clients/admin-ui/src/features/integrations/configure-monitor/ConfigureWebsiteMonitorForm.tsx— mirror of the above for website monitors. Replaces the old "auto-populatestewardsfromsystemData.data_stewardswhen no explicit stewards" behavior, since the server now handles inheritance.Tests
tests/api/test_system_steward_change_hooks.py— 5 unit tests for the steward registry (idempotency, multi-hook dispatch, failure isolation, empty-registry no-op).tests/api/test_system_connection_config_link_change_hooks.py— 5 mirror tests for the link registry.tests/system_integration_link/test_service.py— 4 service-level tests confirming dispatch fires on writes whenbackground_tasksis supplied and stays silent otherwise / on no-op deletes.Steps to Confirm
Best confirmed end-to-end alongside the paired fidesplus propagation PR (#3564), but the BE registries and the FE toggle are independently testable.
Backend registries (isolated)
pytest tests/api/test_system_steward_change_hooks.py tests/api/test_system_connection_config_link_change_hooks.py tests/system_integration_link/test_service.py— 19 tests pass.FE toggle (admin-ui)
clients/admin-ui, start the dev server. Open the integration monitor configuration modal (both database / discovery monitors and website monitors). Confirm the Inherit system stewards toggle is visible, defaults to on when creating a new monitor, and reflects the saved value when editing an existing one.End-to-end (requires the paired fidesplus branch + a seeded property/integration)
SystemandMonitorConfigwithinherit_system_stewards=true(use the new FE toggle).POST /api/v1/system/assign-stewardto add a user → on the fidesplus side, the monitor'sGET /stewardsshould reflect the user as inherited within a tick (BG task fires after response).DELETE /api/v1/user/{user_id}/system-manager/{system_key}→ the inherited row disappears.DELETE /api/v1/connection/{conn_key}/system-links/{system_fides_key}on the monitor's connection → all inherited rows on that monitor disappear (no longer linked to a system).PUT /api/v1/connection/{conn_key}/system-linksto relink → inherited rows return.Pre-Merge Checklist
CHANGELOG.mdupdatedmaindowngrade()migration is correct and works🤖 Generated with Claude Code