Skip to content

fix: cleanup cleanup_workflow_concurrency_slots bug#3318

Open
grutt wants to merge 3 commits intomainfrom
fix--cleanup-orphaned-concurency-multiple-strategies
Open

fix: cleanup cleanup_workflow_concurrency_slots bug#3318
grutt wants to merge 3 commits intomainfrom
fix--cleanup-orphaned-concurency-multiple-strategies

Conversation

@grutt
Copy link
Contributor

@grutt grutt commented Mar 18, 2026

Description

cleanup_workflow_concurrency_slots has a CARDINALITY(child_strategy_ids) <= COUNT(tasks) guard that prevents premature cleanup during DAG execution. But when cancelled tasks are purged from v1_task, the count drops to 0 and CARDINALITY <= 0 is always false -- so the workflow concurrency slot is never deleted.

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Copilot AI review requested due to automatic review settings March 18, 2026 19:13
@vercel
Copy link

vercel bot commented Mar 18, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hatchet-docs Ready Ready Preview, Comment Mar 18, 2026 8:27pm

Request Review

Copilot AI review requested due to automatic review settings March 18, 2026 19:29
@github-actions
Copy link
Contributor

github-actions bot commented Mar 18, 2026

Benchmark results

goos: linux
goarch: amd64
pkg: github.com/hatchet-dev/hatchet/pkg/scheduling/v1
cpu: AMD Ryzen 9 7950X3D 16-Core Processor          
              │ /tmp/old.txt │         /tmp/new.txt          │
              │    sec/op    │    sec/op     vs base         │
RateLimiter-8    50.26µ ± 5%   51.52µ ± 26%  ~ (p=0.818 n=6)

              │ /tmp/old.txt │         /tmp/new.txt          │
              │     B/op     │     B/op      vs base         │
RateLimiter-8   137.7Ki ± 0%   137.7Ki ± 0%  ~ (p=0.455 n=6)

              │ /tmp/old.txt │          /tmp/new.txt          │
              │  allocs/op   │  allocs/op   vs base           │
RateLimiter-8    1.022k ± 0%   1.022k ± 0%  ~ (p=1.000 n=6) ¹
¹ all samples are equal

Compared against main (015b943)

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a PostgreSQL cleanup function so workflow-level concurrency slots can be deleted even when the underlying v1_task rows have been purged, preventing orphaned workflow concurrency slots from accumulating.

Changes:

  • Updates cleanup_workflow_concurrency_slots to allow cleanup when relevant_tasks_for_dags is empty (post-purge case).
  • Adds an integration regression test covering chained strategies and task purge behavior.
  • Adds a goose migration to roll out the function fix to existing databases.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
sql/schema/v1-core.sql Adjusts the cleanup function guard to handle COUNT(*) = 0 correctly.
pkg/scheduling/v1/concurrency_integration_test.go Adds a regression test that simulates task purge and asserts workflow slots are cleaned up.
cmd/hatchet-migrate/migrate/migrations/20260318190742_v1_0_90_fix_orphaned_workflow_concurrency_slots.sql Applies the function fix via migration (with Down restoring prior behavior).

You can also share your feedback on Copilot code review. Take the survey.

Comment on lines +918 to 923
AND (
(SELECT COUNT(*) FROM relevant_tasks_for_dags) = 0
OR CARDINALITY(wcs.child_strategy_ids) <= (
SELECT COUNT(*) FROM relevant_tasks_for_dags
)
)
Comment on lines +279 to +287
sort.Slice(strategies, func(i, j int) bool {
return strategies[i].ID < strategies[j].ID
})

stratCIP := strategies[0]
stratGRR := strategies[1]

require.Equal(t, sqlcv1.V1ConcurrencyStrategyCANCELINPROGRESS, stratCIP.Strategy)
require.Equal(t, sqlcv1.V1ConcurrencyStrategyGROUPROUNDROBIN, stratGRR.Strategy)
Comment on lines +59 to +64
AND (
(SELECT COUNT(*) FROM relevant_tasks_for_dags) = 0
OR CARDINALITY(wcs.child_strategy_ids) <= (
SELECT COUNT(*) FROM relevant_tasks_for_dags
)
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants