fix(#10379): fix scalability workflow not triggering for beta tags

[ncaldera]

Description

This PR addresses issue #10379. The scalability workflow was intended to run automatically when beta tags are pushed, but it wasn’t triggering as expected. This was caused by the use of unsupported filters under workflow_run, which prevented GitHub Actions from recognizing beta tag events. This PR updates the workflow to use a supported trigger/filtering mechanism so that scalability tests correctly run for tagged beta builds. I updated .github/workflows/scalability.yml to use a supported filtering mechanism for beta tags that ensures that the scalability workflow runs after Build and test completes when the triggering ref matches the beta tag format.

Code review checklist

• Readable: Concise, well named, follows the style guide, documented if necessary.
• Tested: Unit and/or e2e where appropriate

License

The software is provided under AGPL-3.0. Contributions to this project are accepted under the same license.

[jkuester]

Sorry for the delayed review here over the new year! 😓

One thing that I still a bit confused about is why this workflow is currently not getting automatically triggered at all.... 🤔 As I note below, the workflow_run.tags config seems to not be supported. Perhaps that is just making the workflow_run config be totally ignored. 🤷

One downside to continuing to keep the workflow_run config (without tags) even with the enhanced job-level filter is that it means this workflow will continue to be "run" each time Build and test completes (for all branches, PRs, etc). However, the job should not be run because of the filter. Not sure if there is any downside to triggering an "emtpy" workflow that runs no jobs. It does seem like the only feasible way to only run the workflow for beta tags is to revert the recent changes which added the workflow_run config and go back to a more simple tag filter.

The downside of that is there is no way wait for the "Build and test" workflow to complete before running these scalability tests. @dianabarsan since you were the one who made the last round of changes here, do you have an opinion on which of these you prefer:

1. Use the current workflow_run approach with the enhanced job filter. The "Scalability workflow" will be triggered each time the "Build and test" workflow completes, but the job will only run automatically for beta tags.
2. Roll back to the previous config where the "Scalability workflow" is triggered directly when beta tags are pushed (without waiting for the "Build and test" workflow).

[jkuester]

I think we need to remove this config since it is not supported. It seems like you can filter based on branches, but not tags.

[jkuester]

I think we are missing a couple of things here.

• I think we still want to allow for manually triggering the workflow (via workflow_dispatch).
• We really should only run this job if the "Build and test" workflow actually completed successfully. Otherwise there is not much point in trying to trigger this after the "Build and test" workflow finishes.
• This would still trigger the workflow run for any branch with a name that contains "beta" (and not just tags).

According to Copilot, something like this might work:

    if: github.event_name == 'workflow_dispatch' ||
      (
        github.event.workflow_run.conclusion == 'success' &&
        startsWith(github.event.workflow_run.head_branch, 'refs/tags/') &&
        contains(github.event.workflow_run.head_branch, 'beta')
      )

[dianabarsan]

Thanks for the tag @jkuester .
My understanding is that if we use the if clause in the job, we will have scalability workflow jobs start for every branch but none of the jobs will run? So our "scalability" rab will be full of cancelled jobs. This is not ideal but not a disaster. If there's a way around it, we should take it, but if there is no solution, I suppose we can live with that.

On the other issue of workflow_dispath, we definitely definitely want to be able to manually trigger the scalability suite for any branch.

[jkuester]

So our "scalability" rab will be full of cancelled jobs. This is not ideal but not a disaster. If there's a way around it, we should take it, but if there is no solution, I suppose we can live with that.

@dianabarsan the only alternative I found is #2 where we roll back to the previous config where the "Scalability workflow" is triggered directly when beta tags are pushed (without waiting for the "Build and test" workflow). That would mean the workflow gets auto-triggered only for beta tags (and we avoid a bunch of cancelled/noop jobs). But it would also mean that we would trigger the build for beta tags _without waiting to see if the "Build and test" workflow passes or not. IMHO this is the best way to go, but wanted your feedback since it would basically be undoing your previous changes.

[dianabarsan]

That's unfortunate. #2 meant we needed to add a continuous check for the images to get published, and that made this job start and wait for 6 hours if a build failed. I really didn't like that.
It feels like it's better to just manually trigger it (what are the odds of getting a green build from one go lol).

[jkuester]

👍 I am good with just a manual trigger.

@ncaldera can you update the workflow to just remove the new if as well as the existing on.tags config? (Just leaving on.workflow_dispatch so the job can be manually triggered. Then, can you update this line in the release issue template to instead say something like:

• Once the CI completes successfully, use the "Run Workflow" drop-down to run the Scalability Workflow. Once it passes, check for the scalability results here. More info in the scalability documentation.

[github-actions]

This PR is now marked "stale" after 30 days without activity. It will be closed automatically in 10 days unless you add a comment, push new changes or remove the "stale" label.