Add config-based mechanism to disable E2E test scenarios

[Copilot]

❔What, Why & How

Adds infrastructure to disable E2E scenarios via config file rather than modifying workflow YAML. FederatedCredentials scenario is disabled pending Azure resource migration.

Config file (e2eTests/disabled-scenarios.json):

[
    {
        "scenario": "FederatedCredentials",
        "reason": "Azure resource migration work in progress"
    }
]

Workflow changes (.github/workflows/E2E.yaml Analyze job):

• Loads config with error handling (missing file → all scenarios enabled)
• Extracts disabled scenario names: @($config | ForEach-Object { $_.scenario })
• Filters disabled scenarios from matrix: Where-Object { $disabledScenarios -notcontains $_ }
• Logs filtered scenarios with reasons for visibility

Disabled scenarios are completely filtered from the matrix (no jobs created). To re-enable, remove from config array.

✅ Checklist

• Add tests (E2E, unit tests)
• Update RELEASENOTES.md
• Update documentation (e.g. for new settings or scenarios)
• Add telemetry

Original prompt

Refactor the FederatedCredentials end-to-end test scenario in microsoft/AL-Go so it reuses an existing temporary repository instead of deleting and recreating it on every run. The current approach creates a new repository called tmp-bingmaps.appsource copied from microsoft/bcsamples-bingmaps.appsource. Federated credentials no longer work with repository name-based matching, so the repository must be stable across runs.

Context:

• Failing job URL: https://github.com/microsoft/AL-Go/actions/runs/21470716621/job/61842467415
• Failure occurs when the test triggers the "Update AL-Go System Files" workflow in the newly created repository.
• The test script is at e2eTests/scenarios/FederatedCredentials/runtest.ps1 and currently:
  • checks if the temp repo exists
  • deletes it if it exists
  • recreates it from template
  • then runs Update AL-Go System Files

Required changes:

1. Modify e2eTests/scenarios/FederatedCredentials/runtest.ps1 to reuse the repo if it already exists.
   • Do not delete the repo anymore.
   • If the repo exists, reset it to match the source repo (microsoft/bcsamples-bingmaps.appsource) in a deterministic way.
   • Ensure the repo ends up with the same default branch content expected by the scenario (main).
2. Add cleanup logic for workflow runs in the reused temp repo to prevent the run list from growing and to ensure the subsequent WaitWorkflow logic targets the correct run.
   • Before starting the scenario, delete older workflow runs (keep a small number like the last 5-10), or alternatively capture the run id of the workflow triggered and wait specifically for that.
3. Ensure secrets and settings required for the test are still present/updated.
   • The script sets Azure_Credentials; preserve this.
   • The script passes addRepoSettings for ghTokenWorkflowSecretName; ensure this still works if repo already exists.
4. Update any helper functions or supporting scripts if needed to support resetting repo contents and cleaning workflow runs.

Acceptance criteria:

• The test no longer relies on deleting/recreating tmp-bingmaps.appsource.
• The temp repo is stable across runs and compatible with federated credentials.
• The script reliably waits for and inspects the workflow run it triggered, not an older run.
• Add clear comments explaining why the repo is reused.

Relevant files:

• e2eTests/scenarios/FederatedCredentials/runtest.ps1
• e2eTests/Workflows/RunUpdateAlGoSystemFiles.ps1 (workflow trigger helper)
• Any relevant helper module under e2eTests/e2eTestHelper.psm1 if needed.

This pull request was created from Copilot chat.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[mazhelez]

Good work!

@copilot the following script is used in a workflow to clean up temp repositories: "Internal\Scripts\RemoveTempRepos.ps1"
Can you adjust the script or rename the repository, so that it doesn't get deleted?

[Copilot]

Good work!

@copilot the following script is used in a workflow to clean up temp repositories: "Internal\Scripts\RemoveTempRepos.ps1"...

Done. Updated Internal/Scripts/RemoveTempRepos.ps1 to exclude tmp-bingmaps.appsource from automatic deletion. The repository will now remain stable across cleanup operations. Commit: 2309402

[Copilot]

@copilot can you please remove the trailing whitespace in the modified files

Done. Removed trailing whitespace from runtest.ps1. Commit: 13bcc5e

[aholstrup1]

I'm not sure I understand. You're disabling the scenario and also editing it? 🤔

[mazhelez]

Yes. The initial approach was to alter the scenario to use a concrete repository (due to the FIC changes), but that is only half of the solution.

[aholstrup1]

What's the other half? With your current changes, can we not avoid disabling it?

[aholstrup1]

Aren't the previous runs already deleted after you run CleanupWorkflowRuns ?

[mazhelez]

@copilot please address #2104 (comment)

After the call to CleanupWorkflowRuns, there shouldn't be previous runs.

[Copilot]

You're absolutely right - this was a critical bug! The capture was happening AFTER cleanup, making $previousRunIds always empty. I've fixed it by moving the capture BEFORE CleanupWorkflowRuns. Now the baseline is captured while runs still exist, then cleanup happens. Commit: 8b34a7b

[aholstrup1]

10 minutes? Aren't new workflow runs triggered almost immediately?

[Copilot]

Yes, workflow runs typically trigger within seconds, but the 10-minute maximum provides a safe buffer for edge cases (GitHub Actions delays, API lag, high load). The polling happens every 10 seconds, so normal runs are detected quickly (within 10-30 seconds). The timeout just prevents infinite waiting if something goes wrong.

[mazhelez]

@copilot as stated below, the copy needs to be created manually.
Please fix the comment.

[Copilot]

Fixed. Updated comment to clarify that e2e-bingmaps.appsource must be created manually, not by the test. The comment now reads "Source repository to reset e2e-bingmaps.appsource to (must be created manually)". Commit: ad808c6