Skip to content

feat: add microsoft-foundry skills and tests from fork#1010

Open
tendau wants to merge 17 commits intomicrosoft:mainfrom
tendau:main
Open

feat: add microsoft-foundry skills and tests from fork#1010
tendau wants to merge 17 commits intomicrosoft:mainfrom
tendau:main

Conversation

@tendau
Copy link
Contributor

@tendau tendau commented Feb 23, 2026

Includes:

  • microsoft-foundry agent sub-skill
  • microsoft-foundry troubleshooting sub-skill
  • microsoft-foundry resource/create sub-skill
  • microsoft-foundry quota management skill
  • microsoft-foundry deploy-model with dynamic models and cross-region quota
  • RBAC management skills
  • Foundry integration and unit tests

Copilot AI review requested due to automatic review settings February 23, 2026 20:04
Copilot AI reviewed Feb 23, 2026
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request adds comprehensive Microsoft Foundry (Azure AI Foundry) skills and test infrastructure from a fork. The changes introduce end-to-end capabilities for deploying models, managing agents, troubleshooting issues, handling RBAC, and managing quotas within Azure AI Foundry projects.

Changes:

  • Adds 5 new troubleshooting sub-skills with diagnostic workflows (429 errors, model availability, deployment health)
  • Introduces agent creation skills supporting both Foundry Agent Service and Microsoft Agent Framework SDK
  • Implements model deployment with intelligent routing (preset, customize, capacity discovery)
  • Provides quota management, RBAC configuration, and resource creation capabilities
  • Includes comprehensive test coverage (unit, trigger, integration tests with snapshots)

Reviewed changes

Copilot reviewed 72 out of 74 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/microsoft-foundry/unit.test.ts Updated expectations to align with refactored skill structure and content
tests/microsoft-foundry/troubleshooting/* Complete test suites for 3 troubleshooting skills (429 errors, availability, health)
tests/microsoft-foundry/resource/create/* Resource creation test updates with path fixes
tests/microsoft-foundry/quota/* Quota management integration test improvements
tests/microsoft-foundry/models/deploy/* Model deployment tests for all deployment modes
tests/microsoft-foundry/agent/create/* Agent framework creation tests with trigger/integration coverage
plugin/skills/microsoft-foundry/troubleshooting/references/* Diagnostic references for rate limiting, scaling, model availability, deployment health
plugin/skills/microsoft-foundry/resource/create/* Resource creation workflow documentation
plugin/skills/microsoft-foundry/quota/references/* Quota optimization and capacity planning guides
plugin/skills/microsoft-foundry/project/* Project creation and connection management
plugin/skills/microsoft-foundry/models/deploy-model/* Model deployment skills with Anthropic support
plugin/skills/microsoft-foundry/agent/create/* Agent creation with multiple frameworks and tools

tmeschter
tmeschter reviewed Feb 23, 2026
View reviewed changes
tmeschter
tmeschter reviewed Feb 23, 2026
View reviewed changes
tmeschter
tmeschter reviewed Feb 23, 2026
View reviewed changes
github-advanced-security[bot]
github-advanced-security bot found potential problems Feb 23, 2026
View reviewed changes
tests/microsoft-foundry/integration.test.ts Outdated
// The agentId of the created agent
let targetAgentId: string | undefined = undefined;
for await (const agent of agentsIter) {
console.log("Found agent", agent.name)

Check notice

Code scanning / CodeQL

Semicolon insertion

Avoid automated semicolon insertion (92% of all statements in [the enclosing function](1) have an explicit semicolon).
@JasonYeMSFT
Copy link
Member

JasonYeMSFT commented Feb 23, 2026

Your PR undo quite a few changes from a previous PR. Are those changes intentional? https://github.com/microsoft/GitHub-Copilot-for-Azure/pull/1004/changes#diff-c5d418f41ef49376666ef566805e989bf5a6f40ba0297088790da1acfc6e818a

ankitbko
ankitbko suggested changes Feb 24, 2026
View reviewed changes
tendau added a commit to tendau/GitHub-Copilot-for-Azure that referenced this pull request Feb 24, 2026
@tendau
fix: resolve Markdown References and ESLint CI failures for PR micros…
a5d51b1 
…oft#1010

- Fix 12 broken language/python.md links in SKILL.md to reference
  references/sdk/foundry-sdk-py.md (resolves orphaned file too)
- Convert static @azure/ai-projects import to dynamic import in
  integration.test.ts to fix TS2307 (package not in tests/package.json)
- Fix 7 bare run() calls to agent.run() in quota/integration.test.ts
  to fix TS2304 errors

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
tendau added a commit to tendau/GitHub-Copilot-for-Azure that referenced this pull request Feb 24, 2026
@tendau
fix: resolve Markdown References and ESLint CI failures for PR micros…
c06672f 
…oft#1010

- Fix 12 broken language/python.md links in SKILL.md to reference
  references/sdk/foundry-sdk-py.md (resolves orphaned file too)
- Convert static @azure/ai-projects import to dynamic import in
  integration.test.ts to fix TS2307 (package not in tests/package.json)
- Fix 7 bare run() calls to agent.run() in quota/integration.test.ts
  to fix TS2304 errors

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
tendau added a commit to tendau/GitHub-Copilot-for-Azure that referenced this pull request Feb 24, 2026
@tendau
fix: resolve Markdown References and ESLint CI failures for PR micros…
84af6cb 
…oft#1010

- Fix 12 broken language/python.md links in SKILL.md to reference
  references/sdk/foundry-sdk-py.md (resolves orphaned file too)
- Convert static @azure/ai-projects import to dynamic import in
  integration.test.ts to fix TS2307 (package not in tests/package.json)
- Fix 7 bare run() calls to agent.run() in quota/integration.test.ts
  to fix TS2304 errors

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
tendau added a commit to tendau/GitHub-Copilot-for-Azure that referenced this pull request Feb 24, 2026
@tendau
fix: resolve Markdown References and ESLint CI failures for PR micros…
81e8b89 
…oft#1010

- Fix 12 broken language/python.md links in SKILL.md to reference
  references/sdk/foundry-sdk-py.md (resolves orphaned file too)
- Convert static @azure/ai-projects import to dynamic import in
  integration.test.ts to fix TS2307 (package not in tests/package.json)
- Fix 7 bare run() calls to agent.run() in quota/integration.test.ts
  to fix TS2304 errors

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ankitbko
ankitbko suggested changes Feb 24, 2026
View reviewed changes
Copilot AI review requested due to automatic review settings February 25, 2026 06:47
Copilot AI reviewed Feb 25, 2026
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review is ineligible. To be eligible to request a review, you need a paid Copilot license, or your organization must enable Copilot code review.

ankitbko
ankitbko reviewed Feb 25, 2026
View reviewed changes
ankitbko
ankitbko suggested changes Feb 25, 2026
View reviewed changes
Copilot AI review requested due to automatic review settings February 26, 2026 01:29
Copilot AI reviewed Feb 26, 2026
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review is ineligible. To be eligible to request a review, you need a paid Copilot license, or your organization must enable Copilot code review.

@tendau tendau requested a review from ankitbko February 26, 2026 01:48
Copilot AI review requested due to automatic review settings February 26, 2026 18:50
Copilot AI reviewed Feb 26, 2026
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 68 out of 71 changed files in this pull request and generated 12 comments.

JasonYeMSFT
JasonYeMSFT reviewed Feb 26, 2026
View reviewed changes
JasonYeMSFT
JasonYeMSFT requested changes Feb 26, 2026
View reviewed changes
@JasonYeMSFT
Copy link
Member

JasonYeMSFT commented Feb 26, 2026

@tendau Could you do us a favor by rebasing the main branch of microsoft/GitHub-Copilot-for-Azure to your branch before merging?

Copilot AI review requested due to automatic review settings February 26, 2026 23:21
Copilot AI reviewed Feb 26, 2026
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 80 out of 83 changed files in this pull request and generated no new comments.

Christopher Earley and others added 14 commits February 26, 2026 17:16
@tendau
feat: add microsoft-foundry skills and tests from fork
1a577c9 
Includes:
- microsoft-foundry agent sub-skill
- microsoft-foundry troubleshooting sub-skill
- microsoft-foundry resource/create sub-skill
- microsoft-foundry quota management skill
- microsoft-foundry deploy-model with dynamic models and cross-region quota
- RBAC management skills
- Foundry integration and unit tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@tendau
fix: resolve Markdown References and ESLint CI failures
e2ac2f5 
- Fix 12 broken language/python.md links in SKILL.md pointing to
  references/sdk/foundry-sdk-py.md (also resolves orphaned file)
- Use createRequire for @azure/ai-projects in integration.test.ts
  to avoid TS2307 (package not in tests/package.json)
- Fix 7 bare run() calls to agent.run() in quota/integration.test.ts

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mukeshagmsft @tendau
delete agent framework. Also refactoring web search tool and adding m…
9c57cb9 
…emory tool
@mukeshagmsft @tendau
merging prompt agent with hosted agents
29c3f2d
@XOEEst @tendau
Add: agent-observability-loop skill as foundry-agent/observe (#31)
cd94b52 
* Add: agent-observability-loop skill as foundry-agent/observe

Eval-driven optimization loop for Foundry agents: deploy, evaluate,
cluster failures, optimize prompt, re-deploy, compare versions, iterate.

- observe.md main skill file with 6 reference files
- Minimal hook in deploy.md to reference observe skill after deployment
- Sub-skill entry added to microsoft-foundry SKILL.md
- Test scaffolding (triggers, unit, integration)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: standardize integration test describe names to match eslint pattern

All top-level describe names now use template literal format:
\\_<sub-name> - Integration Tests\

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: add .env persistence and reposition After Deployment in deploy.md

- Add 'Document Deployment Context' section after 'Display Agent Information'
  to persist AZURE_AI_PROJECT_ENDPOINT, AZURE_AI_AGENT_NAME, etc. to .env
- Move 'After Deployment' (observe skill prompt) from end-of-file to right
  after deployment context, in the natural workflow flow
- Ensures observe skill can auto-detect agent via .env values

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve observe skill discoverability with USE FOR triggers and DO NOT MANUALLY warning

- Add 'When to Use This Skill' section with action-oriented trigger phrases
- Add DO NOT MANUALLY warning against calling eval MCP tools without skill
- Enhance parent SKILL.md observe sub-skill description with workflow emphasis
- Remove Loop Overview (duplicated Entry Points table) to reduce token count

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve deploy skill discoverability with USE FOR triggers and DO NOT MANUALLY warning

- Replace bullet-point 'When to Use' with action-oriented USE FOR trigger phrases
- Add DO NOT MANUALLY warning against calling agent_update/az acr build individually
- Enhance parent SKILL.md deploy sub-skill description with pipeline emphasis

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: add azd trigger phrases to deploy skill and parent SKILL.md

- Add 'azd deploy agent', 'azd up agent', 'azd up for agent' to USE FOR
- Add azd/azd deploy to DO NOT MANUALLY warning in deploy.md
- Prevents AI from running manual azd commands instead of using the skill

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve compare-iterate.md by moving critical warning and adding required params table

- Move displayName/state critical warning above the JSON example
- Add required parameters table matching evaluate-step.md pattern
- Prevents BadRequest errors from missing required fields

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* revert: remove eslint fixes to non-observe test files

Upstream has addressed these separately. Only keep our observe test files.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: standardize integration test describe names and quote style for eslint

- Fix top-level describe names to match pattern: \\_<sub> - Integration Tests\
- Fix backtick-to-doublequote for non-template strings (quotes rule)
- Fix single-to-doublequote in unit test (quotes rule)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mukeshagmsft @tendau
Updating the skill.md file to make it more specific to foundry scenar…
27b979c 
…ios. Also removing hardcoded code from the md files
@mukeshagmsft @tendau
fixing some issues from test runs
7eebed5
@albemuth @tendau
noop
689f3d0
@albemuth @tendau
snapshots
45d9c6c
@albemuth @tendau
remove failing triggers
bda62ab
@albemuth @tendau
revert path change
546c351
@albemuth @tendau
Revert troubleshooting skills
5b7405c
@mukeshagmsft @tendau
adding some helpful doc for file search tool
1a84a22
@XOEEst @tendau
Luechen/try trace skills (#37)
78e77e1 
* Trace skills

* improve for evaluation results traces

* improve

* Fix

* fix

* fix

* fix

* Regenerate snapshots after rebase onto main

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@tendau
fix: replace integration.test.ts with upstream version
e6ab04c 
Remove v1 tests (DefaultAzureCredential, AIProjectClient) that upstream
removed. Switch from fs.appendFileSync pattern to softCheckSkill.
Restore agent lifecycle test with shouldEarlyTerminate.

Addresses review comment from JasonYeMSFT on PR microsoft#1010.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings February 27, 2026 01:22
tendau and others added 2 commits February 26, 2026 17:26
@tendau
fix: replace __dirname with SKILLS_PATH in resource/create test
0b4e4a7 
__dirname is not available in ESM modules. Use the SKILLS_PATH global
(set in jest.setup.ts) instead, matching upstream's pattern.

Addresses review comment from JasonYeMSFT on PR microsoft#1010.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@tendau
fix: use softCheckSkill in capacity integration test
81a6fc5 
Remove fs.appendFileSync file-writing pattern and manual successCount
tracking. Use softCheckSkill from evaluate.ts instead.

Addresses review comment from JasonYeMSFT on PR microsoft#1010.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@tendau tendau requested a review from JasonYeMSFT February 27, 2026 01:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tmeschter tmeschter tmeschter left review comments

Copilot code review Copilot Awaiting requested review from Copilot

@JasonYeMSFT JasonYeMSFT Awaiting requested review from JasonYeMSFT

+3 more reviewers

@smartmuki smartmuki smartmuki left review comments

@ankitbko ankitbko ankitbko approved these changes

@erjohnms erjohnms erjohnms approved these changes

Reviewers whose approvals may not affect merge requirements

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

10 participants

@tendau @JasonYeMSFT @smartmuki @ankitbko @tmeschter @erjohnms @mukeshagmsft @XOEEst @albemuth