Skip to content

Conversation

@m7md7sien
Copy link
Contributor

@m7md7sien m7md7sien commented Jan 12, 2026

Description

Please add an informative description that covers that changes made by the pull request and link all relevant issues.

If an SDK is being regenerated based on a new API spec, a link to the pull request containing these API spec changes should be included above.

All SDK Contribution checklist:

  • The pull request does not introduce [breaking changes]
  • CHANGELOG is updated for new features, bug fixes or other significant changes.
  • I have read the contribution guidelines.

General Guidelines and Best Practices

  • Title of the pull request is clear and informative.
  • There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

  • Pull request includes test coverage for the included changes.

Copilot AI review requested due to automatic review settings January 12, 2026 18:36
@m7md7sien m7md7sien requested a review from a team as a code owner January 12, 2026 18:36
@github-actions github-actions bot added the Evaluation Issues related to the client library for Azure AI Evaluation label Jan 12, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements input validation infrastructure for agentic evaluators in the Azure AI Evaluation SDK. The changes introduce a comprehensive validator system to validate inputs before processing in various evaluator classes.

Changes:

  • Added new validator infrastructure with abstract interface and concrete implementations
  • Integrated validators into 13 evaluator classes to validate inputs before processing
  • Created validation constants for message roles and content types

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 14 comments.

Show a summary per file
File Description
_validator_interface.py Abstract base class defining validation interface
_validation_constants.py Enums for message roles and content types
conversation_validator.py Validates conversation-style query/response inputs
_tool_definitions_validator.py Validates tool definitions alongside conversations
_tool_calls_validator.py Validates tool calls alongside tool definitions
_task_navigation_efficiency_validator.py Validates task navigation inputs
__init__.py Exports validator classes
_tool_selection.py Integrated ToolCallsValidator
_tool_output_utilization.py Integrated ToolDefinitionsValidator
_tool_input_accuracy.py Integrated ToolDefinitionsValidator
_tool_call_success.py Integrated ToolDefinitionsValidator
_tool_call_accuracy.py Integrated ToolCallsValidator
_task_navigation_efficiency.py Integrated TaskNavigationEfficiencyValidator
_task_completion.py Integrated ToolDefinitionsValidator
_task_adherence.py Integrated ToolDefinitionsValidator
_intent_resolution.py Integrated ToolDefinitionsValidator
_groundedness.py Integrated ConversationValidator with dual validator setup
_fluency.py Integrated ConversationValidator
_relevance.py Integrated ConversationValidator
_coherence.py Integrated ConversationValidator

@m7md7sien m7md7sien force-pushed the mohessie/evaluators_validators branch from ebdf9d0 to d524521 Compare January 14, 2026 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Evaluation Issues related to the client library for Azure AI Evaluation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants