Skip to content

New anthropic client to support new generator output design - #410

Merged
liyin2015 merged 3 commits into
mainfrom
change_anthropic_client_to_support_new_generator_output_type
Jul 5, 2025
Merged

New anthropic client to support new generator output design #410
liyin2015 merged 3 commits into
mainfrom
change_anthropic_client_to_support_new_generator_output_type

Conversation

@phi-jkim

@phi-jkim phi-jkim commented Jul 5, 2025

Copy link
Copy Markdown
Contributor

What does this PR do?

These are some of the things this PR does and a link to the design document

Design document: https://www.notion.so/21c294805781808f898ae1800297b043?v=21c294805781801badc1000ca12d6f74&p=22629480578180cbaadde27d0574e135&pm=s

• Updated AnthropicAPIClient to use OpenAI SDK with Anthropic's compatibility
endpoint instead of the native Anthropic API client
• Added Response API compatibility converter (from OpenAI agents SDK) - Created
ChatCompletionToResponseConverter utility to transform ChatCompletion objects into
OpenAI's Response API format for raw_response of the generator output which should be model-agnostic (follow the OpenAI Response API standard)

Design: Anthropic Client Response API Support

The new Anthropic client implementation supports AdalFlow's model-agnostic
raw_response field design through the following architecture:

OpenAI SDK Compatibility Layer

  • Base URL: Uses https://api.anthropic.com/v1/ with OpenAI SDK to call Anthropic's
    compatibility endpoint
  • Authentication: Maintains ANTHROPIC_API_KEY environment variable support
  • API Format: Converts inputs to OpenAI's ChatCompletion format while supporting
    Anthropic-specific features

Response API Converter (ChatCompletionToResponseConverter)

  • Purpose: Transforms OpenAI ChatCompletion objects into Response API format for
    consistency across model providers
  • Streaming Support: Handles both sync and async streaming with proper event
    sequencing
  • Features Supported:
    • Text content streaming with ResponseTextDeltaEvent
    • Refusal handling with ResponseRefusalDeltaEvent
    • Tool calls with ResponseFunctionCallArgumentsDeltaEvent
    • Usage tracking with detailed token counts (including reasoning tokens)

Model-Agnostic Generator Output

  • raw_response Field: Contains the processed Response API compatible output (text or
    stream events)
  • api_response Field: Preserves the original ChatCompletion object for direct API
    access
  • Usage Tracking: Converts ChatCompletion usage to AdalFlow's ResponseUsage format
    with detailed token breakdowns
  • Backward Compatibility: Maintains existing parser interfaces for legacy code
Before submitting
  • Was this discussed/agreed via a GitHub issue? (not for typos and docs)
  • Did you read the contributor guideline?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?


from dotenv import load_dotenv

load_dotenv()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use adalflow setup_env

@liyin2015 liyin2015 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall, it looks fine, will look closer tomorrow

@liyin2015 liyin2015 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. thx!

@liyin2015
liyin2015 merged commit 4d0ce76 into main Jul 5, 2025
4 checks passed
phi-jkim pushed a commit that referenced this pull request Jul 22, 2025
…upport_new_generator_output_type

New anthropic client to support new generator output design
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants