Skip to content

Conversation

@Pouyanpi
Copy link
Collaborator

Summary

  • Remove the custom _langchain_nvidia_ai_endpoints_patch.py module that patched ChatNVIDIA with streaming decorators
  • Update _init_nvidia_model to use standard ChatNVIDIA from langchain_nvidia_ai_endpoints directly
  • Remove associated tests and mock classes for the patched ChatNVIDIA

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Jan 29, 2026

Greptile Overview

Greptile Summary

This PR removes the custom ChatNVIDIA streaming patch that was previously needed to enable streaming functionality for NVIDIA AI Endpoints and NIM models. The refactor simplifies the codebase by:

  • Removing the custom patch module (_langchain_nvidia_ai_endpoints_patch.py) that wrapped ChatNVIDIA with streaming decorators
  • Updating _init_nvidia_model to import ChatNVIDIA directly from langchain_nvidia_ai_endpoints
  • Removing 379 lines of tests for the patch's custom streaming decorators
  • Cleaning up pyproject.toml to remove the coverage exclusion for the deleted file

The changes align with the base branch refactor/drop-streaming-callback which aims to remove LangChain callback dependencies from the streaming infrastructure. This refactor assumes the upstream langchain_nvidia_ai_endpoints package now natively supports streaming, making the custom patch obsolete.

Key considerations:

  • Verify that streaming still works with native ChatNVIDIA in integration tests
  • Check that streaming parameter can still be passed through kwargs if needed
  • Ensure backward compatibility for existing configurations using nim and nvidia_ai_endpoints providers

Confidence Score: 4/5

  • This PR is safe to merge with minor risk
  • The refactor cleanly removes custom streaming patch code that is no longer needed, assuming the upstream langchain_nvidia_ai_endpoints package now natively supports the streaming functionality that was previously patched in. The changes are well-scoped (removes patch module and its tests) and align with the base branch's goal of dropping LangChain callback dependencies. However, the PR lacks explicit verification that native streaming still works.
  • Verify that streaming functionality still works with the native ChatNVIDIA implementation in integration tests

Important Files Changed

Filename Overview
nemoguardrails/llm/models/langchain_initializer.py removed custom patch import, now uses standard ChatNVIDIA from langchain_nvidia_ai_endpoints, simplified docstring and error handling, added type hint to _PROVIDER_INITIALIZERS
nemoguardrails/llm/providers/_langchain_nvidia_ai_endpoints_patch.py removed 107-line custom patch module that wrapped ChatNVIDIA with streaming decorators

Sequence Diagram

sequenceDiagram
    participant App as Application
    participant Init as langchain_initializer
    participant Patch as _langchain_nvidia_ai_endpoints_patch (REMOVED)
    participant Native as langchain_nvidia_ai_endpoints.ChatNVIDIA
    
    Note over App,Native: BEFORE: Custom Patch Flow
    App->>Init: Initialize NVIDIA model
    Init->>Patch: Import custom ChatNVIDIA
    Patch->>Native: Inherit from ChatNVIDIAOriginal
    Patch->>Patch: Apply stream_decorator
    Patch->>Patch: Apply async_stream_decorator
    Patch-->>Init: Return patched ChatNVIDIA
    Init-->>App: Return model with streaming support
    
    Note over App,Native: AFTER: Native Implementation Flow
    App->>Init: Initialize NVIDIA model
    Init->>Native: Import ChatNVIDIA directly
    Native-->>Init: Return ChatNVIDIA
    Init-->>App: Return model with native streaming
Loading

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

Remove the custom ChatNVIDIA patch that added streaming decorators.
Now using the standard ChatNVIDIA from langchain_nvidia_ai_endpoints
directly since LangChain callback-based streaming has been dropped.
@Pouyanpi Pouyanpi force-pushed the refactor/remove-chatnvidia-patch branch from a97a9be to 48a77d8 Compare January 29, 2026 12:44
@codecov
Copy link

codecov bot commented Jan 29, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Copy link
Member

@trebedea trebedea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants