feat(DATAGO-126654): Implement dynamic model provider for services and tools[3/4]#1180
Open
feat(DATAGO-126654): Implement dynamic model provider for services and tools[3/4]#1180
Conversation
…and internal config listener flow
…nd update validation logic
…ic and model provider ID
cyrus2281
changed the title
✅ FOSSA Guard: Licensing (
|
✅ FOSSA Guard: Vulnerability (
|
…mponent type in bootstrap request
cyrus2281
changed the title
cyrus2281
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
Implements part of the dynamic model provider rollout by refactoring gateway services/tools to obtain an LLM instance from the host component (enabling runtime model hot-swapping) rather than constructing LiteLLM clients directly from static config.
Changes:
- Refactors
TitleGenerationServiceandPromptBuilderAssistantto useBaseLlm.generate_content_async(...)and accept an injected LLM instance. - Updates FastAPI wiring and agent tools to pull the model via
component.get_lite_llm_model(). - Adjusts unit tests to align with the new injection pattern (and removes a now-stale dependency test module).
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
src/solace_agent_mesh/gateway/http_sse/services/title_generation_service.py |
Switches title generation from litellm.acompletion to ADK BaseLlm/LiteLlm and supports a title-specific override model. |
src/solace_agent_mesh/gateway/http_sse/services/prompt_builder_assistant.py |
Converts assistant to accept an injected BaseLlm and uses ADK request/response flow. |
src/solace_agent_mesh/gateway/http_sse/routers/prompts.py |
Wires prompt builder endpoints to use component.get_lite_llm_model(). |
src/solace_agent_mesh/gateway/http_sse/dependencies.py |
Updates title generation dependency to inject the component LLM. |
src/solace_agent_mesh/agent/tools/deep_research_tools.py |
Changes default model selection to always use host component’s get_lite_llm_model(). |
src/solace_agent_mesh/agent/tools/builtin_artifact_tools.py |
Uses host component model as fallback default for extraction tool model selection. |
tests/unit/gateway/http_sse/services/test_title_generation_service.py |
Refactors tests to mock generate_content_async(...) responses. |
tests/unit/agent/tools/test_deep_research_tools.py |
Updates expectations to align with host-component model retrieval. |
tests/unit/gateway/http_sse/test_dependencies_title_service.py |
Removes now-outdated dependency tests for title generation service wiring. |
src/solace_agent_mesh/agent/adk/setup.py |
Removes unused LiteLlm import. |
examples/agents/a2a_mcp_example.yaml |
Clarifies comment about inheriting parent component’s model if not provided. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
src/solace_agent_mesh/gateway/http_sse/services/title_generation_service.py
Outdated
Show resolved
Hide resolved
Comment on lines
+33
to
+40
| def _create_service_with_mock_llm(llm_return=None): | ||
| """Create a TitleGenerationService with a mocked LiteLlm.""" | ||
| mock_llm = MagicMock() | ||
| if llm_return is not None: | ||
| mock_llm.generate_content_async = MagicMock( | ||
| return_value=_async_gen(llm_return) | ||
| ) | ||
| service = TitleGenerationService(mock_llm) |
Comment on lines
+68
to
+72
| mock_resp = _mock_llm_response("Test Title") | ||
| service = _create_service_with_mock_llm( | ||
| model_config={"model": "gpt-4", "api_key": "test-key"}, | ||
| llm_return=mock_resp, | ||
| ) |
Comment on lines
+223
to
225
| default_model = host_component.get_lite_llm_model() | ||
|
|
||
|
|
| source_mime_type = source_artifact_data.get("mime_type", "application/octet-stream") | ||
| actual_source_version = source_artifact_data.get("version", "unknown") | ||
|
|
||
| host_component = getattr(inv_context.agent, "host_component", None) |
Comment on lines
761
to
+772
| def get_title_generation_service( | ||
| component: "WebUIBackendComponent" = Depends(get_sac_component), | ||
| ) -> "TitleGenerationService": | ||
| ) -> TitleGenerationService: | ||
| """FastAPI dependency to get an instance of TitleGenerationService.""" | ||
| from .services.title_generation_service import TitleGenerationService | ||
|
|
||
|
|
||
| log.debug("get_title_generation_service called") | ||
| # Get model configuration from component (same pattern as prompt_builder_assistant) | ||
|
|
||
| # Get LiteLlm instance from the component | ||
| model_config = component.get_config("model", {}) | ||
|
|
||
| return TitleGenerationService(model_config=model_config) | ||
| llm = component.get_lite_llm_model() | ||
|
|
||
| return TitleGenerationService(model_config=model_config, llm=llm) |
Comment on lines
+770
to
+772
| llm = component.get_lite_llm_model() | ||
|
|
||
| return TitleGenerationService(model_config=model_config, llm=llm) |
mo-radwan1
reviewed
Mar 17, 2026
Collaborator
mo-radwan1
left a comment
mo-radwan1
left a comment
There was a problem hiding this comment.
Left a few comments - test bugs and a null safety issue in deep_research_tools.
tests/unit/gateway/http_sse/services/test_title_generation_service.py
Outdated
Show resolved
Hide resolved
…l config listening
…n SamComponentBase
…ationService dependencies
…m instance after configuration attempts
…mark_initialized method
src/solace_agent_mesh/gateway/http_sse/services/prompt_builder_assistant.py
Show resolved
Hide resolved
src/solace_agent_mesh/gateway/http_sse/services/title_generation_service.py
Outdated
Show resolved
Hide resolved
Co-authored-by: Copilot Autofix powered by AI <[email protected]> Signed-off-by: Cyrus Mobini <[email protected]>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is the purpose of this change?
How was this change implemented?
Key Design Decisions
How was this change tested?
Is there anything the reviewers should focus on/be aware of?