Bug Description
It's not possible to set **kwargs intended for trulens.core.feedback.endpoint.Endpoint via trulens.providers.bedrock.provider.Bedrock constructor, if no client argument is passed.
The **kwargs passed into Bedrock are being forwarded to BedrockEndpoint, and from there to its super class Endpoint, as intended. The issue is that client_kwargs (set inside BedrockEndpoint constructor) are a super set of kwargs:
client_kwargs = {k: v for k, v in kwargs.items()}
Further down below, a check is performed, whether the client argument is contained in kwargs:
if "client" in kwargs:
...
If not, a boto3 client is instantiated using client_kwargs:
self.client = boto3.client(
service_name="bedrock-runtime", **client_kwargs
)
Since boto3.client expects different arguments than Endpoint, it may raise an error depending on which arguments were passed.
I noticed this issue, when I intended to instantiate Bedrock from a configuration that is stored in a file. Since all involved classes extend from pydantic BaseModel, following should be possible once the mentioned issue is fixed:
# config can be stored in external configuration, e.g. in a .yaml file
config = {
"model_id": "anthropic.claude-3-5-sonnet-20240620-v1:0",
"region_name": "eu-central-1",
"pace": {
"marks_per_second": 0.5,
"seconds_per_period": 60,
},
}
bedrock = Bedrock.model_validate(config)
To Reproduce
from trulens.core.utils.pace import Pace
from trulens.providers.bedrock import Bedrock
bedrock = Bedrock(
model_id="anthropic.claude-3-5-sonnet-20240620-v1:0",
region_name="eu-central-1",
pace=Pace(marks_per_second=0.5, seconds_per_period=60),
)
Expected behavior
I expect the code above to work once pace is being forwarded only to Endpoint (and not to boto3.client as is happening now), and recommend to solve it similarly to how it was done in Huggingface pipelines (see here).
Relevant Logs/Tracebacks
Traceback (most recent call last):
File "/home/user/projects/project/test.py", line 4, in <module>
bedrock = Bedrock(
^^^^^^^^
File "/home/user/miniconda3/envs/my-env/lib/python3.12/site-packages/trulens/providers/bedrock/provider.py", line 50, in __init__
self_kwargs["endpoint"] = bedrock_endpoint.BedrockEndpoint(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniconda3/envs/my-env/lib/python3.12/site-packages/trulens/providers/bedrock/endpoint.py", line 197, in __init__
self.client = boto3.client(
^^^^^^^^^^^^^
File "/home/user/miniconda3/envs/my-env/lib/python3.12/site-packages/boto3/__init__.py", line 92, in client
return _get_default_session().client(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Session.client() got an unexpected keyword argument 'pace'
Environment:
- OS: Ubuntu 22.04
- Python Version: 3.12.5
- TruLens version: 1.2.6
Bug Description
It's not possible to set
**kwargsintended fortrulens.core.feedback.endpoint.Endpointviatrulens.providers.bedrock.provider.Bedrockconstructor, if noclientargument is passed.The
**kwargspassed intoBedrockare being forwarded toBedrockEndpoint, and from there to its super classEndpoint, as intended. The issue is thatclient_kwargs(set insideBedrockEndpointconstructor) are a super set ofkwargs:Further down below, a check is performed, whether the
clientargument is contained in kwargs:If not, a boto3 client is instantiated using
client_kwargs:Since
boto3.clientexpects different arguments thanEndpoint, it may raise an error depending on which arguments were passed.I noticed this issue, when I intended to instantiate
Bedrockfrom a configuration that is stored in a file. Since all involved classes extend from pydanticBaseModel, following should be possible once the mentioned issue is fixed:To Reproduce
Expected behavior
I expect the code above to work once
paceis being forwarded only toEndpoint(and not toboto3.clientas is happening now), and recommend to solve it similarly to how it was done in Huggingface pipelines (see here).Relevant Logs/Tracebacks
Environment: