Skip to content

BUG: LiteLLMModel crashes when multiple consecutive system messages are provided #1972

Description

@graylin-byte

Problem
LiteLLMModel fails with an AssertionError when the input message list contains multiple consecutive system messages. This makes it impossible to express layered or compositional system instructions, which is a common usage pattern across chat-based LLM APIs.

Steps to reproduce

from smolagents import LiteLLMModel
import os

model = LiteLLMModel(
    model_id="gemini/gemini-2.5-flash",
    api_key=os.getenv("GOOGLE_API_KEY"),
)

messages = [
    {"role": "system", "content": "When you say anything Start with 'FOO'"},
    {"role": "system", "content": "When you say anything End with 'BAR'"},
    {"role": "user", "content": "Just say '.'"},
]

response = model(messages)
print(response.content)

Actual behavior and error logs
A clear and concise description of what actually happened. Please include the full traceback if an exception was raised.

Traceback
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
Cell In[2], line 15
      4 model = LiteLLMModel(  
      5     model_id="gemini/gemini-2.5-flash", 
      6     api_key=os.getenv("GOOGLE_API_KEY"),  
      7 )  
      9 messages = [  
     10     {"role": "system", "content": "When you say anything Start with 'FOO'"},  
     11     {"role": "system", "content": "When you say anything End with 'BAR'"},  
     12     {"role": "user", "content": "Just say '.'"}  
     13 ]  
---> 15 response = model(messages)  
     16 print(response.content)

File /opt/homebrew/Caskroom/miniconda/base/envs/smolagents/lib/python3.12/site-packages/smolagents/models.py:581, in Model.__call__(self, *args, **kwargs)
    580 def __call__(self, *args, **kwargs):
--> 581     return self.generate(*args, **kwargs)

File /opt/homebrew/Caskroom/miniconda/base/envs/smolagents/lib/python3.12/site-packages/smolagents/models.py:1274, in LiteLLMModel.generate(self, messages, stop_sequences, response_format, tools_to_call_from, **kwargs)
   1266 def generate(
   1267     self,
   1268     messages: list[ChatMessage | dict],
   (...)   1272     **kwargs,
   1273 ) -> ChatMessage:
-> 1274     completion_kwargs = self._prepare_completion_kwargs(
   1275         messages=messages,
   1276         stop_sequences=stop_sequences,
   1277         response_format=response_format,
   1278         tools_to_call_from=tools_to_call_from,
   1279         model=self.model_id,
   1280         api_base=self.api_base,
   1281         api_key=self.api_key,
   1282         convert_images_to_image_urls=True,
   1283         custom_role_conversions=self.custom_role_conversions,
   1284         **kwargs,
   1285     )
   1286     self._apply_rate_limit()
   1287     response = self.retryer(self.client.completion, **completion_kwargs)

File /opt/homebrew/Caskroom/miniconda/base/envs/smolagents/lib/python3.12/site-packages/smolagents/models.py:523, in Model._prepare_completion_kwargs(self, messages, stop_sequences, response_format, tools_to_call_from, custom_role_conversions, convert_images_to_image_urls, tool_choice, **kwargs)
    521 # Clean and standardize the message list
    522 flatten_messages_as_text = kwargs.pop("flatten_messages_as_text", self.flatten_messages_as_text)
--> 523 messages_as_dicts = get_clean_message_list(
    524     messages,
    525     role_conversions=custom_role_conversions or tool_role_conversions,
    526     convert_images_to_image_urls=convert_images_to_image_urls,
    527     flatten_messages_as_text=flatten_messages_as_text,
    528 )
    529 # Start with messages
    530 completion_kwargs = {
    531     "messages": messages_as_dicts,
    532 }

File /opt/homebrew/Caskroom/miniconda/base/envs/smolagents/lib/python3.12/site-packages/smolagents/models.py:376, in get_clean_message_list(message_list, role_conversions, convert_images_to_image_urls, flatten_messages_as_text)
    373                 element["image"] = encode_image_base64(element["image"])
    375 if len(output_message_list) > 0 and message.role == output_message_list[-1]["role"]:
--> 376     assert isinstance(message.content, list), "Error: wrong content:" + str(message.content)
    377     if flatten_messages_as_text:
    378         output_message_list[-1]["content"] += "\n" + message.content[0]["text"]

AssertionError: Error: wrong content:When you say anything End with 'BAR'

Expected behavior
The framework should transform messages correctly, without any error.

Environment:

  • OS: macOS
  • Python version: 3.12
  • Package version: 1.24.0

Checklist

  • [x ] I have searched the existing issues and have not found a similar bug report.
  • I have provided a minimal, reproducible example.
  • I have provided the full traceback of the error.
  • I have provided my environment details.
  • I am willing to work on this issue and submit a pull request. (optional)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions