Skip to content

MCP image tool results crash with double-prefixed MIME type (image/image/png) #3394

Description

@lmwilki

Summary

When an MCP tool returns an ImageContent result (e.g. a screenshot tool returning format="png"), Inspect crashes with:

ValueError: Unable to read image of type image/image/png

The MIME type is double-prefixed: image/image/png instead of image/png.

Root Cause

In src/inspect_ai/tool/_mcp/sampling.py, as_inspect_content() line 121:

elif isinstance(content, ImageContent):
    return ContentImage(
        image=f"data:image/{content.mimeType};base64,{content.data}"
    )

MCP's ImageContent.mimeType is already the full MIME type (e.g. "image/png" per the MCP spec). The code prepends "image/" again, producing data:image/image/png;base64,....

The same bug exists for audio on line 125:

elif isinstance(content, AudioContent):
    return ContentAudio(
        audio=f"data:audio/{content.mimeType};base64,{content.data}",

Suggested Fix

Use content.mimeType directly without the prefix:

elif isinstance(content, ImageContent):
    return ContentImage(
        image=f"data:{content.mimeType};base64,{content.data}"
    )
elif isinstance(content, AudioContent):
    return ContentAudio(
        audio=f"data:{content.mimeType};base64,{content.data}",

Note: the reverse direction (as_mcp_content at line 143) is already correct — it extracts the full MIME type from the data URI via data_uri_mime_type().

Error Trace

anthropic.py:489  in count_tokens
anthropic.py:1618 in message_param
anthropic.py:2437 in message_block_params
anthropic.py:2856 in image_block_param
  → data_uri_mime_type extracts "image/image/png"
  → is_image_type("image/image/png") returns False
  → ValueError raised

Environment

  • inspect-ai==0.3.185
  • MCP server returning Image(data=..., format="png") via mcp.server.fastmcp.utilities.types.Image
  • Model: anthropic/vertex/claude-opus-4-6 (but provider-agnostic — any model hitting count_tokens with an image in message history would crash)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions