Summary
When an MCP tool returns an ImageContent result (e.g. a screenshot tool returning format="png"), Inspect crashes with:
ValueError: Unable to read image of type image/image/png
The MIME type is double-prefixed: image/image/png instead of image/png.
Root Cause
In src/inspect_ai/tool/_mcp/sampling.py, as_inspect_content() line 121:
elif isinstance(content, ImageContent):
return ContentImage(
image=f"data:image/{content.mimeType};base64,{content.data}"
)
MCP's ImageContent.mimeType is already the full MIME type (e.g. "image/png" per the MCP spec). The code prepends "image/" again, producing data:image/image/png;base64,....
The same bug exists for audio on line 125:
elif isinstance(content, AudioContent):
return ContentAudio(
audio=f"data:audio/{content.mimeType};base64,{content.data}",
Suggested Fix
Use content.mimeType directly without the prefix:
elif isinstance(content, ImageContent):
return ContentImage(
image=f"data:{content.mimeType};base64,{content.data}"
)
elif isinstance(content, AudioContent):
return ContentAudio(
audio=f"data:{content.mimeType};base64,{content.data}",
Note: the reverse direction (as_mcp_content at line 143) is already correct — it extracts the full MIME type from the data URI via data_uri_mime_type().
Error Trace
anthropic.py:489 in count_tokens
anthropic.py:1618 in message_param
anthropic.py:2437 in message_block_params
anthropic.py:2856 in image_block_param
→ data_uri_mime_type extracts "image/image/png"
→ is_image_type("image/image/png") returns False
→ ValueError raised
Environment
inspect-ai==0.3.185
- MCP server returning
Image(data=..., format="png") via mcp.server.fastmcp.utilities.types.Image
- Model:
anthropic/vertex/claude-opus-4-6 (but provider-agnostic — any model hitting count_tokens with an image in message history would crash)
Summary
When an MCP tool returns an
ImageContentresult (e.g. a screenshot tool returningformat="png"), Inspect crashes with:The MIME type is double-prefixed:
image/image/pnginstead ofimage/png.Root Cause
In
src/inspect_ai/tool/_mcp/sampling.py,as_inspect_content()line 121:MCP's
ImageContent.mimeTypeis already the full MIME type (e.g."image/png"per the MCP spec). The code prepends"image/"again, producingdata:image/image/png;base64,....The same bug exists for audio on line 125:
Suggested Fix
Use
content.mimeTypedirectly without the prefix:Note: the reverse direction (
as_mcp_contentat line 143) is already correct — it extracts the full MIME type from the data URI viadata_uri_mime_type().Error Trace
Environment
inspect-ai==0.3.185Image(data=..., format="png")viamcp.server.fastmcp.utilities.types.Imageanthropic/vertex/claude-opus-4-6(but provider-agnostic — any model hittingcount_tokenswith an image in message history would crash)