[BUG] <title>concurrent-tool-calls

[baili123]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

Bug Report

Which version of ShenYu?

master (verified against f7602e324). The affected code has been unchanged since it was introduced in e2cb6f3ab (2025-07-15, #5999).

Expected behavior

Two or more MCP tools/call requests issued concurrently within the same MCP session should each be proxied independently and return their own result. MCP clients routinely issue parallel tool calls, and at the transport level each call is already a separate HTTP POST carrying its own JSON-RPC id.

Actual behavior

Every concurrent tool call fails. Not a rare race — a 100% failure rate in my tests. Failures surface as three different errors that all look like downstream/network problems:

{"code":-103,"message":"Service invocation exception, or no result is returned!"}
{"code":-106,"message":"Can not find url, please check your configuration!"}
""                                     (empty response)
Tool execution failed: ... NullPointerException: Cannot invoke "java.lang.Long.longValue()"

Responses can also be truncated mid-JSON:

{"jsonrpc":"2.0","id":"p-P1","result":{"content":[{"type":"text","text":"{\"code\
                                                                          ^ stream cut, 81 bytes total

How to reproduce

1. Configure an mcpServer selector with one tool whose requestConfig proxies a POST endpoint that echoes its request body, e.g.

{
  "name": "echo_tool",
  "parameters": [{ "name": "note", "type": "string", "description": "echoed back" }],
  "requestConfig": "{\"requestTemplate\":{\"url\":\"/echo\",\"method\":\"POST\",\"argsToJsonBody\":true,\"headers\":[]},\"argsPosition\":{\"note\":\"body\"}}"
}

2. Open one MCP session:

GW=http://<gateway-host>:9195/<mcp-path>/streamablehttp
SID=$(curl -sD- -o/dev/null -X POST "$GW" \
  -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"probe","version":"1.0"}}}' \
  | grep -i '^Mcp-Session-Id:' | tr -d '\r' | awk '{print $2}')
curl -s -o/dev/null -X POST "$GW" -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' -H "Mcp-Session-Id: $SID" \
  -d '{"jsonrpc":"2.0","method":"notifications/initialized"}'

3. Baseline — call the tool serially twice with distinct note values. Both succeed and each response carries its own note.

4. Now fire three calls concurrently on the same session:

for n in X Y Z; do
  curl -s -X POST "$GW" -H 'Content-Type: application/json' \
    -H 'Accept: application/json, text/event-stream' -H "Mcp-Session-Id: $SID" \
    -d "{\"jsonrpc\":\"2.0\",\"id\":\"c-$n\",\"method\":\"tools/call\",\"params\":{\"name\":\"echo_tool\",\"arguments\":{\"note\":\"$n\"}}}" &
done; wait

Results observed

scenario	outcome
2 serial calls (before concurrency)	2/2 correct, each response matched its own note
3 concurrent × 3 rounds	9/9 failed, 0 succeeded
2 concurrent	2/2 failed (one -103, one truncated response)
2 serial calls (after concurrency)	2/2 correct — the session is not poisoned; failures are strictly concurrent-only

Root cause

Each concurrent tool call arrives as its own HTTP POST and therefore already has its own ServerWebExchange. That isolation is then discarded: the exchange is stored in a static map keyed by session id, so N concurrent requests collapse into one slot.

ShenyuMcpExchangeHolder:

private static final Map<String, ServerWebExchange> EXCHANGE_MAP = new ConcurrentHashMap<>();

public static void put(final String sessionId, final ServerWebExchange exchange) {
    EXCHANGE_MAP.put(sessionId, exchange);   // later request overwrites the earlier one
}

ShenyuStreamableHttpServerTransportProvider#configureExchangeForSession (line 566) stores every POST's exchange under that single key, and ShenyuToolCallback#call (line 134) reads it back by session id:

final String sessionId = extractSessionId(mcpExchange);
final ServerWebExchange originExchange = getOriginExchange(sessionId);
final ShenyuPluginChain chain = getPluginChain(originExchange);

Because the tool call reuses the inbound exchange and replays the plugin chain on it, all per-request state lives on that now-shared object and concurrent calls overwrite each other's attributes. Each observed error maps to one clobbered attribute:

error	attribute lost	site
-106 Can not find url	HTTP_URI (written by URIPlugin)	AbstractHttpClientPlugin:67
-103 no result	CLIENT_RESPONSE_CONN_ATTR	NettyClientMessageWriter:60
empty / truncated body	response written by two writers	NettyClientMessageWriter response.writeWith(body)

Suggested fix

Either of:

1. Key the holder by the JSON-RPC request id (or any per-call token) instead of the session id, and clean the entry up when the call completes. MCP explicitly allows concurrent in-flight requests per session, which is exactly what the JSON-RPC id is for.
2. Do not reuse the inbound exchange at all — build a fresh outbound request per tool call rather than mutating and replaying the inbound one.

Option 2 also removes the need for the blocking wait in ShenyuToolCallback:270 (responseFuture.get(60, SECONDS)), which currently blocks inside a reactive pipeline.

Notes

Since the per-tool-call timeout here is 60s while a divide rule with the default retry = 3 can take 4 × timeout, the two limits can also disagree; that is a separate, smaller concern.

Expected Behavior

No response

Steps To Reproduce

No response

Environment

ShenYu version(s):

Debug logs

No response

Anything else?

No response

[github-actions]

👋 Hello @yuluo-yx

A new issue has been opened by @baili123:

• Title: [BUG] <title>concurrent-tool-calls
• Link: [BUG] <title>concurrent-tool-calls #7041

Please review when you have time. Thank you! 🙏

[zhang-arvin]

I will work on this.

[zhang-arvin]

I would like to work on this issue.

[zhang-arvin]

Tracked by PR #7046