Proposal: make maxTokenSize of client transport SSE event handling configurable

[dingliang88526-dev]

mcp/event.go

func scanEvents(r io.Reader) iter.Seq2[Event, error] {
scanner := bufio.NewScanner(r)
const maxTokenSize = 1 * 1024 * 1024 // 1 MiB max line size
scanner.Buffer(nil, maxTokenSize)
...
if err := scanner.Err(); err != nil {
if errors.Is(err, bufio.ErrTooLong) {
err = fmt.Errorf("event exceeded max line length of %d", maxTokenSize)
}
if !yield(Event{}, err) {
return
}
}

}

why set a small and no-configurable limit?

[findleyr]

Is this actually causing you problems? Our assumption is that the in-memory event store is not really practical for production applications.

Of course, we could make this configurable, but need to understand the use-case first.

[zxxf18]

I have also been affected by this 1MB limit. Here is my business scenario:

I registered a tool used to list a resource containing a list of server endpoint information descriptions on the server side. Each resource entry includes descriptive information of varying lengths (some long, some short).

When I enter a relatively large amount of server information (currently around 0.87MB), constructing the JSON-RPC response body—according to the protocol specification, where structured data is placed in StructuredContent and the JSON-serialized data is placed in Content—exceeds the 1MB limit.

It is expected that my list will continue to grow, so this 1MB restriction does have a practical impact on me.

Of course, I could implement alternative strategies at the business level, but I similarly believe that this limit should be exposed as a configurable option.

[zxxf18]

My work involves life science research, particularly in the area of complex molecular analysis. These analyses often include sequences and corresponding analytical content that can easily exceed the 1 MB limit.

As a result, I am currently forced to fork a copy of the code and modify the restriction myself in order to continue using it.

[findleyr]

@jba @markus-kusano any objection to making this configurable?

We did design the memory event store to be somewhat production grade (that's why we set a maximum cache size in the first place), so making this configurable seems OK to me.

[findleyr]

In my haste, I misunderstood the report, FWIW: the issue is not about MemoryEventStore, but rather our event scanner.

Agree that we should fix this. Probably 1MB is too small a default, as well.