Skip to content

StreamServerTransport materializes each stdio line as a string, costing ~9× the frame size per large request, with no upper bound on message size #2004

Open

Description

@asawicki

Summary

StreamServerTransport (and therefore StdioServerTransport) reads each incoming message with ReadLineAsync and parses it with JsonSerializer.Deserialize(string, …):

https://github.com/modelcontextprotocol/csharp-sdk/blob/v2.3.0/src/ModelContextProtocol.Core/Server/StreamServerTransport.cs#L104
https://github.com/modelcontextprotocol/csharp-sdk/blob/v2.3.0/src/ModelContextProtocol.Core/Server/StreamServerTransport.cs#L120

For a large tools/call (e.g. a base64 image argument), this has two costs:

  1. Memory amplification. The whole line exists as a UTF-16 string (2 bytes per char). Deserialize(string) then rents a worst-case transcoding buffer from ArrayPool<byte>.Shared (128 MiB for a ~27 MB line). The pool keeps that buffer after the call, so it stays resident.
  2. No upper bound. No option limits the size of a single line. A client can make the server buffer an arbitrarily large frame before any handler or validation runs.

Parsing the same frame from UTF-8 bytes avoids the string and the 128 MiB rent.

Repro (parsing step only)

A single-file .NET 10 app (dotnet run repro.cs). It builds one tools/call frame with a 20 MiB payload (27,962,028 base64 chars) and parses it twice: from a string, as StreamServerTransport does today, and from UTF-8 bytes.

repro.cs
#:package ModelContextProtocol.Core@2.3.0
#:property EventSourceSupport=true
using System.Diagnostics.Tracing;
using System.Text;
using System.Text.Json;
using ModelContextProtocol;
using ModelContextProtocol.Protocol;

using var listener = new PoolListener();
var payload = Convert.ToBase64String(new byte[20 * 1024 * 1024]);
byte[] frame = Encoding.UTF8.GetBytes(
    "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"tools/call\",\"params\":{\"name\":\"evaluate\",\"arguments\":{\"image_payload\":\""
    + payload + "\"}}}");
string line = Encoding.UTF8.GetString(frame);
var typeInfo = McpJsonUtilities.DefaultOptions.GetTypeInfo<JsonRpcMessage>();
JsonSerializer.Deserialize("{\"jsonrpc\":\"2.0\",\"id\":0,\"method\":\"ping\"}", typeInfo); // warm-up

Measure("Deserialize(string) - current transport", () => JsonSerializer.Deserialize(line, typeInfo));
Measure("Deserialize(byte[]) - parse from UTF-8", () => JsonSerializer.Deserialize(frame, typeInfo));

void Measure(string name, Func<object?> parse)
{
    listener.Rents.Clear();
    listener.Enabled = true;
    long before = GC.GetTotalAllocatedBytes(precise: true);
    var result = parse();
    long after = GC.GetTotalAllocatedBytes(precise: true);
    listener.Enabled = false;
    Console.WriteLine($"{name}: allocated {(after - before) / 1048576.0:F1} MiB; pool rents >= 1 MiB: "
        + string.Join(", ", listener.Rents.Select(r => $"{r / 1048576} MiB")));
    GC.KeepAlive(result);
}

sealed class PoolListener : EventListener
{
    public volatile bool Enabled;
    public readonly List<int> Rents = new();
    protected override void OnEventSourceCreated(EventSource s)
    { if (s.Name == "System.Buffers.ArrayPoolEventSource") EnableEvents(s, EventLevel.Verbose); }
    protected override void OnEventWritten(EventWrittenEventArgs e)
    {
        if (!Enabled || e.EventName != "BufferRented" || e.Payload is null) return;
        int size = (int)e.Payload[1]!;
        if (size >= 1 << 20) lock (Rents) Rents.Add(size);
    }
}

Output (macOS arm64, .NET SDK 10.0.403, ModelContextProtocol.Core 2.3.0; frame = 26.7 MiB):

Parse path Allocated ArrayPool<byte>.Shared rents ≥ 1 MiB
Deserialize(string) (current transport) 186.7 MiB 128 MiB, 32 MiB
Deserialize(byte[]) 26.7 MiB 32 MiB

The results were identical across two runs. The second row benefits from the 32 MiB buffer already being in the pool; in a fresh process that buffer adds 32 MiB, which still leaves the byte path at about a third of the string path. The line string itself (~53 MiB of UTF-16) comes on top of the first row, allocated by ReadLineAsync before this step.

Impact in a running stdio server

We measured a stdio server with a single tool that receives a 20 MiB base64 argument. Each call ran in sequence, the handler ignored the argument, and we report the growth in working set:

SDK StdioServerTransport Custom transport on TransportBase (PipeReader, parse from bytes)
1st call, growth / payload, NativeAOT 15.7× (314 MiB) 7.1× (141 MiB)
1st call, growth / payload, JIT 15.9–27.0× 7.2×
peak after 10 calls, NativeAOT 334 → 1619 MiB 160 → 697 MiB

After a forced full GC, a heap dump of the SDK path still showed 5 × 128 MiB byte[] held by ArrayPool<byte>.Shared, which is consistent with the transcoding rent above. The custom transport removed those.

The custom transport also enforces a frame limit before the line is materialized. A 27 MB frame against a 1 MiB limit grew the working set by about 1.2 MiB, was rejected with -32600 carrying the request id, and the session kept working.

Our contract tests (initialize, tools/list, tools/call, error paths) passed with both transports, under JIT and NativeAOT.

Part of the remaining growth with the custom transport also sits in large pooled buffers that this transport does not rent. We have not pinned down where they come from, so they are left out of this issue.

Suggestion

  • Read the input with PipeReader, find the line end with SequenceReader<byte>, and parse with JsonSerializer.Deserialize(ReadOnlySequence<byte>) or Utf8JsonReader, with no intermediate string.
  • Add an option such as MaxMessageSize to StreamServerTransport and StdioServerTransport. A line over the limit is discarded without buffering it whole and answered with -32600, using the id when it can be recovered from the line's first bytes.

We are happy to contribute a PR if this direction is acceptable.

Related: #1971 (large allocations on the outgoing path, when tool results are serialized).

Activity

  1. asawicki commented on Nov 7, 2026

    @asawicki
    Author

    Follow-up: a second, larger allocation on the same request path, which the repro above misses because its payload is all zeros.

    When a string argument contains characters that the default encoder escapes (base64 contains +, which JavaScriptEncoder.Default escapes), JsonSerializer.Deserialize(request.Params, requestTypeInfo) in RequestHandlers rents a 256 MiB buffer from ArrayPool<byte>.Shared for a ~27 MB argument. Re-serializing the JsonNode reserves worst-case escaping space (about 6× the string length):

    https://github.com/modelcontextprotocol/csharp-sdk/blob/v2.3.0/src/ModelContextProtocol.Core/RequestHandlers.cs#L68

    Stack captured in a running stdio server (ModelContextProtocol 2.3.0), once per tools/call, with both the SDK transport and a custom transport:

    Utf8JsonWriter.WriteStringEscapeValue
    JsonDocument.WriteComplexElement
    JsonObject.WriteTo
    JsonSerializer.ReadFromNode<TValue>
    RequestHandlers.<SetWithAlternate>b__0
    McpServerImpl.InvokeWithInputRequiredResultHandlingAsync
    McpSessionHandler.HandleRequestAsync
    

    To reproduce it with the repro above, replace the payload line with random bytes:

    var raw = new byte[20 * 1024 * 1024];
    new Random(42).NextBytes(raw);
    var payload = Convert.ToBase64String(raw);

    Then time JsonSerializer.Deserialize(request.Params, McpJsonUtilities.DefaultOptions.GetTypeInfo<CallToolRequestParams>()) on the JsonRpcRequest parsed from the frame. The results (macOS arm64, .NET SDK 10.0.403):

    payload allocated by Deserialize(request.Params) ArrayPool<byte>.Shared rents ≥ 1 MiB
    all zeros (no +) 53.4 MiB 32 MiB
    random bytes (contains +) 313.5 MiB 256 MiB, 32 MiB

    Since the pool keeps the 256 MiB buffer after the call, it stays resident. This step also runs after the transport, so a custom transport cannot avoid it.

    Possible directions:

    • Deserialize params from the original UTF-8 bytes or a JsonElement instead of round-tripping through JsonNode.
    • When re-serialization is unavoidable, write with an encoder that does not escape +, or with a buffer sized from the actual escaped length.
  2. spaulah commented on Nov 8, 2026

    @spaulah

    @asawicki, thanks for the detailed write-up. You offered to contribute a PR. Are you already working on it? If so, I'm happy to review or test instead. If not, I'd like to take it.

    For the maintainers: I have a working prototype. With the 20 MiB tools/call repro run through StreamServerTransport itself, allocations drop from 368 MiB to 38 MiB per call, and the 128 MiB ArrayPool rent goes away. With a 1 MiB limit, the frame is rejected using under 1.5 MiB.

    Two things I found while testing the limit:

    • This SDK writes id after params, so the id has to be found by scanning the whole discarded frame, not just its first bytes. Otherwise a C# client's oversized request never gets its -32600.
    • An oversized response (for example, a large sampling result) carries the server's own request id. It should fail the server's pending request locally rather than send -32600 to the client.

    Two questions before I open a PR: for stdio, should the limit go on McpServerOptions or on a WithStdioServerTransport overload? And should it default to no limit, so nothing breaks?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions