Repository navigation
Description
No server scenario sends tools/call arguments that violate the tool's own inputSchema. The scenarios in tools.ts call fixture tools with valid or empty arguments only. SEP-1303 (accepted, in 2025-11-25) has no yaml in src/seps/. The backfill in #385 was closed without merging, and its severity question was left for maintainer sign-off.
Why it is worth a scenario. I ran an execution-based census of the MCP Registry (paper: https://doi.org/10.5281/zenodo.22996025, data and harness: https://doi.org/10.5281/zenodo.22981443). Of 3,685 servers that completed a handshake, 129 (3.5%, 95% CI 3.0–4.1) ran a tool on a wrong-typed required argument their own schema rejects, and returned an ordinary result with no error indication. 112 of the 129 are built on the official TypeScript SDK and none on the Python SDKs, against 2,347 and 1,020 responding servers. On @modelcontextprotocol/sdk 1.30.0, the low-level Server publishes inputSchema in tools/list but never checks tools/call arguments against it, while McpServer returns isError: true (typescript-sdk#2628).
Spec text (2025-11-25 and draft, server/tools):
- Security considerations: Servers MUST "Validate all tool inputs".
- Error handling: input validation errors are Tool Execution Errors, "Reported in tool results with
isError: true" (no RFC 2119 keyword).
Proposal: one new scenario, three checks (introducedIn: '2025-11-25', standalone, so requirements/*.yaml and tier scores are untouched until you choose to promote it).
Fixture: test_input_validation, with one required argument { "value": { "type": "integer", "minimum": 1, "maximum": 10 } }, returning executed: <value>.
| Check id | Probe | Passes when | Severity |
|---|---|---|---|
tools-call-input-validation-valid |
{ "value": 5 } |
ordinary text result (a control: if it fails, the other two report notTestable) |
FAILURE |
sep-1303-input-validation-rejected |
{ "value": "five" }, {}, { "value": 11 } |
none of the three executes; either an isError result or a JSON-RPC error passes |
FAILURE (MUST) |
sep-1303-input-validation-tool-error |
the same three | each rejection is an isError: true result with text content |
WARNING (proposed; see Q1) |
I have a working prototype and ran it against the SDKs' own conformance servers, with the fixture added the idiomatic way (zod / Annotated[int, Field(ge=1, le=10)] / InputSchema + typed AddTool):
| Server | 2025-11-25 | 2026-07-28 |
|---|---|---|
typescript-sdk main, test/conformance everything-server (McpServer) |
3/3 | 3/3 |
python-sdk main, everything-server (MCPServer) |
3/3 | 3/3 |
go-sdk main, conformance/everything-server (stateful / stateless) |
3/3 | 3/3 |
| this repo's everything-server (SDK 1.29) | 3/3 | 3/3 |
negative: low-level Server that runs the handler unchecked |
-rejected FAILURE, -tool-error WARNING |
|
| a server that rejects with JSON-RPC -32602 | -rejected SUCCESS, -tool-error WARNING |
So the scenario fails no Tier-1 SDK on behavior. What each SDK needs is the fixture tool, about 10 lines, and I'd send those PRs to the SDKs myself. The PR would also include the negative example and its vitest case (negative.test.ts), the everything-server fixture, and src/seps/sep-1303.yaml. npm test (643 tests), typecheck, and lint pass locally.
Out of scope on purpose: unknown-tool reporting (#508, #515/#165, and SEP-2145 is still in review).
Questions:
- Severity of
sep-1303-input-validation-tool-error: theisErrorwording has no RFC 2119 keyword. Add SEP-1303 (input validation errors as tool execution errors) requirement-traceability YAML #385 leaned towards FAILURE, and AGENTS.md says severity follows the keyword. Is WARNING right, or should the check wait? - Is one check id that aggregates three probes (cases in
details) the shape you want? - Rust and C# run the server suite without a baseline, so the missing fixture would turn their next pin bump red. Is pairing the merge with fixture PRs to those SDKs the right sequencing, or would you rather start it in
pending?
Happy to open the PR once the shape is agreed.
Activity
feat(server): add tools-call-input-validation scenario (SEP-1303)