Repository navigation
Fix schemas for flattened tuple-struct remainders - #39
Conversation
Use the element schema for direct standard Vec remainders so draft-07 additionalItems describes each trailing value. Preserve the Rust representation and Serde behavior. Custom containers and indirect remainders retain their existing schema behavior. Add serializer/schema regression checks and coverage for nested and referenced elements, crate-path overrides, and custom-container traits. Verified with Rust 1.97.1: 492 tests passed, 14 ignored; formatting, Clippy and documentation checks passed.
ahl
left a comment
There was a problem hiding this comment.
nice find (did you hit this by using it?)
I'd like to fix up all the cases, not just the Vec case.
| let additional_item_id = match typespace.types.get(rest_id) { | ||
| Some(Type::Vec(element_id)) | ||
| if path_text(vec_type.path()) == "::std::vec::Vec" | ||
| && vec_type.provision(TypespaceTrait::JsonSchema) | ||
| == TraitProvision::IfParameters => | ||
| { | ||
| element_id | ||
| } | ||
| _ => rest_id, | ||
| }; |
There was a problem hiding this comment.
I think we need something more like array_bounds below
Custom
// containers (including transparent Vec wrappers or aliases) and
// other remainder shapes retain their existing schema behavior,
// which may still describe an extra array level.
yeah: let's fix this.
There was a problem hiding this comment.
Thank you for taking the time to review my pull request. I really appreciate your time and feedback. Although the PR ended up being much larger than I originally anticipated, I hope the contribution proves useful.
nice find (did you hit this by using it?)
No, I stumbled across Oxide’s RFDs and was inspired to try contributing; I really enjoyed your Typify2 post too.
I've followed the array_bounds approach for the other remainder types. The schema fix handles item schemas and length bounds for vectors, sets, fixed arrays, tuples, nested tuple structs and array-valued literals. It walks aliases, boxes and newtypes; custom and opaque containers supply their own schemas without adding element bounds. Nested element schemas stay unchanged, and all known positional fields remain required.
Native tests exercise transparent wrappers, aliases and nested custom containers. One container deliberately implements JsonSchema independently of its parameter; its test checks the item schema and minimum length.
A private utility shared within each output handles the runtime translation. Its helpers separate reference handling, logical branches, literals and array constraints. Most of the work comes from constraints on the entire tail: contains must find a match after the prefix, and an array-valued const or enum describes the complete tail. Logical combinations need the same translation.
Supporting fewer schema constructs would make the helper smaller. I believe we could keep the same traversal for aliases, wrappers and custom containers, and decide how to handle unsupported constraints. I'd like feedback on whether to limit the scope that way.
Sequence references must resolve to ready definitions registered with Schemars. Unsupported references cause a panic during schema generation that names the generated type. A referenced false is rejected because Schemars also uses false for definitions still being built; inline false works.
Existing json-serde adapters cannot round-trip every supported schema shape. Fixed arrays and tuples have schema-only coverage. For a cycle consisting only of wrappers, the item layout stays unknown: the walk stops with an unconstrained item schema and retains the known prefix. That test checks termination and prefix bounds.
Suffix uniqueness is left for a separate change. A tuple with a String title and flattened BTreeSet<String> tags has two independent fields written into one JSON array. Only the tags must be unique; the title may equal a tag.
| JSON | Expected with unique tags | Current schema | Whole-array uniqueItems |
|---|---|---|---|
["Report"] |
Accept | Accept | Accept |
["Report", "red", "blue"] |
Accept | Accept | Accept |
["Report", "red", "red"] |
Reject | Accept | Reject |
["red", "red"] |
Accept | Accept | Reject |
["red", "red", "blue"] |
Accept | Accept | Reject |
In both draft-07 and 2020-12, uniqueItems checks the whole array. It would reject valid serialized values where the title equals a tag. Uniqueness and dependent logical constraints remain relaxed here; a follow-up must decide how to express uniqueness within the tail or report unsupported cases.
For {"not":{"uniqueItems":true}}, a valid tail must have duplicates. Removing only uniqueItems would make the inner schema accept every tail and not reject every tail. The helper records which translations are exact, so it can omit affected negations, oneOf or conditionals and keep accepting values allowed by the original constraint.
Note Consumers relying on the old schema may need to update validators, generated clients or workarounds.
095c773 to
a2ffe6f
Compare
Resolve chains of ready definitions registered with Schemars when shifting an opaque remainder schema past a nonempty tuple prefix. Unavailable or cyclic references now cause a schema-generation panic naming the generated type. Reject referenced false targets because Schemars also uses false for in-progress definitions; intentional inline false remains supported. Leave references within item schemas unchanged. Extend flattened remainder handling through aliases, wrappers, nested tuples, fixed sequences and custom containers without adding element JsonSchema bounds. Share the generated translator and split it into documented private helpers. Add native regression cases and generated snapshots. Keep suffix uniqueness relaxed and leave existing json-serde behavior unchanged. Validation: 532 workspace tests passed, plus Clippy, formatting and docs.
Assert that a flattened set schema accepts duplicate tail values while an ordinary set schema rejects them. Document that suffix uniqueness and logical constraints depending on it remain relaxed, with enforcement deferred to a separate follow-up. Validation: 42 relevant tests, targeted Clippy and formatting pass.
ahl
left a comment
There was a problem hiding this comment.
The general case turned out to be more involved than anticipated. I suggest we address the initial, narrower case, and then consider the right generalized approach.
Let's go back to a2ffe6f and I'll review that more narrowly. Sorry for the churn.
| // We could do something cute here, like export its schema and | ||
| // then--for tuples with flattened remainders--modify the | ||
| // schema at runtime... but that's pretty fragile since we have | ||
| // no idea of the structure of the output schema. |
There was a problem hiding this comment.
I think the approach you've taken is more or less what we had discarded as fragile.
There was a problem hiding this comment.
Thanks for the clarification! I got a bit carried away exploring the general case and probably should have checked in before expanding the scope. I've restored the narrower fix, and I agree that tackling this incrementally makes more sense. Thanks again for taking the time to review it!
| // TODO 11/10/2026 | ||
| // Consider deriving JsonSchema for remainder-free tuple structs where | ||
| // the derive matches the generated schema. One-field TupleStructs must | ||
| // still describe arrays, unlike the default derive for newtypes. |
There was a problem hiding this comment.
Certainly! While revisiting the existing TODO about potentially using #[derive(JsonSchema)] for tuple structs without a rest field, I noticed a possible representation mismatch.
My understanding was based on the existing implementation:
-
test_tuple_marker_struct verifies that a single-field TupleStruct serializes as
[7], whereas a newtype serializes as7. -
TupleStruct::render generates a custom sequence serializer regardless of whether rest is present.
-
The custom JsonSchema implementation explicitly describes the array representation.
Furthermore, I checked Schemars 0.8.22. Its derive implementation treats a struct with one unnamed field as a newtype and generates the schema from the field's type.
This can be reproduced with a small addition inside the existing test_tuple_marker_struct test's inner() function:
#[derive(schemars::JsonSchema)]
struct DerivedListed(u32);
let schema = serde_json::to_value(
schemars::schema_for!(DerivedListed)
).unwrap();
let value = serde_json::to_value(
import::Listed(7)
).unwrap();
assert_eq!(schema["type"], "integer");
assert_eq!(value, serde_json::json!([7]));
assert!(
!jsonschema::draft7::new(&schema)
.unwrap()
.is_valid(&value)
);
// Read the schema-only helper's field to avoid an unused-field warning.
let _ = DerivedListed(7).0;
println!("Derived schema type: {}", schema["type"]);
println!("Typespace Wrapped(7) JSON: {}", serde_json::to_value(import::Wrapped(7)).unwrap());
println!("Typespace Listed(7) JSON: {}", value);
println!("Derived schema accepts Listed(7): {}", jsonschema::draft7::new(&schema).unwrap().is_valid(&value));
// Rust 1.97.1, targeting aarch64-apple-darwin
// running 1 test
// Derived schema type: "integer"
// Typespace Wrapped(7) JSON: 7
// Typespace Listed(7) JSON: [7]
// Derived schema accepts Listed(7): false
// test test_tuple_marker_struct ... ok
Here, DerivedListed has the same Rust struct shape as the generated Listed, but uses Schemars' automatic derivation. The comparison illustrates why replacing the custom schema implementation with a plain #[derive(JsonSchema)] could introduce a mismatch: the derived schema expects an integer, while Typespace’s serializer produces an array.
That was my reasoning behind the additional qualification in the TODO. It was intended to document a consideration for a possible future cleanup, not to introduce a functional change.
Revert c6b6468 and 3083cd3, restoring the exact a2ffe6f tree. Keep the direct standard-Vec correction in this PR. Leave generalized remainder handling and opaque schema translation for separate design discussion, as requested in review. Validation: 492 workspace tests passed; Clippy, formatting and docs passed.
| // Exercise generated paths through a facade as well as the default crate name. | ||
| pub mod schema_facade { | ||
| pub use schemars::*; | ||
| } |
There was a problem hiding this comment.
At this point the facade probably adds unnecessary complexity.
| } | ||
| } | ||
|
|
||
| #[test] |
There was a problem hiding this comment.
I don't think we want a test with no comment that asserts behavior that we know to be incorrect.
| } | ||
|
|
||
| #[test] | ||
| fn indirect_vec_remainders_keep_existing_schema() { |
There was a problem hiding this comment.
I agree that we shouldn't preserve assertions about schema behavior we already know to be incorrect.
However, I think there's a separate regression worth guarding against here: custom containers may implement JsonSchema independently of their element type.
I've narrowed this test to verify only that invariant. It uses a custom OpaqueVec<T> with an unconditional JsonSchema implementation and an element type that deliberately does not implement JsonSchema.
If schema generation accidentally introduces an additional T: JsonSchema requirement, the test will fail to compile.
The revised test no longer makes any assertions about the existing additionalItems representation. I've also added a comment explaining the distinction.
Resolve changelog and import conflicts while preserving the direct standard Vec remainder schema fix and upstream string-constrained newtype changes.
A tuple struct with a flattened
Vec<String>tail serializes as["hello",42,"a","b"], but its schema expects arrays such as["hello",42,["a"],["b"]]. It rejects valid output and accepts data that the deserializer rejects.Draft-07 §6.4.2 applies
additionalItemsto each trailing element. This uses the element schema for direct standardVec<T>remainders with conditionalJsonSchema. The Rust representation and Serde behavior stay the same.Custom containers and indirect remainders are unchanged. A vector remainder behind a type alias still gets the container schema instead of the element schema.
Note This can break consumers that depend on the old schema. Nested tails that used to validate will fail, and generated clients or workarounds may need changes.