Repository navigation
Conversation
… APIs retained - Implemented Rust-only policy API with lower/upper/preserve options - Added explicit override for policy settings - Ensured consistent discovery, binding, and DDL - Retained legacy bool APIs - No session setting included
…acy normalization - Fixed function lookup to retain legacy normalization despite explicit object policy. - Added regression coverage for policies, quoted names, legacy modes, and ordinary calls.
- datafusion/sql/tests/sql_integration.rs: exact UPDATE assertions with unquoted uppercase and quoted lowercase cases. - docs/source/library-user-guide/upgrading/57.0.0.md: explains explicit Preserve vs legacy false.
…multipart - Fixed variables retaining legacy boolean folding independent of object policy. - Ensured consistent behavior across identifier expressions.
- Implement strict schema binding with exact expressions and types, including quoted and unquoted values - Add escaped partition quote for exact key-schema name matching
- statement.rs: renamed private legacy helpers; policy-aware methods unchanged. - sql_integration.rs: explicit COPY partition expectations.
…ble Logic - Merged struct declaration/reference checks for clarity. - Consolidated partition cases and lateral setup into a single pattern. - Implemented table-driven variable expectations for consistent behavior.
github-actions
Bot
added
documentation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Related to #26580.
Rationale for this change
Rust SQL planning currently supports lowercase folding or preservation of unquoted identifiers, but not uppercase folding. Identifier binding, relation discovery, and DDL naming also use different normalization paths, which can produce inconsistent catalog keys.
This change allows Rust callers to select a consistent uppercase policy without rewriting SQL or introducing a session configuration setting. It preserves existing behavior for callers that do not explicitly select a policy.
What changes are included in this PR?
IdentNormalization::{Lower, Upper, Preserve}with ASCII-only case folding and preservation of quoted identifiers.ParserOptions, including a fluent builder and effective-policy accessor. Retain existing boolean APIs and compatibility wrappers.ParserOptionsconstruction sites and document the public API migration in the 57.0.0 upgrade guide.No session configuration setting is introduced.
What is the testing strategy for this PR?
The patch adds the following unit and integration tests:
ident_normalizer_policy: ASCII folding, quoted identifiers, empty names, legacy compatibility, and policy precedence.resolve_table_references_uppercase_policy: Exact physical-table and CTE discovery, quoted and multipart names, shadowing, and recursive references.parse_ident_normalization_uppercase: Table and column binding, aliases, CTEs, qualified wildcards, builtin functions, scalar variables, and configuration keys.ident_normalization_partitions: COPY and external-table partition identifiers, including quoted forms.ident_normalization_pipe_alias: Alias normalization across pipe-query stages.ident_normalization_update: Exact UPDATE target, projection, and table-scan names.ident_normalization_struct_fields: Structured-field declarations and references.ident_normalization_expression_alias: Direct expression-planning aliases.create_schema_uppercase_policy: Catalog, schema, table, and view naming, including legacy behavior.ident_normalization_scalar_variables: Preservation of legacy scalar-variable normalization.lateral_table_function_lookup_preserves_legacy_name_normalization: Independent table-function registry lookup.lambda_ident_normalization_uppercase: Lambda binding, quoted distinctions, duplicate detection, and legacy compatibility.The tests use distinguishable
FOOandfooschemas to verify actual binding behavior.Test execution results, code coverage, and planning benchmark results have not been provided.
Are there any user-facing changes?
Yes. Rust SQL API consumers can explicitly select uppercase identifier folding through
ParserOptions::with_ident_normalization()and use matching policy-aware relation-discovery APIs.Adding the public
ParserOptions.ident_normalizationfield is a source-breaking API change for callers using complete struct literals. Such callers must addident_normalization: Noneor use a default-based construction. This is documented in the 57.0.0 upgrade guide and requires theapi changelabel.Explicit
Preservealso differs from the legacy disabled-normalization flag for certain historically lowercase-only paths.Folding remains ASCII-only. Quoted identifiers retain their spelling. The existing limitation involving dots inside quoted schema-name components in schema DDL remains unchanged.
LLM-generated code disclosure
This PR includes LLM-generated code and comments. All LLM-generated content has been manually reviewed.