Skip to content

[CALCITE-7918] SQL validation takes too long for wider row types - #5391

Open
darpan-14 wants to merge 2 commits into
apache:mainfrom
darpan-14:CALCITE-7918
Open

darpan-14 wants to merge 2 commits into
apache:mainfrom
darpan-14:CALCITE-7918

Conversation

@darpan-14

@darpan-14 darpan-14 commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Jira Link

CALCITE-7918

Changes Proposed

The improvements are separated into two commits for review:

  1. Reuse SqlNameMatcher for field ambiguity checks. Route ambiguity and JOIN-column uniqueness checks through SqlNameMatcher, retaining duplicate fields in the exact-name index.
  2. Index case-insensitive field-name lookups. Add an independent lazy case-insensitive index in RelRecordType, preserving duplicate matches and runtime Unicode matching semantics.

Validation

27 focused tests passed: 11 field-lookup tests, 6 case-insensitive-index tests, 2 matcher-extension tests, and 8 SQL-validator tests. Core Checkstyle (main/test), Autostyle, Javadoc and git diff --check passed.

The source tree is identical to the validated version; this update only consolidates the commits into this PR and clarifies their subjects. The full Gradle build was not run.

Previously posted timings describe the original combined patch, including qualified-column reuse, which remains a separate follow-up.

Reuse SqlNameMatcher for field-name ambiguity and JOIN-column uniqueness
checks. Retain all exact-name candidates in RelRecordType so duplicate
checks can reuse indexed lookup while insensitive matching still scans.
Add an independent lazy case-insensitive field-name index to RelRecordType.
Retain duplicate candidates and preserve runtime Unicode matching semantics.
@sonarqubecloud

sonarqubecloud Bot commented Nov 5, 2026

Copy link
Copy Markdown


/** Candidate indexes are derived from the fields and rebuilt on demand. */
private transient volatile @Nullable Map<String, List<RelDataTypeField>> fieldNameMap;
private transient volatile @Nullable Map<CaseInsensitiveFieldName, List<RelDataTypeField>>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why does this have to be volatile?

* @param elideRecord Whether to find fields nested within records
* @return Matching fallback field, or null if there is none
*/
protected @Nullable RelDataTypeField resolveFieldFallback(String fieldName,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this method have to be protected?

return null;
}

/** Returns a fallback after declared, nested, and extra-field lookup fail.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand: what is a fallback?


/** Returns all declared fields matching a name, in field order.
*
* <p>Duplicate matches are retained. This method does not synthesize matches

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does it mean "does not synthesize matches"?

An empty list is returned when there are no matches? Then maybe "All matching fields" is enough.

return fields.build();
}

/** Counts occurrences of a field name among a row type's declared fields.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I usually think of frequency as a ratio, and a count as absolute.

* @param fieldName Name to count
* @return Number of matching field names according to this matcher
*/
default int fieldFrequency(RelDataType rowType, String fieldName) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this method needed?

}

@Override public List<RelDataTypeField> fields(RelDataType rowType, String fieldName) {
if (rowType instanceof RelDataTypeImpl) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is ugly.
Can you think of making this nicer?
Maybe with some interface?

}
}

private static List<RelDataTypeField> scan(List<RelDataTypeField> fields,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you really need this many tests?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants