Repository navigation
Conversation
### What problem does this PR solve? Issue Number: None Related PR: apache#56874, apache#64304 Problem Summary: Catalogs configured with an external namespace root discover root.db but subsequent operations append the root, selecting db.root or root.db.root. A same-named table in the reversed namespace can load as the wrong object. Resolve rooted database names as the inverse of discovery: direct children use relative names and nested names already contain the configured root. Keep the root as one level, preserve child whitespace, and reject ambiguous nested child levels. Correct existing parity fixtures and cover table, view, provider, DDL and SQL metadata-identity paths. ### Release note Fix Iceberg catalogs with external_catalog.name loading the wrong namespace or failing to find discovered databases and tables. Rooted nested names must include the configured root, and ambiguous dotted child levels are rejected explicitly. ### Check List (For Author) - Test: Unit Test - run-fe-ut.sh: 3151 tests, 0 failures/errors, 11 existing skips; Iceberg has 1968 tests - All 19 new namespace cases pass, including the three original correctness failures - Checkstyle, connector import gate, Groovy 4.0.19 syntax and diff checks pass - Standard FE build is in progress; live SQL/REST regression has not run - Behavior changed: Yes (namespace operations use the object originally discovered) - Does this need documentation: No new configuration; the corrected contract is in the release note
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
Iceberg catalogs can expose databases below the namespace configured by
external_catalog.name. Ordinary discovery returns each direct child's relative name, while REST nested discovery returns complete names containing the root.What problem does this PR solve?
Issue Number: None
Related PR: #56874 (introduced the reversed root order), #64304 (preserved it during the connector migration)
PR Type: Bugfix
Problem Summary:
A catalog rooted at
rootdiscoversroot.db, but subsequent operations can resolvedb.rootorroot.db.root. If the reversed namespace contains a same-named table, loading succeeds with the wrong object. Resolve database names as the inverse of discovery in the common catalog implementation, keeping the configured root in one level. Reject rooted nested names outside that root and child levels that cannot be represented unambiguously.Details
toNamespacealways appended the external root after splitting the display name, while discovery started below that root and used two different display contracts.branch-4.2also has the defect at7523405f6aa8e9df3d110eb432fed1ff6e1c3418; it needs a semantic backport in its legacyIcebergMetadataOps, rather than a mechanical cherry-pick of connector classes. That backport is not included here.Testing
run-fe-ut.sh: 1,968 tests, zero failures/errors, 11 existing skips. Including upstream reactor dependencies: 3,151 tests, zero failures/errors, the same 11 skips.CUSTOM_MVNrestricts the reactor to the Iceberg module and its dependencies and enables Checkstyle; the repository test entry point is unchanged.git diff --checkpass.LIMIT 0, expected unknown-column errors, cache refresh, and CREATE/RENAME routing. Its syntax compiles with the regression framework's Groovy 4.0.19. Live SQL/REST execution is pending../build.sh --fe -j2 --exclude-obs-dependencieswith two Maven workers. The optional OBS dependency closure was excluded.Release note
Fix Iceberg catalogs configured with
external_catalog.nameloading the wrong namespace or failing to find discovered databases and tables. Rooted nested names must include the configured root, and ambiguous dotted child levels are rejected explicitly.Check List (For Author)
Check List (For Reviewer who merge this PR)
AI Disclosure