[vldpyatkov]

Jira Link

CALCITE-7815

Changes Proposed

Adds support for SEARCH DEPTH FIRST BY ... SET ... and SEARCH BREADTH FIRST BY ... SET ... in recursive CTEs. The generated column can be used in ORDER BY to obtain the corresponding traversal order.

The clause is represented by SqlSearchClause on SqlWithItem. During validation, RecursiveWithRewriter builds an ARRAY<ROW<...>> path for depth-first ordering or a row containing depth and keys for breadth-first ordering. SEARCH can be combined with CYCLE.

Also updates enumerable comparators to handle composite keys in multi-column sorts and adds Java and iq tests.

[rubenada]

@vldpyatkov thanks for pushing this forward!
Quick question: IIUC the usage of SEARCH on a diamond shape graph could lead to "duplicated" results, e.g.:

Edges: 0→1, 0→2, 1→3, 2→3. Node 3 can be reached two ways.

WITH RECURSIVE edges(src, dst) AS (VALUES (0,1), (0,2), (1,3), (2,3)),
walk(node) AS (
  VALUES (0)
  UNION
  SELECT e.dst FROM walk AS w JOIN edges AS e ON e.src = w.node)
SELECT node FROM walk

Without SEARCH: 0,1,2,3
With SEARCH DEPTH: (0, [0]), (1, [0,1]), (2, [0,2]), (3, [0,1,3]), (3, [0,2,3])
On the second case, 3 is returned two times (because paths differ).

Analogously, on a graph: 0→1, 1→3, 0→3
Without SEARCH: 0,1,3
With SEARCH BREADTH: (0, ROW(0,0)), (1, ROW(1,1)), (3, ROW(1,3)), (3, ROW(2,3)).
On the second case, 3 is returned two times (because depths differ).

If this is the case, could we have some tests showing these cases? Shall we document somehow this phenomenon?

[vldpyatkov]

@vldpyatkov thanks for pushing this forward! Quick question: IIUC the usage of SEARCH on a diamond shape graph could lead to "duplicated" results, e.g.:

Edges: 0→1, 0→2, 1→3, 2→3. Node 3 can be reached two ways.

WITH RECURSIVE edges(src, dst) AS (VALUES (0,1), (0,2), (1,3), (2,3)),
walk(node) AS (
  VALUES (0)
  UNION
  SELECT e.dst FROM walk AS w JOIN edges AS e ON e.src = w.node)
SELECT node FROM walk

Without SEARCH: 0,1,2,3 With SEARCH DEPTH: (0, [0]), (1, [0,1]), (2, [0,2]), (3, [0,1,3]), (3, [0,2,3]) On the second case, 3 is returned two times (because paths differ).

Analogously, on a graph: 0→1, 1→3, 0→3 Without SEARCH: 0,1,3 With SEARCH BREADTH: (0, ROW(0,0)), (1, ROW(1,1)), (3, ROW(1,3)), (3, ROW(2,3)). On the second case, 3 is returned two times (because depths differ).

If this is the case, could we have some tests showing these cases? Shall we document somehow this phenomenon?

@rubenada The generated SEARCH column participates in duplicate elimination. Documented this behavior in the SQL reference.

[rubenada]

@vldpyatkov sorry, I just added two final minor remarks that I had missed on my last review. Please take a look. I think the PR is in a good shape, so you can also proceed and squash commits into a single one to prepare the merge.

[vldpyatkov]

@rubenada I've addressed the last two comments and squashed everything into a single commit.
Thanks for the thorough review!

[sonarqubecloud]

Quality Gate passed

Issues
8 New issues
0 Accepted issues

Measures
0 Security Hotspots
96.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[rubenada]

LGTM, the build-website error seems an unrelated problem to be handled separately, no reason to block the current PR because of that. I'll merge in the next 24-48h if not further comment arrives.

[vldpyatkov]

LGTM, the build-website error seems an unrelated problem to be handled separately, no reason to block the current PR because of that. I'll merge in the next 24-48h if not further comment arrives.

I closed and reopened PR by misclicking.
Really, I wanted to mention that @rubenada promised to merge it.