[shiliang]

Background

array_map builds a per-element lambda block for nested lambdas. Outer lambda arguments that remain visible on the LambdaExecutionContext stack used to be copied into that block whether or not the nested body read them. SlotRefs are collected separately; name-bound lambda arguments are resolved through resolve_column_position.

What problem does this PR solve?

Issue Number: close #69918

Related PR: None

PR Type: Bug Fix

Problem Summary: A nested array_map copied every visible outer lambda argument into the inner lambda block once per element, including outer arrays the body never reads. For array_map(x -> array_map(y -> y + 1, x), arr_of_arrays) this made memory quadratic in array length. Restrict capture to bindings the body actually resolves by name (innermost shadowing), and rename the pad column from temp to __DORIS_LAMBDA_BLOCK_PAD__ so it cannot collide with a user argument named temp.

Details

• Change map: ~30 lines production (varray_map_function.cpp, lambda_execution_context.h), ~105 lines unit test.
• Frequency: Any nested array_map whose outer arguments include unused wide arrays; worse as array length grows.
• Root cause: collect_visible_binding_column_positions inserted every visible binding position into required_input_column_ids, so unused outer arrays were expanded with insert_indices_from / clone_resized per inner element.
• Why existing tests missed it: Nested tests asserted result correctness for referenced captures, not that unused outer arguments stayed unmaterialized.
• Fix: When bind_by_name, walk the lambda body and collect only is_column_ref names that resolve_column_position finds; keep SlotRef collection unchanged. Pad unused sparse slots with __DORIS_LAMBDA_BLOCK_PAD__.
• Other paths and branches: Legacy bind_by_name=false path unchanged. array_sorted_merge still uses collect_visible_bindings by name on its private block.

Testing

• Added BE UT NestedLambdaDoesNotMaterializeUnusedOuterArgument: asserts the inner lambda block has only the y column while still returning correct y+1 results with a large unused outer array.
• Existing nested array_map UTs in array_map_function_test.cpp remain the regression net for referenced outer captures and shadowing (to be re-run in CI run beut).

Notes for Reviewers

Start at ArrayMapFunction::execute where required_input_column_ids is built; the new helper is _collect_referenced_binding_column_positions.

Release note

Nested array_map no longer materializes unused outer lambda arguments into each inner element, avoiding quadratic memory growth on large arrays. Query results are unchanged.

Check List (For Author)

• Behavior changed: No
• Does this need documentation: No
• Pre-submission status: Partially verified: unit test added; BE UT to be run in CI

AI Disclosure

• Tool: Cursor Agent
• Generated: code, unit test, and this PR description
• Human review: Please review capture selection, pad rename, and the new UT against [Enhancement] Nested lambda copies every visible outer lambda argument per element, so memory is quadratic in array length #69918