Skip to content

[pathfinder] Respect site-packages root priority across wheel layouts - #3370

Open
0z5a wants to merge 3 commits into
NVIDIA:mainfrom
0z5a:codex/pathfinder-site-root-precedence-20261104
Open

0z5a wants to merge 3 commits into
NVIDIA:mainfrom
0z5a:codex/pathfinder-site-root-precedence-20261104

Conversation

@0z5a

@0z5a 0z5a commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Description

Closes #3369.

In a virtual environment with system site-packages enabled, a CUDA 13 wheel layout in the user site can be selected before a CUDA 12 layout in the venv, despite the venv site having higher priority. The same issue can occur between user and system sites. The existing search loop considers wheel layouts before site roots.

Search site-packages roots first, then each declared wheel layout within a root. Apply the same ordering to dynamic libraries on Linux and Windows, headers, binary utilities, static libraries, and bitcode libraries. The declared preference between layouts within a single root remains unchanged. This addresses a case not covered by #1717's single-layout fix for #1716.

Validation

  • Ubuntu 24.04.4 x86_64, Python 3.12.3, CPU-only: 1,474 passed across the affected pathfinder finder, search, descriptor, and mocked loader tests, including new root/layout precedence, fallback, and non-venv regressions.
  • Before the fix, all six new venv-versus-user/system cases failed; after the fix, all passed.
  • The 30 new cases are marked thread-unsafe because they modify process-wide site state and caches; with --parallel-threads=4, all 30 passed in the serial portion of the run.
  • In a real --system-site-packages venv with placeholder files, bitcode, static library, header, and binary lookups changed from the user site to the venv site. With two small CPU-only test shared libraries, a fresh load_nvidia_dynamic_lib("cudart") process changed from loading the user-site marker 13 to the venv marker 12. These are synthetic test files, not NVIDIA runtimes.
  • Ruff and the applicable pre-commit checks, including mypy-pathfinder, passed. Windows search behavior was exercised by platform-model tests on Linux; no Windows host or GPU test was run.

Checklist

  • I have read the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

The issue is not yet assigned; maintainer coordination and review are requested.

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
@copy-pr-bot

copy-pr-bot Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions Bot added the cuda.pathfinder Everything related to the cuda.pathfinder module label Nov 4, 2026
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
@coderabbitai

coderabbitai Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Summary

Summary by CodeRabbit

  • Bug Fixes
    • CUDA binaries, headers, dynamic libraries, and static libraries are now searched consistently across virtual-environment, user, and system package locations. A matching item in a higher-priority location is selected before considering alternate layouts in lower-priority locations.
    • If no match is available in a higher-priority location, discovery continues through the remaining locations and layouts, preserving fallback behavior.

Site-package searches now check candidate layouts within each root before moving to a lower-priority root. Binary, dynamic-library, header, static-library, and bitcode finders use this traversal. Tests cover root and layout precedence.

Changes: Site-package search precedence

Layer / File(s) Summary
Order site roots and layouts
cuda_pathfinder/cuda/pathfinder/_utils/find_sub_dirs.py, cuda_pathfinder/tests/test_site_packages_precedence.py
The shared helpers search roots in priority order and check supplied patterns within each root. Parameterized tests cover root priority, layout priority, and fallback behavior.
Use ordered traversal in finders
cuda_pathfinder/cuda/pathfinder/_binaries/*, cuda_pathfinder/cuda/pathfinder/_dynamic_libs/*, cuda_pathfinder/cuda/pathfinder/_headers/*, cuda_pathfinder/cuda/pathfinder/_static_libs/*, cuda_pathfinder/tests/test_find_*.py, cuda_pathfinder/tests/test_search_steps.py
The finders search candidate layouts through the ordered helper. Tests update their mocks and pattern expectations. The Windows wheel-directory docstring describes CUDA 13 preference and CUDA 12 as an x64 fallback.

Assessment against linked issues

Objective Addressed Explanation
Search all applicable layouts in each higher-priority site-package root before moving to the next root, while preserving layout order within each root. [#3369] ✅

Possibly related issues

  • NVIDIA/cuda-python#1093: The new tests cover venv and user/system site-package lookup scenarios. The PR does not add a CI matrix.

Suggested reviewers: rwgk, cpcloud

Change: Bug fix · Severity of issue fixed: Medium

Merge Risk: Minimal · up to 184bd

This change makes site-package lookups prefer a higher-priority root over a lower-priority one across all artifact types. No actionable risk was found in the code. The failing PR metadata check should be fixed by adding an assignee, labels, and a milestone before merge.

Security architecture risk: Low · up to 184bd

The change makes site-package root priority consistent across wheel layouts. It can select different installed native code, but the inspected changes preserve eligible roots, accepted filenames, and existing platform checks. No introduced or materially worsened security issue was established. Validation did not include a Windows host or production NVIDIA runtimes.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — A selected dynamic library executes with the loading process's authority, with downstream dependency resolution delegated to the native loader. An attacker able to populate an eligible root with accepted filenames can influence native-code selection. The PR changes the winner among eligible artifacts but does not itself grant filesystem access, process privileges, or new loader authority.

Trust Boundaries and Controls

  • observed — Linux retains descriptor-approved shared-library filenames and file checks. Windows retains descriptor-approved DLL filenames, suppressed-DLL rejection, and conditional PE architecture validation. Root-first traversal does not remove these eligibility checks.

Resilience and Maintainability Implications

  • observed — The existing loader does not retry sibling root candidates after a selected file fails native loading. Successful results remain process-cached; exceptions are not published as successful results, and already-loaded modules retain precedence. These policies are unchanged, although the newly selected file can differ. The added tests establish fallback for absent files, not present-but-unloadable files.
- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between b361e24 and 184bdec.

Files selected for processing: 11

  • cuda_pathfinder/cuda/pathfinder/_binaries/find_nvidia_binary_utility.py
  • cuda_pathfinder/cuda/pathfinder/_dynamic_libs/descriptor_catalog.py
  • cuda_pathfinder/cuda/pathfinder/_dynamic_libs/search_platform.py
  • cuda_pathfinder/cuda/pathfinder/_headers/find_nvidia_headers.py
  • cuda_pathfinder/cuda/pathfinder/_static_libs/find_bitcode_lib.py
  • cuda_pathfinder/cuda/pathfinder/_static_libs/find_static_lib.py
  • cuda_pathfinder/tests/test_find_bitcode_lib.py
  • cuda_pathfinder/tests/test_find_nvidia_binaries.py
  • cuda_pathfinder/tests/test_find_nvidia_headers.py
  • cuda_pathfinder/tests/test_find_static_lib.py
  • cuda_pathfinder/tests/test_search_steps.py
Files skipped from review as they are similar to previous changes: 1

  • cuda_pathfinder/cuda/pathfinder/_dynamic_libs/descriptor_catalog.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 9 remain after this review.

…ex/pathfinder-site-root-precedence-20261104

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cuda.pathfinder Everything related to the cuda.pathfinder module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG][pathfinder] Wheel layout order can override site-packages root priority

1 participant