Repository navigation
Conversation
So that a launch can link more than one job for the same kernel, and look at the cached results of each.
KernelInterface kernels were all compiled for the sub-group width that `KI.sub_group_size` promises, 32 where the device supports it. Register-heavy kernels spill heavily at that width: a register-blocked matmul runs 1.6x (Float32) to 5x (ComplexF32) slower than at SIMD16 on Xe-LP and Xe-HPG. Forcing 16 for every kernel isn't better: kernels that don't spill are up to ~20% faster at 32, and IGC's own choice, when no width is required, drops such kernels to SIMD8. Only kernels that can observe the width need the promised one. When a KernelInterface kernel spills at that width, check its SPIR-V for anything that could make the width observable: a capability beyond the ones kernels use without sub-groups, a sub-group built-in variable or kernel query, or a barrier, fence or atomic whose scope is the sub-group or not a constant. If there is none, compile the kernel again for SIMD16, and use that unless the driver limits it to smaller work-groups or doesn't report its limit. Kernels launched with `@oneapi` keep the width they ask for. Fixes #772.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #773 +/- ##
==========================================
- Coverage 87.34% 86.87% -0.48%
==========================================
Files 56 55 -1
Lines 5263 5180 -83
==========================================
- Hits 4597 4500 -97
- Misses 666 680 +14 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
maleadt
marked this pull request as draft
November 20, 2026 08:55
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #772.
KernelInterface kernels, and so every KernelAbstractions
@kernel, are compiled for the sub-group width thatKI.sub_group_sizepromises: 32 where the device supports it. Register-heavy kernels spill heavily at that width. For the register-blocked matmul from JuliaGPU/GPUArrays.jl#856, launched through KernelInterface at 1024³:Promising 16 instead doesn't help in general: kernels that don't spill are faster at 32. On Iris Xe, GPUArrays' tiled matmul runs 20% slower at 16, and AcceleratedKernels' reductions and scans 10–20% slower. Leaving the width to IGC isn't better either. Without a required size, IGC compiles the blocked matmul for SIMD8 on Xe-LP and Xe-HPG (7.4 ms vs 4.9 ms at SIMD16 on Iris Xe). On Xe-HPC it keeps a spilling SIMD32. And kernels that use sub-groups need the promised width: AcceleratedKernels sizes local memory from it, and returns wrong sums without it.
So the promise stays, but only for kernels that can observe it. When a KernelInterface kernel spills at the promised width, its SPIR-V is checked for anything that could make the width observable:
If there is none, the kernel is compiled again for SIMD16. That version is used unless the driver limits it to smaller work-groups, or doesn't report its limit. The check runs on the emitted SPIR-V rather than LLVM IR, so that it fails closed: anything it doesn't recognize keeps the promised width. Kernels that don't spill compile exactly as before, and so do kernels launched with
@oneapi. A spilling kernel is compiled twice the first time it is used: on Iris Xe, the blocked matmul takes 0.5–0.6 s instead of 0.2–0.3 s.The tests check:
Validated on Iris Xe and Arc A770, including GPUArrays'
sum,cumsum,findmax/findmin,sum(dims),permutedimsand matmul, which go through AcceleratedKernels' sub-group kernels.Follow-ups, not in this PR:
sum1.83 → 1.17 ms) and 32 where they aren't (Xe-LP). That is a KernelInterface change; see KernelInterface 1.0: a better API for writing kernels KernelAbstractions.jl#810, item 4.@openclones, and has the same problem on Intel GPUs.