Skip to content

POCL: claim the exception output with an atomic max - #976

Merged
maleadt merged 1 commit into
mainfrom
tb/exception-lock-stack
Nov 17, 2026
Merged

maleadt merged 1 commit into
mainfrom
tb/exception-lock-stack

Conversation

@maleadt

@maleadt maleadt commented Nov 13, 2026

Copy link
Copy Markdown
Member

Fixes #974.

Kernels with many bounds checks overflowed the thread stack at large work-groups on the CPU back end. The stack went to the exception lock in claim_output, not to the BoundsError arguments, which #966 already removed.

claim_output took the lock with a compare-and-swap. PoCL implements that with atomic_compare_exchange_strong_explicit, which returns the old value through a stack slot. From there:

  • claim_output reads work-item ids, so PoCL inlines it into each of its call sites (two per throw site) before forming the work-item loops. The loops then give every copy of the slot an array over the work-group.
  • With LLVM 22, alias analysis treats the acquire-release cmpxchg as clobbering the slot. The stored value isn't forwarded and the arrays stay: about 8 bytes per bounds check per work-item. The issue's kernel (256 checks, 64×64 work-items) had an 8.4 MB work-group frame.
  • PoCL's stack estimate, reported as CL_KERNEL_PRIVATE_MEM_SIZE, counts the slot only once (4 bytes). Neither KernelAbstractions' 1 MiB check nor PoCL's CL_KERNEL_WORK_GROUP_SIZE limit caught it.

LLVM 23 knows that a local that never escapes isn't affected by the cmpxchg (llvm/llvm-project#193939) and removes the arrays, so PoCL builds on LLVM 23 don't have this problem.

An atomic max takes the lock the same way (0 becomes 1; 1 and 2 stay) and returns the old value directly. The issue's kernel now has a 216-byte frame.

The new test runs 256 bounds checks in a work-group of the device's maximum size. Without this change it fails with a StackOverflowError.

OpenCL.jl has the same lock, and the same problem at debug level 2: JuliaGPU/OpenCL.jl#609.

PoCL implements a compare-and-swap with its
`atomic_compare_exchange_strong_explicit` builtin, which returns the old
value through a stack slot. `claim_output` reads work-item ids, so PoCL
inlines it into each of its call sites, two per throw site, before it forms
the work-item loops, and those give every copy of the slot an array over the
work-group. With LLVM 22, alias analysis treats the acquire-release cmpxchg
as clobbering the slot, so the stored value isn't forwarded and the arrays
stay: about 8 bytes per bounds check per work-item. PoCL's stack estimate,
which keeps host work-groups within 1 MiB, counted the slot only once, so
kernels with a few hundred bounds checks overflowed the stack at large
work-groups. LLVM 23 knows that a local that never escapes isn't affected
by the cmpxchg (llvm/llvm-project#193939) and removes the arrays.

An atomic max takes the lock the same way (0 to 1, leaving 1 and 2 as they
are) and returns the old value directly.

Fixes #974.
@maleadt maleadt added the pocl The PoCL-based CPU back-end. label Nov 13, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Benchmark results

Compared with 7c56d9a95, the merge base.
The environment differs from the baseline's: GPUCompiler 2.22.1 → 2.22.2, StructUtils 2.12.0 → 2.12.1, StructUtilsStaticArraysCoreExt 2.12.0 → 2.12.1.

Times are the minimum time of every benchmark. A change is unusual when it exceeds both three times the benchmark's typical change between consecutive runs on the default branch and a minimum per category (performance 3%, latency 5%, applications 3%, other 3%). A benchmark that starts or stops allocating is listed however few the allocations. A single run can still be a fluke, so rerun before drawing conclusions.

Geometric mean Unusual: slower Unusual: faster Compared
Performance −0.5% 0 1 32
Latency −0.7% 0 0 13
Applications +0.0% 0 0 5
Benchmark Baseline This PR Change
Performance
pocl/parallel/private/across-barrier 43.8 µs 42.2 µs −3.7% 🟢 faster
All benchmarks
Benchmark Baseline This PR Change
Performance
julia/serial/arithmetic/muladd-reference 655 µs 655 µs −0.0%
julia/serial/math/exp-f64-reference 284 µs 284 µs −0.1%
julia/serial/narrow/masked-reference 735 µs 730 µs −0.7%
pocl/parallel/atomics/histogram 5.24 ms 5.18 ms −1.1%
pocl/parallel/groupreduce/add-f64 1.10 ms 1.11 ms +0.9%
pocl/parallel/groupreduce/fallback 4.38 ms 4.38 ms +0.1%
pocl/parallel/groupscan/add-i32 394 µs 394 µs +0.2%
pocl/parallel/indexing/cartesian3d 40.0 µs 40.9 µs +2.1%
pocl/parallel/indexing/linear3d 40.2 µs 40.9 µs +1.9%
pocl/parallel/indexing/ntuple3d 40.4 µs 40.6 µs +0.4%
pocl/parallel/indexing/ntuple4d 128 µs 127 µs −0.2%
pocl/parallel/localmem/transpose-naive 433 µs 432 µs −0.3%
pocl/parallel/localmem/transpose-tiled 62.5 µs 62.3 µs −0.2%
pocl/parallel/private/across-barrier 43.8 µs 42.2 µs −3.7% 🟢 faster
pocl/parallel/scheduling/default 117 µs 117 µs +0.1%
pocl/parallel/scheduling/explicit 117 µs 117 µs −0.0%
pocl/parallel/stream/copy/cache 680 ns 669 ns −1.6%
pocl/parallel/stream/copy/dram 6.30 ms 6.36 ms +1.0%
pocl/parallel/stream/triad/cache 1.32 µs 1.09 µs −17.4%
pocl/parallel/stream/triad/dram 9.54 ms 9.36 ms −1.9%
pocl/parallel/util/copyto-small 25.0 ns 24.0 ns −4.0%
pocl/parallel/util/zeros-large 909 µs 935 µs +2.8%
pocl/parallel/util/zeros-small 627 ns 648 ns +3.3%
pocl/serial/arithmetic/contract 1.85 ms 1.85 ms +0.0%
pocl/serial/arithmetic/muladd 705 µs 705 µs −0.0%
pocl/serial/const/marked 668 µs 668 µs +0.0%
pocl/serial/const/unmarked 860 µs 859 µs −0.1%
pocl/serial/math/exp-f64 118 µs 126 µs +7.0%
pocl/serial/math/sin-f64 165 µs 169 µs +2.5%
pocl/serial/narrow/masked 635 µs 631 µs −0.6%
pocl/serial/narrow/unmasked 632 µs 629 µs −0.5%
pocl/serial/stream/triad/dram 8.65 ms 8.49 ms −1.9%
Latency
host/import/ka 581 ms 579 ms −0.4%
host/precompile/ka 24.3 s 24.8 s +2.1%
host/startup 129 ms 129 ms −0.1%
pocl/parallel/launch/dynamic-default 165 ns 165 ns +0.0%
pocl/parallel/launch/groupreduce-f64 810 ns 806 ns −0.5%
pocl/parallel/launch/many-args 658 ns 654 ns −0.6%
pocl/parallel/launch/partial-workgroup 218 ns 215 ns −1.4%
pocl/serial/compile/groupreduce 1.48 s 1.45 s −2.5%
pocl/serial/compile/localmem-barrier 729 ms 728 ms −0.1%
pocl/serial/compile/new-kernel 129 ms 128 ms −0.2%
pocl/serial/compile/new-workgroupsize 8.81 ms 8.83 ms +0.2%
pocl/serial/first-kernel/cold-cache 400 ms 392 ms −1.9%
pocl/serial/first-kernel/warm-cache 326 ms 316 ms −3.1%
Applications
pocl/parallel/dgsem/fluxdiff-node 251 µs 253 µs +1.1%
pocl/parallel/dgsem/weak-element 93.1 µs 93.1 µs −0.1%
pocl/parallel/dgsem/weak-node-localmem 83.2 µs 83.5 µs +0.4%
pocl/parallel/stencil/timestep 7.63 ms 7.63 ms +0.1%
pocl/serial/stencil/timestep 9.91 ms 9.78 ms −1.4%

History: https://juliagpu.org/perf/?pkg=kernelabstractions

@codecov

codecov Bot commented Nov 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.78%. Comparing base (7c56d9a) to head (c9a98ef).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #976      +/-   ##
==========================================
- Coverage   85.62%   84.78%   -0.84%     
==========================================
  Files          31       31              
  Lines        2970     2925      -45     
==========================================
- Hits         2543     2480      -63     
- Misses        427      445      +18     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@giordano

Copy link
Copy Markdown
Collaborator

Thanks! As mentioned on Slack this does fix the reproducer in #974 and help with resolving the issue we observed in Oceananigans tests, however there's also another related issue, below is a summary from the bot


There's another 8 bytes per throw site per work-item left, which only shows up when the outermost dimension of the work-group is 2. This is a variant of the reproducer from #974 which takes the work-group size as argument:

using KernelAbstractions

# a stencil with `M` bounds-checked reads, unrolled
@generated function stencil(a, i, j, k, ::Val{M}) where M
    reads = [:(s += a[i + $m, j, k]) for m in 1:M]
    return :(s = zero(eltype(a)); $(reads...); s)
end

@kernel function many_reads!(out, a, v)
    i, j, k = @index(Global, NTuple)
    @inbounds out[i, j, k] = stencil(a, i, j, k, v)
end

M = 256
workgroup = Tuple(parse.(Int, split(ARGS[1], ",")))
a = rand(64 + M, 64, 64)
out = zeros(64, 64, 64)
many_reads!(CPU(), workgroup)(out, a, Val(M); ndrange=size(out))
KernelAbstractions.synchronize(CPU())
println("OK, workgroup = $workgroup")
$ julia --project repro.jl 64,64,1
OK, workgroup = (64, 64, 1)
$ julia --project repro.jl 64,32,2
Warning: detected a stack overflow; program state may be corrupted, so further execution might be unreliable.
ERROR: LoadError: StackOverflowError:

Frame of the work-group function (sub $N, %rsp in the cached .so.o), 256 throw sites, default flags:

work-group work-items this PR per work-item result base of this PR
64×64×1 4096 200 B 0 OK 8.4 MB, overflow
32×32×4 4096 248 B 0 OK 8.4 MB, overflow
2×64×1 128 216 B 1 OK
64×2×1 128 265 KB 2067 B OK 531 KB
32×32×2 2048 4.2 MB 2050 B OK 8.4 MB, overflow
64×32×2 4096 8.4 MB 2049 B StackOverflowError 16.8 MB, segfault

32×32×3 has a 6.3 KB frame and 16×16×8 one of 264 B, so it's specifically a 2 in the outermost dimension (2×64×1 is fine). Before this PR this cost came on top of the cmpxchg slot, so these shapes needed about 16 B per site.

32×32×2 is what KernelAbstractions picks by default for an ndrange of 32×32×10, which is the case of the Oceananigans test: with this PR the frame of its largest kernel (480 throw sites) went from 12.9 MB to 6.5 MB. The test passes, but with less than 2 MB to spare on the 8 MiB stack, and a 4096-item work-group of the same shape would overflow again.

In the machine code of that kernel, PoCL reserves 788 arrays of 8 bytes per work-item. They're written only on the throw paths, with the result of the inlined claim_output (e.g. movq $0x2,(%rdx,%rsi,8) right after the lock cmpxchg loop that implements the atomic max), and the block they all jump to afterwards stores 0 in them. I couldn't find any load from them. They aren't the BoundsError index tuples, which are still in the device IR as write-only allocas: throwing a BoundsError() without arguments in our array type's getindex only reduced the frame by about 4%.

I haven't looked into PoCL to see why a trip count of 2 in the outermost work-item loop makes a difference; my guess is that the loop gets unrolled or peeled, and something per throw site that is otherwise a scalar ends up in a context array.

Julia 1.13.3, Linux x86_64, pocl_standalone_jll 7.2.3+8.

@maleadt
maleadt merged commit b81493f into main Nov 17, 2026
31 of 34 checks passed
@maleadt
maleadt deleted the tb/exception-lock-stack branch November 17, 2026 07:48
maleadt added a commit that referenced this pull request Nov 18, 2026
Every bounds check in a kernel carried a copy of the exception reporting
code, which PoCL inlines into the work-item loops. Values in that code could
get an array over the work-group, so kernels with many bounds checks needed
megabytes of stack (#974): with a work-group
whose outermost dimension is 2, e.g. 32x32x2, still 8 bytes per bounds check
per work-item after #976.

Use GPUCompiler's reporting by throw site instead: a throw only records its
site's id and work-item, through a single copy of the reporting code per
kernel, and the host looks up what was thrown. A kernel with 256 bounds
checks now has no stack frame at any work-group shape, instead of up to
4.2 MB, and 6.6 KB of code instead of 83-122 KB.

`KernelException` now names the exception, the work-item and the reason, and
at debug level 2 has a backtrace that reaches the kernel, instead of the
kernel printing them. Kernels that launch work-groups of both the full and
the bounds-checked variant look the site up in the table of the variant
that ran the work-item.
maleadt added a commit that referenced this pull request Nov 19, 2026
Every bounds check in a kernel carried a copy of the exception reporting
code, which PoCL inlines into the work-item loops. Values in that code could
get an array over the work-group, so kernels with many bounds checks needed
megabytes of stack (#974): with a work-group
whose outermost dimension is 2, e.g. 32x32x2, still 8 bytes per bounds check
per work-item after #976.

Use GPUCompiler's reporting by throw site instead: a throw only records its
site's id and work-item, through a single copy of the reporting code per
kernel, and the host looks up what was thrown. A kernel with 256 bounds
checks now has no stack frame at any work-group shape, instead of up to
4.2 MB, and 6.6 KB of code instead of 83-122 KB.

`KernelException` now names the exception, the work-item and the reason, and
at debug level 2 has a backtrace that reaches the kernel, instead of the
kernel printing them. Kernels that launch work-groups of both the full and
the bounds-checked variant look the site up in the table of the variant
that ran the work-item.
maleadt added a commit that referenced this pull request Nov 19, 2026
Every bounds check in a kernel carried a copy of the exception reporting
code, which PoCL inlines into the work-item loops. Values in that code could
get an array over the work-group, so kernels with many bounds checks needed
megabytes of stack (#974): with a work-group
whose outermost dimension is 2, e.g. 32x32x2, still 8 bytes per bounds check
per work-item after #976.

Use GPUCompiler's reporting by throw site instead: a throw only records its
site's id and work-item, through a single copy of the reporting code per
kernel, and the host looks up what was thrown. A kernel with 256 bounds
checks now has no stack frame at any work-group shape, instead of up to
4.2 MB, and 6.6 KB of code instead of 83-122 KB.

`KernelException` now names the exception, the work-item and the reason, and
at debug level 2 has a backtrace that reaches the kernel, instead of the
kernel printing them. Kernels that launch work-groups of both the full and
the bounds-checked variant look the site up in the table of the variant
that ran the work-item.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pocl The PoCL-based CPU back-end.

Projects

None yet

2 participants