Skip to content

Report exceptions by throw site, for back-ends that opt in - #1100

Merged
maleadt merged 4 commits into
mainfrom
tb/exception-sites
Nov 17, 2026
Merged

maleadt merged 4 commits into
mainfrom
tb/exception-sites

Conversation

@maleadt

@maleadt maleadt commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Every throw site currently reports its exception with a sequence of runtime calls (report_exception, at debug level 2 a report_exception_frame per frame, signal_exception). On SPIR-V and Metal, throwing code is inlined into kernels, so a kernel with many bounds checks carries many copies of that sequence, and that is expensive. With 256 bounds checks per thread, compared to @inbounds:

Back end Run time today
Metal (M1) 6.8x
oneAPI (Iris Xe) 5.2x
AMDGPU (gfx1036 iGPU) 15x
CUDA (RTX 5080) 1.17x

On PoCL, values in the inlined code get an array over the work-group, which made KernelAbstractions' CPU kernels overflow the stack (JuliaGPU/KernelAbstractions.jl#974).

This adds an opt-in alternative, without changing anything for back-ends that don't opt in:

  • uses_exception_sites(job) (default false). An opted-in back-end implements one runtime function, exception_site(id::Int32), instead of report_exception, report_exception_name, report_exception_frame and signal_exception.
  • A throw site passes only the id of the site. Compilation returns the table mapping ids to the exception's name, reason and backtrace as meta.exception_sites (a Vector{ExceptionSite}, plain data, so it can be stored with cached results). Id 0 means nothing is known (debug level 0).
  • Until the module is final, ids are symbolic: the reporting call takes the result of an intrinsic whose operands are the name and reason. They survive caching of the runtime library's bitcode, deferred compilation and linking. finalize_exception_sites! then numbers them, deriving the backtrace at debug level 2 from the reporting call's location after inlining. Where throwing code is inlined into kernels (SPIR-V, and now early in finish_ir! for opted-in Metal jobs), backtraces therefore reach the kernel.
  • merge_exception_sites! makes every function branch to a single reporting call, with a phi of ids, and inlines it.
  • GPUCompiler.@gputhrow name reason (not exported) names a throw's exception and reason, for back-ends' overrides.
  • Calls rewritten to pass the kernel state or Metal's kernel inputs keep noinline and nomerge, which keep a report out of line until it's finalized.

The old runtime functions stay, and remain the default; they can go in the next breaking release.

Two fixes this needed:

  • The runtime library was linked a second time after GC lowering, which introduces calls to gc_pool_alloc. By then the runtime functions already in the module were internal, so the linker imported fresh external copies of everything gc_pool_alloc calls and renamed the originals. Every kernel that allocates and throws carried two copies of the reporting functions, which ended up exported in the PTX, GCN and SPIR-V output; with exception sites the kernel's ids went to the renamed copy and could not be resolved. gc_pool_alloc is now linked with the rest of the runtime when the module allocates, kept external during optimization, and internalized after GC lowering, so nothing is linked during optimization any more.
  • On GCN, lower_throw_extra! replaced calls to throw_boundserror and similar functions before back-ends' overrides were inlined, so a @gputhrow in an override got named after the mangled function.

KernelAbstractions' PoCL back-end is the first user (JuliaGPU/KernelAbstractions.jl, branch tb/pocl-exception-sites): a kernel with 256 bounds checks goes from up to 4.2 MB of stack to no frame, and from 83-122 KB of code to 6.6 KB. Prototypes for Metal, oneAPI and AMDGPU showed run time going from 6.8x to 1.4-1.8x on Metal, and from 5.7x to 3.0x on oneAPI.

Tested on Julia 1.13 (full suite) and 1.10 (ptx, spirv, metal, gcn and native; the precompile tests fail there on master too).

GC lowering introduces calls to `gc_pool_alloc` during optimization, so
the runtime library was linked a second time after it. By then the runtime
functions already in the module were internal, so the linker imported fresh
external copies of everything `gc_pool_alloc` calls and renamed the
originals: kernels that allocate and throw carried two copies of the
reporting functions, and the relinked functions were never internalized,
ending up as exported functions in the PTX, GCN and SPIR-V output.

Instead, link `gc_pool_alloc` with the rest of the runtime when the module
allocates, keep it external (and thus alive and with its ABI intact) during
optimization, and internalize it right after GC lowering.
Every throw site reports its exception with a sequence of runtime calls that
pass the exception's name, and at debug level 2 every frame of its backtrace.
On SPIR-V and Metal, throwing code is inlined into kernels, so a kernel with
many bounds checks carries many copies of that sequence. That costs run time
(Metal and oneAPI kernels with 256 bounds checks run 5-7x slower than
without), code size, and on PoCL stack: per-site values in the inlined code
get an array over the work-group (JuliaGPU/KernelAbstractions.jl#974).

With `uses_exception_sites`, a throw site instead passes an `Int32` id to the
`exception_site` runtime function, and compilation returns the table that
maps ids to the exception's name, reason and backtrace as
`meta.exception_sites`. The ids stay symbolic, as an intrinsic taking the name
and reason, until the module is final, which keeps them valid through
caching of the runtime library, deferred compilation and linking. Every
function then gets a single call that its throw sites branch to, which is
inlined. `GPUCompiler.@gputhrow` lets back-ends name an exception and its
reason. Backtraces are derived after inlining, so where throwing code is
inlined into kernels (SPIR-V, Metal) they reach the kernel.

Back-ends that don't opt in are unaffected. The other runtime functions will
be removed in the next breaking release.
`lower_throw_extra!` replaces calls to functions such as
`julia_throw_boundserror` by a report named after the function, before
back-ends' overrides of those functions are inlined. When an override names
its exception with `@gputhrow`, that replaced its name and reason by the
mangled function name, so skip functions that report a site themselves.
@maleadt
maleadt added this pull request to stack #1101 November 17, 2026 15:49
@maleadt
maleadt merged commit 672469a into main Nov 17, 2026
34 of 36 checks passed
@maleadt
maleadt deleted the tb/exception-sites branch November 17, 2026 20:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

metal Stuff about the Apple metal back-end. spirv Stuff about the SPIR-V back-end.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant