Skip to content

Sched: make the cost estimate of a task with many candidate processors cheap - #791

Open
AnnaLena77 wants to merge 4 commits into
JuliaParallel:masterfrom
AnnaLena77:pr/a0c-cost-estimate
Open

AnnaLena77 wants to merge 4 commits into
JuliaParallel:masterfrom
AnnaLena77:pr/a0c-cost-estimate

Conversation

@AnnaLena77

@AnnaLena77 AnnaLena77 commented Nov 20, 2026 •

Copy link
Copy Markdown

Stacked on #790.
This branch contains the three commits of #790 (placement constants) followed by one new commit, "Sched: make the cost estimate of a task with many candidate processors cheap". Please review only that last commit. Once #790 is merged, this PR will shrink to that single commit. I'm opening it now so the review can run in parallel.

An unscoped task on a process with many threads is placed by estimate_task_costs!, which evaluated every candidate thread with dictionaries keyed by the abstractly typed Processor and a dynamic dispatch on every element. The per-task cost therefore grew with the thread count of one process: 126 µs at 8 threads and 553 µs at 64 for an empty task.

  • Costs live in a vector aligned with the candidates instead of a Dict{Processor,Float64}, so there is no key boxing per insertion and no lookup per comparison in the sort. The dictionary is only filled when the caller asks for it (estimate_task_costs, used by Datadeps and the tests); the scheduler passes nothing.
  • What is the same for every candidate is computed once: with no Chunk argument the transfer-cost table is not built, and the per-worker pressure and transfer-rate tables are looked up once per run of candidates of one worker.
  • The two per-candidate loops are function barriers specialised on the candidates' common concrete type (every thread of a process), so get_parent, root_worker_id and the keyed lookups are no longer dynamic calls. Mixed candidates keep the generic path. Equal-cost candidates are still shuffled and then stably sorted, so ties keep their random order.

Measured in one process without MPI (AMD Opteron 6272), unscoped empty task, µs per task at 8 / 16 / 32 / 64 threads:

8 16 32 64
before 125.7 158.0 251.6 552.7
after 101.0 110.0 151.0 331.2

That is -20 / -30 / -40 / -40 %. The cost still rises with the thread count (at 64 threads the scheduler lock and the dispatch dominate), but much more slowly.

test/scheduler.jl "Scheduler algorithms" (cost values, per-processor pressure, transfer rate, signatures) passes unchanged, and test/allocations.jl is unchanged. On a laptop (Julia 1.13) the Allocations, Thunk, Scheduler, Processors, Uniform Placement and Options suites pass.

AI disclosure: I prepared this change with the help of Claude (Anthropic) and reviewed it myself.
🤖 Generated with Claude Code

…per task

Under a uniform-execution acceleration (MPI) every task re-sorted all of its
candidate processors with a comparison sort whose key (processor_order_key)
formats a short_name string on every comparison, and schedule_one! first
built the candidate vector with a quadratic 'proc in input_procs' scan and
ran estimate_task_costs!, whose result the uniform selection discards.
Unscoped @Spawn therefore cost O(P log P) string constructions per task and
grew with the processor count: ~85 ms/task at 832 processors (see JuliaParallel#771).

* sort_processors_canonical!: compute the keys once per processor, then sortperm.
* compatible_processors_ordered: cache the canonical order of a candidate set
  next to the compatible set (CompatEntry), so it is built once per set.
* place_processors_uniform!: the per-task part only -- owner preference
  (binary search over the owner-sorted canonical order, three in-place
  reversals) and the thunk-id rotation. No scratch, no per-candidate boxing.
* schedule_one!: under uniform execution start from the cached order and skip
  estimate_task_costs!; otherwise drop the redundant quadratic membership
  scan (the candidates come from a Set).
* select_processors_uniform! keeps its contract (Datadeps and proc grids call
  it) and is now sort_processors_canonical! + place_processors_uniform!.

The resulting order is identical to the old algorithm's, which uniformity
across ranks depends on: test/uniform-placement.jl compares it against a copy
of the old implementation on randomized candidate sets (partial scopes, absent
preferred ranks, all rotations; 5770 assertions) and test/mpi/spawn.jl still
passes (2 ranks x 2 threads).

Measured with the new benchmark/suites/tasks.jl (suite 'tasks', ntasks=256),
4 ranks x 16 threads per node, Fulda (Opteron 6272), master vs this commit,
median of 7 samples, same suite and nodes for both:

  unscoped spawn, 64 tasks     64 procs (1 node)    244.3 ms -> 11.7 ms   (21x)
                              128 procs (2 nodes)   547.2 ms -> 12.2 ms   (45x)
                              256 procs (4 nodes)  1257.9 ms -> 14.0 ms   (90x)
  allocations of that benchmark                      0.75 M -> 13 k, 1.74 M -> 19 k, 4.0 M -> 18.7 k

The unscoped cost no longer grows with the processor count (about 0.2 ms/task
at 256 processors). Pinned (ExactScope) benchmarks have a single candidate (the old
cost estimate returned immediately for it) and are within noise; test/allocations.jl is
unchanged (task spawn 183 allocs vs a bound of 500).
do_tasks ends every enqueue by 'kicking' the doorbell of every other processor
of the scheduler, to make it try to steal. A runner whose processor may not
steal (stealing_permitted false: every MPIProcessor) wakes up, finds that out
and goes back to sleep, so under MPI each enqueued task woke ranks x threads
runners on every rank: 256 wake-ups per task at 256 processors, about 832 at
the application's size. The wake-ups are task switches on threads that could be
running tasks, plus the allocations of the runner loop.

Skip processors for which stealing_permitted is false. A kick only makes a
runner try to steal, so skipping them cannot change which task runs where.

Measured with benchmark/suites/tasks.jl on Fulda (Opteron 6272), 4 ranks x 16
threads per node, median of 7, before = 83475c6 (previous commit), 1/2/4 nodes
= 4/8/16 ranks:
  unscoped (64 tasks)              11.0 / 12.8 / 15.9 ms -> 7.3 / 9.8 / 10.4 ms
  pinned, round robin (256)        44.6 / 58.9 / 47.8 ms -> 38.3 / 39.3 / 39.8 ms
  dependent hops (64)              55.4 / 66.2 / 64.8 ms -> 41.8 / 43.5 / 42.8 ms
  fan-in tree (256 leaves)         95.5 / 98.4 / 134.2 ms -> 87.9 / 90.4 / 94.0 ms
  allocations, pinned round robin  64.6 k / 83.1 k / 67.5 k -> 40.7 k / 41.1 k / 42.9 k
  allocations per pinned task in a clean loop (4 ranks x 16 threads): 183.7 -> 159.0
Every one of the 27 benchmark/size cells is faster or equal; allocations fall by
20-65 %. No semantic change: the skipped runners would not have stolen anything.
…eping it

compat_entry inserted with BasicLFUCache.get!, which for a present key returns
the existing value. After the processor list changed without a cache flush
(here: a one-rank communicator's processors joining the context after the
world's tasks had warmed the cache) the entry for that key no longer matched
its procs, so every following lookup missed, recomputed the compatible set and
re-sorted it, and the stale entry stayed. Measured on 4 ranks x 16 threads: a
suite that registers the extra processors mid-run made unscoped tasks 5.3x
slower with 10x the allocations (7.5 -> 40.1 ms per 64 tasks); without the
registration they stay at 1.15x. Replace the entry on a miss.
…s cheap

An unscoped task on a process with many threads is placed by estimate_task_costs!,
which evaluated every candidate thread with dictionaries keyed by the abstractly
typed Processor and a dynamic dispatch on every element. The per-task cost grew
with the thread count (the maintainers' MPI notes list 'per-task cost grows with the
thread count of one process' as an unexplained observation): 126 us at 8 threads
and 553 us at 64 on the Fulda nodes, for an empty task, in a single process.

* Costs live in a vector aligned with the candidates instead of a
  Dict{Processor,Float64}: boxing a key per insertion and a lookup per comparison
  in the sort are gone. The dictionary is only filled when the caller asks for it
  (estimate_task_costs, used by Datadeps and the tests); the scheduler passes nothing.
* What is the same for every candidate is computed once: with no Chunk argument the
  transfer-cost table is not built at all, and the per-worker pressure and
  transfer-rate tables are looked up once per run of candidates of one worker.
* The two per-candidate loops are function barriers specialised on the candidates'
  common concrete type (every thread of a process), so get_parent, root_worker_id and
  the keyed lookups are no longer dynamic calls. Mixed candidates keep the generic
  path. Equal-cost candidates are still shuffled and then stably sorted, so ties
  keep their random order as before.

Measured in one process without MPI on Fulda (Opteron 6272), unscoped empty task,
us per task, 8 / 16 / 32 / 64 threads:
  before   125.7 / 158.0 / 251.6 / 552.7
  after    101.0 / 110.0 / 151.0 / 331.2   (-20 % / -30 % / -40 % / -40 %)
The cost still rises with the thread count (at 64 threads the scheduler lock and
the dispatch of the task dominate), but much more slowly.

test/scheduler.jl 'Scheduler algorithms' (cost values, per-processor pressure,
per-processor transfer rate, signatures; 82 assertions) passes unchanged and
test/allocations.jl is unchanged (task spawn 183 allocations, bound 500).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant