Broadcast Kernel Fusion - #119
Merged
Merged
Conversation
ejmeitz
marked this pull request as draft
April 24, 2026 18:14
Co-authored-by: Copilot <copilot@github.com>
…cript to set legate config and call worker) -> benchmark/single.jl (worker to call benchmark)
…idual files in src/benchmarks
Align fused Number leaves with unfused promotion, refuse dense-linear fusion on slice views, and add a Gray-Scott CPU-reference correctness check in the benchmark harness. Co-authored-by: Cursor <cursoragent@cursor.com>
Pack Legate element strides for RunPTXBroadcastTask so Gray-Scott-style views fuse correctly, and materialize Array(NDArray) with a C-order transpose so CPU comparisons match logical indexing. Co-authored-by: Cursor <cursoragent@cursor.com>
Convert N≥2 Arrays via permutedims before attach so NDArray layout matches C-order stores, and update lifetime tests accordingly. Co-authored-by: Cursor <cursoragent@cursor.com>
Path filters already decide when the workflow starts; the old if-gate skipped the docs job on PRs whose branch history touched the C++ wrapper.
krasow
marked this pull request as ready for review
July 27, 2026 01:28
ejmeitz
added a commit
that referenced
this pull request
Aug 12, 2026
* Allow for unary reductions with arbitrary dims (#120) Co-authored-by: krasow <krasow@u.northwestern.edu> * adjust CI pipelines * Fix developer build. * Fix memory leak (#136) * fix memory leak with cxxwrap * fix get_ptr missing modifcations for get_store changes * Update versioninfo() output. Remove the storage of .githash (#138) * update versioninfo() output. Remove the storage of .githash * cpu ci patch. Has the ability to use a specific legate branch by comment within PR. However, if any wrapper code is changed in the Legate branch && there is no released wrapper- tests will fail on purpose. This signifies a new wrapper needs to be released. * add script to check versions for PRs into main (#139) * bump version and try guards if they fail resolving * add registry update? * add General registry? * cleanup /dev * Build refactor (#137) * Linear algebra type stability enhancements and matrix solve support (#132) --------- Co-authored-by: krasow <krasow@u.northwestern.edu> * disable Float16 in supported float types. Left as a TODO. * skip Int8 unary reduction (with dims) tests on minimum and maximum on GPU. (#141) * Fix infinite loop on high memory pressure. (#145) * update GC memory.jl to only trigger if memory pressure is actually changing between GC iterations. * Better Benchmark Harness (#140) --------- Co-authored-by: krasow <krasow@u.northwestern.edu> * Update Buildkite pipeline for the new JuliaGPU cluster. (#147) Switch to the per-backend queues of the new JuliaGPU cluster. Co-authored-by: David Krasowska <krasowska89@gmail.com> * Broadcast Kernel Fusion (#119) Broadcast fusion support, layout transformations, memory leak patches, new documentation, pretty printers for lifetime analysis and broadcast fusion kernels. --------- Co-authored-by: krasow <krasow@u.northwestern.edu> * Hdf5 impl (#129) --------- Co-authored-by: krasow <krasow@u.northwestern.edu> * SVD and QR linalg operators (#143) --------- Co-authored-by: krasow <krasow@u.northwestern.edu> * Update nda_get_slice impl to remove std::vector alloc (#151) * nda_move destruction changes (#153) * Remove uncessary size checks for launching fused broadcast exprs (#152) * Remove complex type mappings for Julia (#159) * Remove complex type mappings for Julia Removed mapping for C++ complex types to Julia complex types. * add accessor tests * retrigger * retrigger --------- Co-authored-by: David Krasowska <krasowska89@gmail.com> * Buildkite CI Filtering (#162) * patch ci: compare against main since main has published JLL * patch ci: CPU JLL CI is triggering when the PR has no wrapper changes, but there is a difference between main and the PR * Cartesian indexing for broadcast fusion (#156) * Inter-broadcast expr kernel fusion and MacroTools refactor (#154) * inter-broadcast kernel fusion. Leverage lifetime analysis to rewrite expr blocks to maximize fusion and reducing temps * Refactor lifetime analysis w/ MacroTools. (#157) * encapsulate scoped expr in a let block * lifetime scoping: scalars don't need to become hoisted temps. And updated pretty printers * hints for undef * add documentation about our inter-broadcast fusion anti pattern regarding pre-allocations * handle dot macro expansion properly * materialize scalar broadcasts to fix test case: @analyze_lifetimes begin tmp = @. A + B result .= @. tmp * multiplier + T(1.0) end * 26.06 changes (#146) * VERSION mismatch patches * Downgrade CNPreferences version Downgrade CNPreferences from version 0.1.3 to 0.1.2. * Revert "Downgrade CNPreferences version" This reverts commit 11b21c5. * CI patches: fix sources in test Project.toml and update buildkite JLL caching conflicts * new finalizer for 1.12 * add same caching mech to developer pipeline * Revert "new finalizer for 1.12" This reverts commit d23f819. * Reduction tolerances scaling w/ N (#169) * Fix 1.12 finalizer threading issues w/ queue read by main thread (#168) * update to 0.2.1 legate and rm build caches for wrappers * PTX Compat Selection (#167) * update cache again. * Retrigger CI * rm some compile cache * timeout to 90 * user-decision lifetime decisions in our scoping analysis (#170) * Fix return args in user-defined lifetime macro (#171) * Use ParallelTestRunner (#173) --------- Co-authored-by: David Krasowska <krasowska89@gmail.com> --------- Co-authored-by: Nader <107228500+Nader-Rahhal@users.noreply.github.com> Co-authored-by: Ethan Meitz <54505069+ejmeitz@users.noreply.github.com> Co-authored-by: Tim Besard <tim.besard@gmail.com>
krasow
added a commit
that referenced
this pull request
Aug 12, 2026
- Added HDF5 dataset reading and writing (#129). - Added broadcast kernel fusion with layout transformations, Cartesian indexing, and MacroTools-based lifetime analysis with pretty printers (#119, #154, #156, #157). - Added user-controlled lifetime decisions in scoping analysis (#170, #171). - Added SVD and QR linear-algebra operators (#143). - Added matrix solve support and linear-algebra type-stability improvements (#132). - Added unary reductions over arbitrary dimensions (#120). - Fixed memory leaks, slice/move allocations, and unnecessary broadcast size checks (#136, #151, #152, #153). - Fixed infinite loop under high memory pressure and finalizer threading issues on Julia 1.12 (#145, #168). - Removed complex type mappings for Julia (#159). - Improved reduction tolerance scaling and skipped unstable Int8 reduction tests (#141, #169). - Improved developer builds, benchmark harness, versioninfo() output, and PR version validation (#137, #138, #139, #140). - Improved Buildkite CI for the new JuliaGPU cluster, PR filtering, and ParallelTestRunner (#147, #162, #173). - Updated compatibility to Legate 26.06 and added PTX compat selection (#146, #167). - Additional CI adjustments, developer-build fixes, and Float16 handling landed as direct commits without PR numbers. --------- Co-authored-by: Nader <107228500+Nader-Rahhal@users.noreply.github.com> Co-authored-by: Ethan Meitz <54505069+ejmeitz@users.noreply.github.com> Co-authored-by: Tim Besard <tim.besard@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
requires legate_branch: develop
This PR is implements the automatic fusion of broadcast expressions with unary & binary operations. Really any intrinsic you could use in a CUDA kernel.
Goals
NDArraysthat leverage any binary/unary op and may be combined with some number of scalar arguments.High Level Implementation
NDArraysasCuDeviceArraysLoadPTXTaskandRunPTXTasklegate tasks defined in cuda.cpp with the registered kernelThis PR does not aim to support fusion of functions. That will be a future PR.