feat: LLVM lowering for borrow arrays using bitmasks #2574

mark-koch · 2025-09-22T16:45:18Z

Substantial duplication between existing hugr-llvm/src/extension/collections/array.rs and new ...borrow_array.rs.

codecov · 2025-09-22T16:50:12Z

Codecov Report

❌ Patch coverage is 91.66667% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.32%. Comparing base (cd04cf8) to head (14dced7).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
hugr-llvm/src/emit/func.rs	91.48%	1 Missing and 3 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2574      +/-   ##
==========================================
+ Coverage   83.05%   83.32%   +0.27%     
==========================================
  Files         255      256       +1     
  Lines       48480    50491    +2011     
  Branches    44003    46014    +2011     
==========================================
+ Hits        40265    42074    +1809     
+ Misses       6123     6065      -58     
- Partials     2092     2352     +260

Flag	Coverage Δ
python	`91.53% <ø> (ø)`
rust	`82.53% <91.66%> (+0.33%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

acl-cqc · 2025-09-24T12:30:33Z

hugr-llvm/src/emit/func.rs

+    let func = ctx
+        .get_current_module()
+        .get_function(func_name)
+        .map(Ok::<_, anyhow::Error>)


Nit: Option of Result, etc., is a bit hard to figure out. How about let func = match ctx.get_current_module().get_function(func_name) { Some(f) => f, None => { .... } } which moves your closure into the body and removes the nested ?.

Or perhaps consider let func = if let Some(f) = ctx.blah() {f} else {....}

tatiana-s · 2025-09-24T12:41:53Z

hugr-llvm/src/extension/collections/borrow_array.rs

+}
+
+impl<'a, H: HugrView<Node = Node> + 'a> CodegenExtsBuilder<'a, H> {
+    /// Add a [`BorrowArrayCodegenExtension`] to the given [`CodegenExtsBuilder`] using `ccg`


nit: comments should be the other way round?

acl-cqc

Well I thought I'd take a look at this, but +4,000 lines and I'm stopping a fraction of the way through.... the PR we are thinking this replaces is a third as long as this, right? How many opportunities are there for bugs here? (ReplaceTypes BorrowArray has the benefit that we get type-checking of the output Hugr by the hugr type system, too)

acl-cqc · 2025-09-24T12:31:27Z

hugr-llvm/src/emit/func.rs

+/// The first time this helper is called with a given function name, a function is built
+/// using the provided closure. Future invocations with the same name will just emit calls
+/// to this function.
+pub fn outline_into_function<'c, H: HugrView<Node = Node>, const N: usize>(


Perhaps get_or_make_function

acl-cqc · 2025-09-24T12:33:00Z

hugr-llvm/src/extension/collections/borrow_array.rs

+//! to at least `n * sizeof(T)` bytes. The extra `usize` is an offset pointing to the
+//! first element, i.e. the first element is at address `ptr + offset * sizeof(T)`.
+//!
+//! The rational behind the additional offset is the `pop_left` operation which bumps


I like this explanation, but is there no push_left / unpop ?

Also, rational -> rationale

is there no push_left / unpop ?

No, as this would require reallocating the array in general

acl-cqc · 2025-09-24T12:33:29Z

hugr-llvm/src/extension/collections/borrow_array.rs

+//! the offset instead of mutating the pointer. This way, we can still free the original
+//! pointer when the array is discarded after a pop.
+//!
+//! We provide utility functions [`array_fat_pointer_ty`], [`build_barray_fat_pointer`], and


Why are some of these barray and others just array ?

Because I forgot to rename those after copy pasting 😅

hugr-llvm/src/extension/collections/borrow_array.rs

tatiana-s · 2025-09-25T09:49:08Z

Well I thought I'd take a look at this, but +4,000 lines and I'm stopping a fraction of the way through.... the PR we are thinking this replaces is a third as long as this, right? How many opportunities are there for bugs here? (ReplaceTypes BorrowArray has the benefit that we get type-checking of the output Hugr by the hugr type system, too)

But most of this PR just copies the existing array lowering (including many snapshots) so it is only the bit mask code that is new which is a lot less than the type replacement

ss2165 · 2025-10-06T11:28:03Z

~~Please include #2609 before merging~~ done

…putation

…moning up

… MaskCheck)

acl-cqc · 2025-10-08T10:01:04Z

Ok I guess this is now a joint PR ;-), hopefully this link shows just the changes I've made to Mark's original: https://github.com/CQCL/hugr/pull/2574/files/439f09c27e6591c125088000e03d8556b88fef2b..6cf427bee12c8f65fa35e3374cf4b47c5c9dd9ed

ss2165

Just some minor docstring/refactor suggestions

ss2165 · 2025-10-08T10:21:36Z

hugr-llvm/src/extension/collections/borrow_array.rs

-        go(ctx, block)
-    })
+#[derive(Clone, Copy, Debug, PartialEq, Eq)]
+enum MaskCheck {


docstrings on these would be good

ss2165 · 2025-10-08T10:22:02Z

hugr-llvm/src/extension/collections/borrow_array.rs


-/// Emits instructions to flip the borrowed mask flag for a given index.
-fn build_mask_flip<'c, H: HugrView<Node = Node>>(
+fn check_mask_elem<'c, H: HugrView<Node = Node>>(


docstring 🙏

this would seem to be a natural method on MaskCheck

:-D agreed, I was just thinking that myself. Done...

## 🤖 New release * `hugr-model`: 0.23.0 -> 0.24.0 * `hugr-core`: 0.23.0 -> 0.24.0 (✓ API compatible changes) * `hugr-llvm`: 0.23.0 -> 0.24.0 (✓ API compatible changes) * `hugr-passes`: 0.23.0 -> 0.24.0 (✓ API compatible changes) * `hugr-persistent`: 0.3.0 -> 0.3.1 (✓ API compatible changes) * `hugr`: 0.23.0 -> 0.24.0 (✓ API compatible changes) * `hugr-cli`: 0.23.0 -> 0.24.0 (✓ API compatible changes) <details><summary>Changelog</summary> ## `hugr-model` <blockquote> ## [0.23.0](hugr-model-v0.22.4...hugr-model-v0.23.0) - 2025-09-30 ### Bug Fixes - [**breaking**] Appease `cargo-audit` by replacing unmaintained dependencies ([#2572](#2572)) ### New Features - Documentation and error hints ([#2523](#2523)) </blockquote> ## `hugr-core` <blockquote> ## [0.24.0](hugr-core-v0.23.0...hugr-core-v0.24.0) - 2025-10-13 ### Bug Fixes - Preserve offset for CFG edges when serializing to JSON ([#2606](#2606)) ### New Features - LLVM lowering for borrow arrays using bitmasks ([#2574](#2574)) - *(py, core, llvm)* add `is_borrowed` op for BorrowArray ([#2610](#2610)) ### Refactor - [**breaking**] consistent inout order in borrow array ([#2621](#2621)) </blockquote> ## `hugr-llvm` <blockquote> ## [0.24.0](hugr-llvm-v0.23.0...hugr-llvm-v0.24.0) - 2025-10-13 ### New Features - LLVM lowering for borrow arrays using bitmasks ([#2574](#2574)) - *(py, core, llvm)* add `is_borrowed` op for BorrowArray ([#2610](#2610)) ### Refactor - [**breaking**] consistent inout order in borrow array ([#2621](#2621)) </blockquote> ## `hugr-passes` <blockquote> ## [0.24.0](hugr-passes-v0.23.0...hugr-passes-v0.24.0) - 2025-10-13 ### New Features - Add handler for copying / discarding borrow arrays to default lineariser ([#2602](#2602)) </blockquote> ## `hugr-persistent` <blockquote> ## [0.3.1](hugr-persistent-v0.3.0...hugr-persistent-v0.3.1) - 2025-10-13 ### Bug Fixes - *(test)* No extension serialisation in persistent-hugr testing ([#2612](#2612)) ### New Features - *(persistent)* Redesign CommitStateSpace, bound Commit lifetime ([#2534](#2534)) </blockquote> ## `hugr` <blockquote> ## [0.24.0](hugr-v0.23.0...hugr-v0.24.0) - 2025-10-13 ### Bug Fixes - Preserve offset for CFG edges when serializing to JSON ([#2606](#2606)) ### New Features - Add handler for copying / discarding borrow arrays to default lineariser ([#2602](#2602)) - LLVM lowering for borrow arrays using bitmasks ([#2574](#2574)) - *(py, core, llvm)* add `is_borrowed` op for BorrowArray ([#2610](#2610)) ### Refactor - [**breaking**] consistent inout order in borrow array ([#2621](#2621)) </blockquote> ## `hugr-cli` <blockquote> ## [0.24.0](hugr-cli-v0.23.0...hugr-cli-v0.24.0) - 2025-10-13 ### Documentation - update cli readme ([#2601](#2601)) </blockquote> </details> --- This PR was generated with [release-plz](https://github.com/release-plz/release-plz/).

## 🤖 New release * `hugr-model`: 0.24.0 -> 0.24.1 (✓ API compatible changes) * `hugr-core`: 0.24.0 -> 0.24.1 (✓ API compatible changes) * `hugr-llvm`: 0.24.0 -> 0.24.1 (✓ API compatible changes) * `hugr-passes`: 0.24.0 -> 0.24.1 (✓ API compatible changes) * `hugr-persistent`: 0.3.1 -> 0.3.2 (✓ API compatible changes) * `hugr`: 0.24.0 -> 0.24.1 (✓ API compatible changes) * `hugr-cli`: 0.24.0 -> 0.24.1 (✓ API compatible changes) <details><summary>Changelog</summary> ## `hugr-model` <blockquote> ## [0.24.1](hugr-model-v0.24.0...hugr-model-v0.24.1) - 2025-11-03 ### Bug Fixes - Correct conversion of `table::Term::Tuple` to `ast::Term` ([#2653](#2653)) </blockquote> ## `hugr-core` <blockquote> ## [0.24.1](hugr-core-v0.24.0...hugr-core-v0.24.1) - 2025-11-03 ### Bug Fixes - validation outside entrypoint, normalize_cfgs w/ nonlocal edges ([#2633](#2633)) - SiblingSubgraph::try_from_nodes for non-entrypoint region ([#2655](#2655)) - Correct conversion of `table::Term::Tuple` to `ast::Term` ([#2653](#2653)) ### New Features - track package descriptions when loading ([#2639](#2639)) - *(cli)* describe sub-command ([#2650](#2650)) </blockquote> ## `hugr-llvm` <blockquote> ## [0.24.0](hugr-llvm-v0.23.0...hugr-llvm-v0.24.0) - 2025-10-13 ### New Features - LLVM lowering for borrow arrays using bitmasks ([#2574](#2574)) - *(py, core, llvm)* add `is_borrowed` op for BorrowArray ([#2610](#2610)) ### Refactor - [**breaking**] consistent inout order in borrow array ([#2621](#2621)) </blockquote> ## `hugr-passes` <blockquote> ## [0.24.1](hugr-passes-v0.24.0...hugr-passes-v0.24.1) - 2025-11-03 ### Bug Fixes - validation outside entrypoint, normalize_cfgs w/ nonlocal edges ([#2633](#2633)) </blockquote> ## `hugr-persistent` <blockquote> ## [0.3.2](hugr-persistent-v0.3.1...hugr-persistent-v0.3.2) - 2025-11-03 ### New Features - *(persistent)* More efficient HugrView iterators for PersistentHugr ([#2595](#2595)) </blockquote> ## `hugr` <blockquote> ## [0.24.1](hugr-v0.24.0...hugr-v0.24.1) - 2025-11-03 ### Bug Fixes - validation outside entrypoint, normalize_cfgs w/ nonlocal edges ([#2633](#2633)) - SiblingSubgraph::try_from_nodes for non-entrypoint region ([#2655](#2655)) - Correct conversion of `table::Term::Tuple` to `ast::Term` ([#2653](#2653)) ### New Features - track package descriptions when loading ([#2639](#2639)) - *(cli)* describe sub-command ([#2650](#2650)) </blockquote> ## `hugr-cli` <blockquote> ## [0.24.1](hugr-cli-v0.24.0...hugr-cli-v0.24.1) - 2025-11-03 ### New Features - track package descriptions when loading ([#2639](#2639)) - *(cli)* describe sub-command ([#2650](#2650)) </blockquote> </details> --- This PR was generated with [release-plz](https://github.com/release-plz/release-plz/).

## 🤖 New release * `hugr-model`: 0.24.1 -> 0.24.2 * `hugr-core`: 0.24.1 -> 0.24.2 * `hugr-llvm`: 0.24.1 -> 0.24.2 * `hugr-passes`: 0.24.1 -> 0.24.2 (✓ API compatible changes) * `hugr`: 0.24.1 -> 0.24.2 (✓ API compatible changes) * `hugr-cli`: 0.24.1 -> 0.24.2 (✓ API compatible changes) * `hugr-persistent`: 0.3.2 -> 0.3.3 <details><summary>Changelog</summary> ## `hugr-model` <blockquote> ## [0.24.1](hugr-model-v0.24.0...hugr-model-v0.24.1) - 2025-11-03 ### Bug Fixes - Correct conversion of `table::Term::Tuple` to `ast::Term` ([#2653](#2653)) </blockquote> ## `hugr-core` <blockquote> ## [0.24.1](hugr-core-v0.24.0...hugr-core-v0.24.1) - 2025-11-03 ### Bug Fixes - validation outside entrypoint, normalize_cfgs w/ nonlocal edges ([#2633](#2633)) - SiblingSubgraph::try_from_nodes for non-entrypoint region ([#2655](#2655)) - Correct conversion of `table::Term::Tuple` to `ast::Term` ([#2653](#2653)) ### New Features - track package descriptions when loading ([#2639](#2639)) - *(cli)* describe sub-command ([#2650](#2650)) </blockquote> ## `hugr-llvm` <blockquote> ## [0.24.0](hugr-llvm-v0.23.0...hugr-llvm-v0.24.0) - 2025-10-13 ### New Features - LLVM lowering for borrow arrays using bitmasks ([#2574](#2574)) - *(py, core, llvm)* add `is_borrowed` op for BorrowArray ([#2610](#2610)) ### Refactor - [**breaking**] consistent inout order in borrow array ([#2621](#2621)) </blockquote> ## `hugr-passes` <blockquote> ## [0.24.2](hugr-passes-v0.24.1...hugr-passes-v0.24.2) - 2025-11-03 ### Bug Fixes - ReplaceTypes: operate on whole Hugr, with set_regions ([#2662](#2662)) </blockquote> ## `hugr` <blockquote> ## [0.24.2](hugr-v0.24.1...hugr-v0.24.2) - 2025-11-03 ### Bug Fixes - ReplaceTypes: operate on whole Hugr, with set_regions ([#2662](#2662)) </blockquote> ## `hugr-cli` <blockquote> ## [0.24.1](hugr-cli-v0.24.0...hugr-cli-v0.24.1) - 2025-11-03 ### New Features - track package descriptions when loading ([#2639](#2639)) - *(cli)* describe sub-command ([#2650](#2650)) </blockquote> ## `hugr-persistent` <blockquote> ## [0.3.2](hugr-persistent-v0.3.1...hugr-persistent-v0.3.2) - 2025-11-03 ### New Features - *(persistent)* More efficient HugrView iterators for PersistentHugr ([#2595](#2595)) </blockquote> </details> --- This PR was generated with [release-plz](https://github.com/release-plz/release-plz/).

mark-koch requested a review from tatiana-s September 22, 2025 16:45

mark-koch requested a review from a team as a code owner September 22, 2025 16:45

mark-koch requested review from acl-cqc and removed request for a team and acl-cqc September 22, 2025 16:45

acl-cqc reviewed Sep 24, 2025

View reviewed changes

tatiana-s reviewed Sep 24, 2025

View reviewed changes

acl-cqc reviewed Sep 24, 2025

View reviewed changes

mark-koch force-pushed the feat/barray-llvm branch 4 times, most recently from d82bb30 to ba4df3d Compare September 29, 2025 14:05

Base automatically changed from test/panic to main September 29, 2025 15:46

mark-koch requested review from acl-cqc and tatiana-s September 30, 2025 08:57

mark-koch and others added 11 commits October 6, 2025 15:21

Make copy of existing array lowering

bdc4795

Rename everything

5bc67e3

Implement bitmask lowering

fe5d575

Handle 0 sized arrays

b23b593

Clippy

b3d5b8e

Stop using lazy_static

877771c

Fix docs

924a6d3

Refactor and rename outline_into_function

2d19e2f

Rename ERR_NOT_FREE to ERR_NOT_BORROWED

cb58a3a

Add codegen for conversion ops

56eee57

refactor: allow non-void get_or_make_function (#2609)

439f09c

acl-cqc force-pushed the feat/barray-llvm branch from 91f42b2 to 439f09c Compare October 6, 2025 14:22

fix: reorder loads/stores in build_mask_padding

d5d5999

acl-cqc force-pushed the feat/barray-llvm branch from 5be9249 to d5d5999 Compare October 7, 2025 14:59

acl-cqc added 10 commits October 7, 2025 21:20

Fix accessing/padding beyond end of mask

4a64ae4

trivial refactor: pass last_idx into build_mask_padding, avoids recom…

6c45cbf

…putation

refactor: split build_mask_padding into two calls

e619155

refactor: build_mask_padding1d tests value before direction, some com…

bf4b6d6

…moning up

build_barray_alloc: do not return mask_ptr

b056e4d

build_barray_fat_pointer takes BArrayFatPointerComponents

b80bc0e

build_(idx_(free,not_borrowed)_check,mask_flip) -> check_mask_eq(enum…

5ca60ce

… MaskCheck)

Avoid xor in mask-check methods by flipping bbs; update more snapshots

dc99387

common up array allocation w/bit-cast

c2cdf79

check_mask_idx -> check_mask_elem, does not check the index is correct

6cf427b

acl-cqc requested a review from ss2165 October 8, 2025 11:01

ss2165 approved these changes Oct 8, 2025

View reviewed changes

acl-cqc added 4 commits October 8, 2025 13:45

refactor: Make check_mask_elem into MaskCheck.emit

e5a364a

refactor: rename CheckPresent -> CheckNotBorrowed

b7e50ab

doc MaskCheck

5aaba8f

Merge remote-tracking branch 'origin/main' into feat/barray-llvm

14dced7

acl-cqc enabled auto-merge October 8, 2025 12:57

acl-cqc added this pull request to the merge queue Oct 8, 2025

Merged via the queue into main with commit 4379a0f Oct 8, 2025
27 checks passed

acl-cqc deleted the feat/barray-llvm branch October 8, 2025 13:08

hugrbot mentioned this pull request Oct 7, 2025

chore: release #2605

Merged

hugrbot mentioned this pull request Oct 14, 2025

chore: release #2625

Merged

hugrbot mentioned this pull request Nov 3, 2025

chore: release #2661

Merged

feat: LLVM lowering for borrow arrays using bitmasks #2574

feat: LLVM lowering for borrow arrays using bitmasks #2574

Uh oh!

Conversation

mark-koch commented Sep 22, 2025 • edited by acl-cqc Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

acl-cqc left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tatiana-s commented Sep 25, 2025

Uh oh!

ss2165 commented Oct 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

acl-cqc commented Oct 8, 2025

Uh oh!

ss2165 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

mark-koch commented Sep 22, 2025 •

edited by acl-cqc

Loading

codecov bot commented Sep 22, 2025 •

edited

Loading

ss2165 commented Oct 6, 2025 •

edited

Loading