feat(shape-inference): container types inc2 — sequence mutation + tensor⇔sequence conversion (#449)

[justinchuby]

Increment 2 of the #449 container-type roadmap. Foundation (PR #477, the additive ValueType layer + SequenceEmpty/Construct/Length/At) is merged to main; this builds on it. refs #449.

Ops added (domain "", opset 11, handlers/container.rs)

Sequence mutation:

• SequenceInsert (seq, tensor, [position]) → seq — element type unifies existing element ⨝ inserted tensor (dtype-mismatch → error, extent disagreement → fresh symbol, dtype-only/rank-mismatch → unknown shape). position is type-irrelevant.
• SequenceErase (seq, [position]) → seq — element type preserved.

Tensor⇔sequence conversion:

• SplitToSequence (tensor, [split], axis, keepdims) → seq — element dtype = input dtype; explicit split → split-axis symbolic; no split → chunk size 1 with keepdims deciding keep-at-1 vs remove-axis. Matches runtime sequence/split.rs.
• ConcatFromSequence (seq, axis, new_axis) → tensor — the sequence→tensor direction of the seam. Recovers a tensor TypeInfo from the element type: concat axis symbolic; new_axis=1 inserts a symbolic stack dim (rank +1); axis mandatory. dtype-only element → unresolved (unknown rank), same as SequenceAt.

DRY

• Generalised the foundation's merge_element → merge_tensor(ctx, op, acc, other), now shared by SequenceConstruct + SequenceInsert; merge_shape reused unchanged.
• New sequence_element_tensor(ctx, i) reader shared by Insert/Concat.
• handlers::checked_axis reused; output_rank = rank + new_axis makes one axis check cover concat and stack.

Guarantees

• Tensor path byte-identical — tensor_only_path_is_byte_identical_after_container_type_model unchanged & GREEN. No tensor handler touches the container layer.
• Catalog 213→217 ops / 258→262 entries (pinned test updated; no phantom decls).
• Tests +21 (op_rules 254→275): dtype/shape asserts, symbolic-dim preservation, dtype/rank-mismatch error/degrade, and round-trips — Construct→Insert→At and SplitToSequence→ConcatFromSequence recovers a rank-2 f32 tensor.
• cargo fmt --check, clippy -D warnings clean; onnx-runtime-session/-eager build.

Scoped OUT → inc3+

SequenceMap (needs subgraph/body type-threading = container-aware control flow), plus Optional/Map op rules. See .squad/decisions/inbox/harry-449-inc2.md.

CPU-only, pure shape-inference crate work. Draft — do not merge.

[codecov]

Codecov Report

❌ Patch coverage is 87.93103% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.58%. Comparing base (e2b7565) to head (5d1b256).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...-runtime-shape-inference/src/handlers/container.rs	87.93%	12 Missing and 2 partials ⚠️

Additional details and impacted files

@@           Coverage Diff            @@
##             main     #486    +/-   ##
========================================
  Coverage   80.57%   80.58%            
========================================
  Files         315      315            
  Lines      122793   122900   +107     
  Branches   122793   122900   +107     
========================================
+ Hits        98944    99037    +93     
- Misses      19826    19838    +12     
- Partials     4023     4025     +2     

Flag	Coverage Δ
cli-ort-linux	83.27% <ø> (ø)
cli-ort-windows	82.78% <ø> (+0.10%)	⬆️
mlas	77.91% <ø> (ø)
offline	80.50% <87.93%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...-runtime-shape-inference/src/handlers/container.rs	88.83% <87.93%> (-2.28%)	⬇️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

🔴 Benchmark Regression Detected

Comparison of criterion micro-benchmarks: PR head vs merge-base, measured on the same runner in the same job (base first → PR second).

ℹ️ Absolute times are informational only — they vary with runner load. The % change column is the reliable signal because both sides ran under identical conditions.

Status	Scenario	Base	PR	Change
🔴	matmul/large_generic_bf16_threads=8/32x1024x1024	1.46 ms	2.00 ms	+36.4%
⚠️	gather/large_f16_threads=1-internal/131072	9.73 µs	11.68 µs	+20.1%
⚠️	matmul/small_generic_f16_threads=1/1x256x256	27.68 µs	32.88 µs	+18.8%
⚠️	add/small_f32_threads=1-internal/1024	179.1 ns	210.3 ns	+17.4%
✅	tokenization/decode_tokens_per_second	6.19 ms	7.04 ms	+13.8%
✅	add/large_f32_threads=1-internal/4194304	37.69 ms	41.50 ms	+10.1%
✅	add/medium_f16_threads=1-internal/262144	2.40 ms	2.63 ms	+9.5%
✅	add/small_bf16_threads=1-internal/1024	11.97 µs	13.01 µs	+8.7%
✅	add/medium_bf16_threads=1-internal/262144	2.46 ms	2.65 ms	+7.7%
✅	sampling_latency/min_p_per_token	322.68 µs	343.23 µs	+6.4%
✅	add/medium_f32_threads=1-internal/262144	2.43 ms	2.58 ms	+6.0%
✅	matmul/small_generic_f32_threads=8/1x256x256	32.60 µs	34.46 µs	+5.7%
✅	gather/medium_f32_threads=1-internal/32768	3.48 µs	3.68 µs	+5.7%
✅	matmul/medium_generic_f32_threads=8/32x512x512	895.42 µs	946.16 µs	+5.7%
✅	add/small_f16_threads=1-internal/1024	12.00 µs	12.51 µs	+4.3%
✅	gather/small_bf16_threads=1-internal/4096	433.1 ns	450.7 ns	+4.1%
✅	add/large_f16_threads=1-internal/4194304	38.73 ms	40.23 ms	+3.9%
✅	tokenization/encode_tokens_per_second	376.31 µs	388.92 µs	+3.4%
✅	reduce_mean/small_f32_threads=1-internal/4096	13.78 µs	14.20 µs	+3.1%
✅	add/large_bf16_threads=1-internal/4194304	38.73 ms	39.74 ms	+2.6%
✅	gather/small_f32_threads=1-internal/4096	616.1 ns	632.1 ns	+2.6%
✅	sampling_latency/top_p_per_token	973.17 µs	994.29 µs	+2.2%
✅	gather/small_f16_threads=1-internal/4096	432.8 ns	439.6 ns	+1.6%
✅	matmul/small_generic_bf16_threads=8/1x256x256	29.00 µs	29.42 µs	+1.5%
✅	gather/medium_f16_threads=1-internal/32768	2.22 µs	2.26 µs	+1.4%
✅	matmul/small_generic_bf16_threads=1/1x256x256	28.40 µs	28.76 µs	+1.2%
✅	matmul/medium_generic_f16_threads=8/32x512x512	27.73 µs	27.90 µs	+0.6%
✅	matmul/medium_generic_f16_threads=1/32x512x512	27.32 µs	27.45 µs	+0.5%
✅	matmul/large_generic_f32_threads=8/32x1024x1024	3.66 ms	3.67 ms	+0.4%
✅	matmul/small_generic_f32_threads=1/1x256x256	33.90 µs	34.01 µs	+0.3%
✅	sampling_latency/top_k_per_token	475.23 µs	476.73 µs	+0.3%
✅	matmul/medium_generic_f32_threads=1/32x512x512	2.15 ms	2.15 ms	+0.1%
✅	gather/large_bf16_threads=1-internal/131072	9.76 µs	9.73 µs	-0.2%
✅	matmul/medium_generic_bf16_threads=1/32x512x512	491.05 µs	488.18 µs	-0.6%
✅	reduce_mean/large_f32_threads=1-internal/262144	920.76 µs	915.06 µs	-0.6%
✅	gather/large_f32_threads=1-internal/131072	26.27 µs	26.08 µs	-0.7%
✅	matmul/large_generic_f32_threads=1/32x1024x1024	8.66 ms	8.58 ms	-0.9%
✅	matmul/large_generic_f16_threads=1/32x1024x1024	73.33 µs	72.54 µs	-1.1%
✅	kv_cache/alloc_dealloc_pages	37.61 µs	37.17 µs	-1.2%
✅	reduce_mean/medium_f32_threads=1-internal/65536	231.35 µs	227.49 µs	-1.7%
✅	sampling_latency/greedy_per_token	3.28 µs	3.23 µs	-1.7%
✅	matmul/medium_generic_bf16_threads=8/32x512x512	360.24 µs	352.01 µs	-2.3%
✅	matmul/small_generic_f16_threads=8/1x256x256	29.07 µs	28.37 µs	-2.4%
✅	gather/medium_bf16_threads=1-internal/32768	2.35 µs	2.21 µs	-5.8%
✅	matmul/large_generic_bf16_threads=1/32x1024x1024	2.02 ms	1.85 ms	-8.3%
✅	logit_processing/seven_processor_chain_per_step	1.45 ms	1.24 ms	-14.4%
🟢	matmul/large_generic_f16_threads=8/32x1024x1024	97.47 µs	77.92 µs	-20.1%
🟢	grammar_masking/llguidance_compute_mask/32	105.98 µs	81.47 µs	-23.1%

Visual flags: ⚠️ ≥ 15% slower, 🔴 ≥ 30% slower — calibrated against measured runner noise (~27% worst-case on multi-threaded matmul)

Host info

CPU: Apple M1 (Virtual)
Cores: 3
OS: Darwin 25.5.0 arm64
Rust: rustc 1.97.1 (8bab26f4f 2026-07-14)
Load avg: { 5.00 4.66 5.57 }

What this cannot catch

• Regressions in code paths not covered by these benchmarks (e.g., end-to-end decode with a real model)
• Sub-threshold regressions that compound over multiple PRs
• Performance changes that only manifest under GPU execution
• Latency changes in the ORT integration path (these benchmarks exercise the native Rust kernels)

[justinchuby]

VERDICT: APPROVE

Independent review by Melina (reviewer). Worktree from origin/squad/449-container-inc2 @ 5d1b256, base main, refs #449. CPU-only, pure shape-inference crate. Author Harry locked out.

Summary

Clean, spec-faithful increment on the merged #477 container foundation. All four opset-11 ops correct, the byte-identical tensor invariant holds, the DRY generalization is genuinely shared with no divergence, tests are non-tautological, and the catalog pin is exact. Full CPU verification green.

1. Byte-identical tensor path preserved (THE invariant) — CONFIRMED

• Regression test tensor_only_path_is_byte_identical_after_container_type_model passes unchanged (tests/graph_inference.rs, 27 passed).
• This PR touches only src/handlers/container.rs, tests/op_rules.rs, and a decisions inbox file (git diff HEAD~1..HEAD). No tensor handler, infer.rs, or lib.rs changed.
• The merge generalization did NOT leak into the tensor path: merge_element -> merge_tensor is defined in and called only from container.rs (src/handlers/container.rs:80, callers at :59 SequenceConstruct, :168 SequenceInsert).
• Behavior-preserving: TypeInfo -> TensorType always yields Some(shape) (context.rs:74-80 From impl), so SequenceConstruct's new (Some, Some) arm (container.rs:96-99) is identical to the old match acc.shape logic. No semantic change.

2. ConcatFromSequence (seq -> tensor) — CORRECT vs ONNX

container.rs:238-277.

• new_axis=0: output_rank = element rank; concat axis extent set to a fresh symbol; other dims (incl. symbolic) preserved; rank unchanged. Verified by concat_from_sequence_recovers_tensor_with_symbolic_concat_axis and concat_from_sequence_preserves_symbolic_non_concat_dims (non-concat symbolic dim survives).
• new_axis=1: output_rank = rank+1; fresh symbolic dim inserted at axis (shape.insert). checked_axis(axis, rank+1) correctly allows the append position (index r), matching ONNX's [-r-1, r] insertion range (handlers/mod.rs:39). Verified by concat_from_sequence_new_axis_inserts_a_symbolic_stack_dim (rank 3, [c(2), sym, c(3)]).
• axis mandatory: missing -> Invalid (concat_from_sequence_missing_axis_errors).
• dtype-only element -> honest unknown-rank degradation, output left unresolved, same as SequenceAt (concat_from_sequence_dtype_only_element_stays_unresolved asserts type_info.is_none()).
• element-tensor -> TypeInfo recovery via ctx.set_output(dtype, shape) is correct.
• Round-trip is NON-tautological: split_to_sequence_then_concat_from_sequence_recovers_a_tensor splits [2,6] on axis 1 (element [2,1]), then ConcatFromSequence axis 1 -> [c(2), symbolic]. Batch dim c(2) genuinely recovered, concat axis honestly symbolic. Real values flow end to end.

3. SplitToSequence — CORRECT vs ONNX / runtime reference

container.rs:194-234.

• Explicit split input present -> split axis degrades to a fresh symbol (chunk extents vary); keepdims ignored, matching ONNX (keepdims only applies without split). Verified split_to_sequence_explicit_split_makes_axis_symbolic (non-split axis c(4) preserved).
• No split + keepdims=1 (default) -> chunk-of-1, axis kept at extent 1 (split_to_sequence_default_keeps_split_axis_at_extent_one, parameterized over Float32/Int64).
• No split + keepdims=0 -> axis removed (split_to_sequence_keepdims_zero_removes_split_axis).
• Element dtype = input dtype. Negative axis normalized (split_to_sequence_negative_axis_and_symbolic_dims). rank-0 rejected with InvalidRank (split_to_sequence_scalar_input_errors).

4. SequenceInsert / SequenceErase + unify — CONSISTENT with foundation

• SequenceInsert (container.rs:161-179): element = unify(existing, inserted) via the SHARED merge_tensor. dtype mismatch -> error (sequence_insert_mismatched_dtype_errors); extent disagreement -> fresh symbol with rank preserved (sequence_insert_disagreeing_extent_degrades_but_keeps_rank, agreeing dim c(2) kept); symbolic dims preserved (sequence_insert_preserves_symbolic_element_dims); dtype-only/unconfirmed side -> dtype kept, shape unknown (sequence_insert_into_dtype_only_sequence_keeps_dtype_but_unknown_shape); untyped seq adopts inserted (sequence_insert_into_untyped_sequence_adopts_inserted_tensor). Semantics match SequenceConstruct's merge exactly (same merge_tensor/merge_shape).
• SequenceErase (container.rs:184-192): element type preserved unchanged; only length (untracked) changes. Verified sequence_erase_preserves_element_type + read-back via SequenceAt.

5. DRY confirmation — CONFIRMED, no divergence

• merge_element renamed/generalized to merge_tensor(ctx, op, acc, other) taking two TensorType and an op label; single definition (container.rs:80). Called by both SequenceConstruct (:59) and SequenceInsert (:168) — no divergent copy. merge_shape and checked_axis reused as-is. sequence_element_tensor helper (container.rs:151) shared by SequenceInsert and ConcatFromSequence.

6. Tests non-tautological — CONFIRMED

19 new #[test] functions added; real dtype/shape/symbolic assertions, mismatch->error, dtype-only->unresolved, two round-trips (construct+insert+at; split+concat), one parameterized over dtypes. Not mere non-crash checks.

7. Catalog exactness — EXACT

• expanded_registry_catalog_count_is_pinned updated 213->217 ops, 258->262 entries (op_rules.rs) and passes. Exactly the 4 added ops (SequenceInsert, SequenceErase, SplitToSequence, ConcatFromSequence), each a single opset-11 entry (+4 ops, +4 entries). No phantom.

Verification (CPU-only) — ALL GREEN

• cargo test -p onnx-runtime-shape-inference: lib 16 passed; graph_inference 27 passed (byte-identical GREEN); op_rules 275 passed; doctests 1 passed. 0 failed.
• cargo fmt --all --check: clean.
• cargo clippy -p onnx-runtime-shape-inference --all-targets -D warnings: clean, no warnings.
• cargo build -p onnx-runtime-session -p onnx-runtime-eager: Finished OK (NodeIo consumers still build).

Minor note (non-blocking)

The PR description says op_rules +21 (254->275); the actual delta is 19 new #[test] fns (256->275). A description/count inaccuracy only — the authoritative harness reports 275 passing and the catalog pin is correct. No action required.