Skip to content

Fix CSE array formula evaluation: count booleans in arithmetic operators#1720

Merged
tonyqus merged 2 commits into
nissl-lab:masterfrom
swyfft-insurance:fix/cse-array-boolean-arithmetic
Mar 10, 2026
Merged

Fix CSE array formula evaluation: count booleans in arithmetic operators#1720
tonyqus merged 2 commits into
nissl-lab:masterfrom
swyfft-insurance:fix/cse-array-boolean-arithmetic

Conversation

@ken-swyfft
Copy link
Copy Markdown
Contributor

@ken-swyfft ken-swyfft commented Mar 9, 2026

Problem

CSE (Ctrl+Shift+Enter) array formulas that use the common pattern of boolean array multiplication fail with #VALUE! errors:

{INDEX(..., MATCH(1, ($A$1:$A$10=val1)*($B$1:$B$10=val2), 0), col)}

The = comparison operators correctly produce CacheAreaEval arrays of BoolEval values (TRUE/FALSE) when in array mode. However, when the * operator then tries to multiply these arrays element-wise, all the boolean values are silently dropped, producing empty/wrong-sized arrays that cascade to #VALUE! errors.

Root Cause

In TwoOperandNumericOperation.ArrayEval, the MutableValueCollector is constructed with isReferenceBoolCounted=false:

private readonly MatrixFunction.MutableValueCollector instance = 
    new MatrixFunction.MutableValueCollector(false, true);

When CollectValues iterates over a CacheAreaEval (which is a TwoDEval), it calls CollectValue(ve, isViaReference: true, temp). With isReferenceBoolCounted=false, all BoolEval values are silently skipped in MultiOperandNumericFunction.CollectValue():

if (ve is BoolEval boolEval)
{
    if (!isViaReference || _isReferenceBoolCounted)
    {
        temp.Add(boolEval.NumberValue);
    }
    return;  // ← silently skips when isViaReference=true and _isReferenceBoolCounted=false
}

This is correct behavior for SUM-like functions (where booleans in ranges are ignored per Excel spec), but wrong for arithmetic operators in array mode, where booleans must be coerced to 0/1.

Fix

Change isReferenceBoolCounted from false to true in TwoOperandNumericOperation.ArrayEval:

private readonly MatrixFunction.MutableValueCollector instance = 
    new MatrixFunction.MutableValueCollector(true, true);

Also added ErrorEval handling in Match.EvaluateLookupRange() so that upstream errors propagate as EvaluationException rather than throwing a raw Exception.

Testing

New unit tests (TestCseArrayBooleanArithmetic.cs — 5 tests):

  1. TestIndexMatchWithBooleanArrayMultiplication — End-to-end test of the {INDEX(range, MATCH(1, (col1=val1)*(col2=val2), 0))} pattern, the exact CSE formula used in real-world workbooks
  2. TestBooleanArrayMultiplicationDifferentMatches — Verifies correct results across multiple lookup value combinations
  3. TestBooleanArrayMultiplicationNoMatch — Confirms #N/A (not #VALUE!) when no match exists
  4. TestMatchWithErrorEvalInLookupRange — Validates that Match.EvaluateLookupRange handles ErrorEval gracefully instead of throwing a raw Exception
  5. TestSimpleBooleanArrayMultiplication — Direct test of element-wise multiplication of two boolean arrays

All existing NPOI tests pass (4,572 passed across net8.0 and net472, 0 failed).

Real-world validation against 56 Excel rater workbooks containing 460,000+ formula cells and 1,228 CSE array formula cells:

  • Before fix: 158+ CSE array formula failures, cascading #VALUE! exceptions
  • After fix: 0 CSE array formula failures, 98.4%–100% formula evaluation success across all workbooks

ken-swyfft and others added 2 commits March 9, 2026 12:38
The MutableValueCollector in TwoOperandNumericOperation's ArrayEval used
isReferenceBoolCounted=false, which silently dropped BoolEval values from
area references during array arithmetic. This caused CSE formulas like
{INDEX(...,MATCH(1,($range=val1)*($range=val2),0),col)} to fail because
the boolean arrays from comparisons were collected as empty/wrong-sized
arrays, cascading to #VALUE! errors.

Fix: Set isReferenceBoolCounted=true so boolean values from relational
operators (=, <, >, etc.) are properly coerced to 0/1 during array
multiplication.

Also: Handle ErrorEval in Match.EvaluateLookupRange() gracefully instead
of throwing a raw Exception.

Tested against 8 Swyfft rater workbooks (56 total available):
- Before: 158+ CSE array formula failures, cascading exceptions
- After: 0 CSE failures, 98.4-100% formula evaluation success rate

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Five tests covering the fix for boolean array handling in CSE formulas:

1. TestIndexMatchWithBooleanArrayMultiplication - End-to-end test of the
   INDEX/MATCH pattern with boolean array multiplication (the exact
   pattern used in real-world rater workbooks)

2. TestBooleanArrayMultiplicationDifferentMatches - Verifies correct
   results for different lookup value combinations

3. TestBooleanArrayMultiplicationNoMatch - Verifies #N/A (not #VALUE!)
   when no match exists

4. TestMatchWithErrorEvalInLookupRange - Verifies Match handles ErrorEval
   gracefully instead of throwing raw Exception

5. TestSimpleBooleanArrayMultiplication - Direct test of element-wise
   multiplication of two boolean arrays

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@tonyqus
Copy link
Copy Markdown
Member

tonyqus commented Mar 10, 2026

Thank you for the detailed description.

@tonyqus
Copy link
Copy Markdown
Member

tonyqus commented Mar 10, 2026

LGTM

@tonyqus tonyqus merged commit 63723f0 into nissl-lab:master Mar 10, 2026
2 of 3 checks passed
@tonyqus
Copy link
Copy Markdown
Member

tonyqus commented Mar 10, 2026

I glance at your Github profile and found that we have something in common.

My last company is VRSK (insurance industry company) and the HQ of VRSK is also in NJ. What a coincidence 👯‍♂️

@ken-swyfft
Copy link
Copy Markdown
Contributor Author

I glance at your Github profile and found that we have something in common.

My last company is VRSK (insurance industry company) and the HQ of VRSK is also in NJ. What a coincidence 👯‍♂️

Oh, interesting. Yeah, probably like most insurance companies, we use Verisk for a whole bunch of things. Swyfft is almost entirely remote (I'm in the Seattle area), but our headquarters are in New Jersey.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants