Skip to content

Fix scalar as column view for null scalars - #22773

Merged
rapids-bot[bot] merged 1 commit into
NVIDIA:mainfrom
mhaseeb123:fix/scalar-as-column-view-binary-ops
Jun 3, 2026
Merged

Fix scalar as column view for null scalars#22773
rapids-bot[bot] merged 1 commit into
NVIDIA:mainfrom
mhaseeb123:fix/scalar-as-column-view-binary-ops

Conversation

@mhaseeb123

@mhaseeb123 mhaseeb123 commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Description

Closes #22757. Supersedes #22759.

This PR fixes scalar_as_column_view to handle null input input scalar by allocating a single-element bitmask for them (kept alive using the aux column). Valid scalars are still directly just converted to column views with nullptr as their nullmask.

The newly added tests surface the memcheck error when compute-sanitizer is run with --rmm_mode=cuda.

Credits: @simoneves for the original PR.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@mhaseeb123
mhaseeb123 requested a review from a team as a code owner June 3, 2026 22:10
@github-actions github-actions Bot added the libcudf Affects libcudf (C++/CUDA) code. label Jun 3, 2026
@mhaseeb123
mhaseeb123 requested review from bdice and simoneves June 3, 2026 22:11
@mhaseeb123 mhaseeb123 added bug Something isn't working 4 - Needs Review Waiting for reviewer to review or respond non-breaking Non-breaking change Velox Functionality that helps Velox-cudf labels Jun 3, 2026
@coderabbitai

coderabbitai Bot commented Jun 3, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

This PR fixes a memory safety bug in cuDF's scalar-to-column-view conversion where null-aware binary operations incorrectly reinterpret scalar device bool validity data as a column bitmask, causing out-of-bounds reads. The fix explicitly materializes proper null masks for invalid scalars and omits masks for valid ones in both fixed-width and string_view specializations, validated by new null-aware fixed-point test cases.

Changes

Scalar null-mask handling in binary operations

Layer / File(s) Summary
Include and fixed-width scalar null-mask conversion
cpp/src/binaryop/compiled/binary_ops.cu
Added cudf/detail/null_mask.hpp header. Reworked scalar_as_column_view for fixed-width types to branch on validity: invalid scalars now create a proper single-element null mask via create_null_mask and return an auxiliary column retaining that mask; valid scalars construct column_view without a null mask. Removed prior invalid reinterpret_cast of scalar validity storage.
String_view scalar null-mask conversion
cpp/src/binaryop/compiled/binary_ops.cu
Updated cudf::string_view scalar specialization with the same validity branching: valid scalars produce column_view without null mask while keeping offsets child alive; invalid scalars create all-null mask, attach it to the column_view, and return auxiliary column retaining both mask and offsets child.
Test coverage for null-aware scalar operations
cpp/tests/binaryop/binop-compiled-fixed_point-test.cpp
Added FixedPointBinaryOpNullMaxColumnScalar test validating NULL_MAX with non-null scalar produces nullable output with all rows valid. Added FixedPointBinaryOpNullEqualsNullColumnScalar test validating NULL_EQUALS with invalid scalar produces boolean results that are true only where column is also null.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title 'Fix scalar as column view for null scalars' directly summarizes the main change: addressing how null scalars are handled in the scalar_as_column_view function.
Linked Issues check ✅ Passed The PR addresses all coding requirements from issue #22757: materializing a proper 1-element null bitmask for null scalars, keeping it alive via auxiliary column, and maintaining nullptr nullmask for valid scalars. Tests verify null-aware binary ops (NULL_MAX, NULL_EQUALS) work correctly.
Out of Scope Changes check ✅ Passed All changes are directly related to fixing scalar_as_column_view for null scalar handling as specified in issue #22757; no unrelated modifications detected.
Description check ✅ Passed The PR description clearly relates to the changeset, explaining the fix to scalar_as_column_view for null scalars and referencing the added tests.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@bdice bdice left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

The newly added tests surface the memcheck error when compute-sanitizer is run with --rmm_mode=cuda.

Apologies, I should have suggested this earlier. I hope this didn't take too long to discover. OOB accesses are often reproducible only with the cuda_memory_resource. (Which is why we set it in our sanitizer tests.)

!s.is_valid(stream));
return std::pair{col_v, std::unique_ptr<column>(nullptr)};

// Valid scalar needs no null mask

@bdice bdice Jun 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a reminder that we need to be careful with the data model here. A non-nullable column and a nullable column with valid data are not identical states. They can round-trip differently to/from various file formats.

If I recall correctly, cudf scalars don't distinguish "valid" from "non-nullable" states today, and thankfully this doesn't have much impact on I/O because we only read/write columns and not scalars, but it's important to keep that in mind.

No action needed, just want to build awareness.

@mhaseeb123 mhaseeb123 Jun 3, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A non-nullable column and a nullable column with valid data are not identical states. They can (and do) round-trip differently to/from various file formats

Learned this the hard way back when I was a new hire while working on round trips with Arrow 😄

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I recall correctly, cudf scalars don't distinguish "valid" from "non-nullable" states today, and thankfully this doesn't have much impact on I/O because we only read/write columns and not scalars, but it's important to keep that in mind.

Yup, this makes sense. Good for a temporary column view state but not when writing/reading.

@mhaseeb123

Copy link
Copy Markdown
Contributor Author

I hope this didn't take too long to discover.

I had just asked Claude to write the smallest possible repro (the test in the PR) and run it under compute-sanitizer. This was the first thing it did when the test didn't fail. All this happened while I was working on something else 😄

@simoneves simoneves left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
Tested in Velox
All decimal tests run clean under compute-sanitizer --tool memcheck

@mhaseeb123

Copy link
Copy Markdown
Contributor Author

/merge

@rapids-bot
rapids-bot Bot merged commit 3e66fd6 into NVIDIA:main Jun 3, 2026
127 checks passed
@mhaseeb123
mhaseeb123 deleted the fix/scalar-as-column-view-binary-ops branch June 3, 2026 23:23
mhaseeb123 added a commit to mhaseeb123/cudf that referenced this pull request Jul 7, 2026
…snapshot]

Backport of the effective diff of rapidsai/cudf PR NVIDIA#23077
("Prepend row index column in Parquet reader"), which was still OPEN at
the time this branch was created.

This captures the PR's net change as a single squashed commit computed as
the diff between its merge-base with main and its head:
  merge-base: 52d322c
  head:       2574b9d

Applied on top of the 26.06.01 pin + NVIDIA#22773 + NVIDIA#22879 backport.

NOTE: This is a PRE-MERGE snapshot; re-sync to the squashed merge commit
once NVIDIA#23077 lands upstream. Unrelated main-only tests present as diff
context (MismatchedSchema*) were intentionally excluded.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

4 - Needs Review Waiting for reviewer to review or respond bug Something isn't working libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change Velox Functionality that helps Velox-cudf

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Possible bug with fixed-point scalar handling in some binary operators

3 participants