Skip to content

Fix orc pytest failures - #21847

Merged
galipremsagar merged 1 commit into
NVIDIA:pandas3from
galipremsagar:orc_fixes
Mar 19, 2026
Merged

Fix orc pytest failures#21847
galipremsagar merged 1 commit into
NVIDIA:pandas3from
galipremsagar:orc_fixes

Conversation

@galipremsagar

@galipremsagar galipremsagar commented Mar 18, 2026

Copy link
Copy Markdown
Contributor

Description

This PR fixes all 16 orc pytest failures.
Cpp bug:

// The conversion map tells the statistics system how to normalize
// column types before computing min/max. For ORC, all timestamps
// must be converted to nanoseconds because split_nanosecond_timestamp()
// in stats_enc.cu expects nanosecond input.

What happened without the fix: A datetime64[ms] column with min value 1970-07-11 07:00:35.528 (stored as 16527635528 ms since epoch) was passed directly to split_nanosecond_timestamp() which interpreted it as nanoseconds:


split_nanosecond_timestamp(16527635528)
  → treats as 16,527,635,528 nanoseconds
  → 16,527 milliseconds + 635,528 nanoseconds remainder
  → ORC stats report min = 16,527 ms = 16.527 seconds
  → 1970-01-01 00:00:16.527  ← WRONG (off by ~1,000,000x)
With the fix: The value is first converted to nanoseconds via time_point_cast:


16527635528 ms → 16,527,635,528,000,000 ns (via conversion map)
split_nanosecond_timestamp(16527635528000000)
  → 16,527,635,528 milliseconds + 0 nanoseconds remainder
  → ORC stats report min = 16,527,635,528 ms
  → 1970-07-11 07:00:35.528  ← CORRECT

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@galipremsagar galipremsagar self-assigned this Mar 18, 2026
@galipremsagar
galipremsagar requested review from a team as code owners March 18, 2026 19:12
@galipremsagar
galipremsagar requested review from mhaseeb123 and removed request for a team March 18, 2026 19:13
@galipremsagar galipremsagar added bug Something isn't working libcudf Affects libcudf (C++/CUDA) code. Python Affects Python cuDF API. improvement Improvement / enhancement to an existing function labels Mar 18, 2026
@GPUtester GPUtester moved this to In Progress in cuDF Python Mar 18, 2026
@mhaseeb123
mhaseeb123 requested a review from vuule March 18, 2026 20:06

@mhaseeb123 mhaseeb123 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

two-liner C++ changes look in order to me but requested a review from @vuule

@vuule

vuule commented Mar 18, 2026

Copy link
Copy Markdown
Contributor

Changes look good.
Is there an issue for this? would love to see the issue linked in the PR description and a short description of what the fix is.

@galipremsagar

galipremsagar commented Mar 19, 2026

Copy link
Copy Markdown
Contributor Author

Changes look good. Is there an issue for this? would love to see the issue linked in the PR description and a short description of what the fix is.

Thanks @mhaseeb123 & @vuule ! This is a fix to pandas3 feature branch. I don't have a bug opened for this but updated this pr description to have the bug description. #21847 (comment)

@galipremsagar galipremsagar added the 5 - Ready to Merge Testing and reviews complete, ready to merge label Mar 19, 2026
@galipremsagar
galipremsagar merged commit 9b3d56d into NVIDIA:pandas3 Mar 19, 2026
62 of 100 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in cuDF Python Mar 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

5 - Ready to Merge Testing and reviews complete, ready to merge bug Something isn't working improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. Python Affects Python cuDF API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants