Raise OutOfBoundsDatetime for datetime conversions that exceed the target unit's range by galipremsagar · Pull Request #23216 · NVIDIA/cudf

galipremsagar · 2026-07-10T15:45:21Z

Description

cuDF silently wrapped the underlying int64 values when a datetime conversion could not fit the target resolution — e.g. cudf.Series(np.array(["2263-01-01"], dtype="datetime64[D]"), dtype="datetime64[ns]") produced Timestamp('1678-06-12 00:25:26.290448384'). pandas bounds-checks every narrowing conversion (astype_overflowsafe) and raises pd.errors.OutOfBoundsDatetime. This PR makes cuDF raise the same error, fixing all 16 xfailed parametrizations of tests/base/test_constructors.py::test_constructor_datetime_outofbound in the pandas test suite (Series/Index/DataFrame constructors × datetime64[D] array, object arrays of datetime.datetime, np.datetime64, and strings).

Changes:

DatetimeColumn.as_datetime_column bounds-checks unit-narrowing casts on the column's integer view (a host round-trip through pd.Timestamp can itself overflow for extreme values) and raises with a pandas-matching message. The check is relative to the target unit: year 9999 still fits in datetime64[s]/datetime64[us], just not nanoseconds.
StringColumn.strptime parses to seconds first for sub-second datetime targets and rejects values whose whole-second part falls outside the target unit's range (libcudf otherwise parses straight into wrapped int64). Overflows of less than a second inside the boundary second are not detected, but in-bounds values are never rejected.
to_datetime's dual ns/us %f parse falls back to microsecond precision when the nanosecond parse raises, matching pandas' unit inference. Nanosecond-precision strings beyond the ns range (e.g. "2263-01-01 00:00:00.123456789") now raise like pandas instead of returning wrapped values.
The tz transition-table casts in tz_localize/_local_time switch to the raw cast: those tables contain sentinel entries beyond the finer units' bounds that intentionally wrap around.
can_cast_safely now checks the negative bound (pre-epoch values were never checked, so e.g. 1600-01-01 in ms reported as safely castable to ns) and explicitly treats coarser-resolution casts as unsafe: callers (replace, join key matching) rely on "safely" meaning lossless, which the old wrapped-bound formula only provided accidentally.
Scalar .loc lookups with an out-of-range datetime key raise KeyError like pandas; list keys raise OutOfBoundsDatetime in both.
Removes the 16 now-passing pandas-testing plugin entries.

Known minor divergence: when the target dtype is a pd.ArrowDtype timestamp, cuDF raises OutOfBoundsDatetime where pandas surfaces pyarrow.lib.ArrowInvalid; raising is still strictly better than the previous silent wraparound.

Tests added cover constructors (5 input flavors × Series/Index/DataFrame), astype (above/below range, with nulls, all-null, tz-aware, string→tz), to_datetime, replace with sub-resolution values, .loc, and can_cast_safely (negative bound, coarser-resolution).

Checklist

I am familiar with the Contributing Guidelines.
New or existing tests cover these changes.
The documentation is up to date with these changes.

…rget unit's range Forcing values beyond the target resolution's representable range (e.g. constructing with dtype='datetime64[ns]' from year-2263 data, or astype from a coarser unit) silently wrapped the underlying int64 and produced garbage timestamps. pandas bounds-checks every narrowing conversion (astype_overflowsafe) and raises pd.errors.OutOfBoundsDatetime. - DatetimeColumn.as_datetime_column bounds-checks narrowing casts on the column's integer view (a host round-trip through pd.Timestamp can itself overflow for extreme values) and raises with a pandas-matching message. The check is relative to the target unit, so e.g. year 9999 still fits in datetime64[s] and datetime64[us]. - StringColumn.strptime parses to seconds first for sub-second targets and rejects values whose whole-second part falls outside the target unit's range; overflows of less than a second inside the boundary second are not detected, but in-bounds values are never rejected. - to_datetime's dual ns/us %f parse falls back to microsecond precision when the nanosecond parse raises, matching pandas' unit inference. - The tz transition-table casts in tz_localize/_local_time use the raw cast: those tables contain sentinel entries beyond the finer units' bounds that intentionally wrap around. - can_cast_safely now checks the negative bound (pre-epoch values were never checked) and explicitly treats coarser-resolution casts as unsafe: callers (replace, join key matching) rely on 'safely' meaning lossless, which the old wrapped-bound formula provided accidentally. - Scalar .loc lookups with an out-of-range datetime key raise KeyError like pandas; list keys raise OutOfBoundsDatetime in both. - Remove 16 pandas-testing plugin entries for tests/base/test_constructors.py::test_constructor_datetime_outofbound, which now passes. - Add tests covering constructors (Series/Index/DataFrame), astype (including tz and string->tz), to_datetime, replace, .loc, and can_cast_safely.

copy-pr-bot · 2026-07-10T15:45:25Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

galipremsagar · 2026-07-10T15:46:17Z

/okay to test 69fbab8

coderabbitai · 2026-07-10T15:56:43Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e9c015b8-7a45-440b-a67c-1ec6c674256f

📥 Commits

Reviewing files that changed from the base of the PR and between 03c9059 and 48cfadd.

📒 Files selected for processing (2)

python/cudf/cudf/core/column/string.py
python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py

💤 Files with no reviewable changes (1)

python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py

🚧 Files skipped from review as they are similar to previous changes (1)

python/cudf/cudf/core/column/string.py

📝 Walkthrough

Summary by CodeRabbit

Bug Fixes
- Strengthened datetime/timedelta resolution handling to prevent unsafe truncation, overflow, and wraparound during casts.
- Added unit-aware out-of-bounds validation with pandas-compatible OutOfBoundsDatetime for datetime parsing and localization.
- Improved to_datetime/strptime fractional parsing to fall back to microseconds when nanoseconds are out of range.
- Updated Series.loc so scalar out-of-bounds datetime labels raise KeyError, while list-like lookups still raise OutOfBoundsDatetime.
Tests
- Expanded coverage for out-of-bounds casting, construction, parsing, indexing, and replacement (including tz-aware cases).
Chores
- Removed obsolete testing-plugin expectations for out-of-bounds datetime failures.

Walkthrough

Changes

Datetime safety and pandas compatibility

Layer / File(s)	Summary
Temporal bounds and string parsing `python/cudf/cudf/utils/temporal.py`, `python/cudf/cudf/core/column/string.py`, `python/cudf/cudf/core/tools/datetimes.py`, `python/cudf/cudf/tests/general_functions/test_to_datetime.py`	Adds unit-aware datetime bounds validation and handles nanosecond parsing fallback to microsecond precision.
Datetime casting and resolution checks `python/cudf/cudf/core/column/temporal_base.py`, `python/cudf/cudf/core/column/datetime.py`, `python/cudf/cudf/tests/private_objects/test_column.py`, `python/cudf/cudf/tests/series/methods/test_astype.py`, `python/cudf/cudf/tests/series/methods/test_replace.py`	Rejects unsafe resolution casts, checks finer-unit bounds, preserves timezone sentinel wrapping, and adds naive/tz-aware casting coverage.
Datetime construction validation `python/cudf/cudf/tests/series/test_constructors.py`	Tests pandas-compatible exceptions for out-of-bounds datetime construction across Series, Index, and DataFrame.
Datetime label indexing behavior `python/cudf/cudf/core/indexing_utils.py`, `python/cudf/cudf/tests/series/indexing/test_loc.py`, `python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py`	Maps scalar out-of-bounds datetime labels to `KeyError` while preserving list-like errors and removes obsolete test mappings.

Estimated code review effort: 4 (Complex) | ~45 minutes

Suggested reviewers: matt711, mroeschke, tomaugspurger

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 11.11% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: raising OutOfBoundsDatetime for overflow in datetime conversions.
Description check	✅ Passed	The description is directly related to the changeset and matches the implementation and tests described in the summary.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

… comment

galipremsagar · 2026-07-13T20:26:31Z

/okay to test 48cfadd

galipremsagar · 2026-07-14T02:42:27Z

/merge

galipremsagar requested a review from a team as a code owner July 10, 2026 15:45

galipremsagar requested review from Matt711 and TomAugspurger July 10, 2026 15:45

galipremsagar added bug Something isn't working non-breaking Non-breaking change Python Affects Python cuDF API. cudf.pandas Issues specific to cudf.pandas labels Jul 10, 2026

github-project-automation Bot added this to cuDF Python Jul 10, 2026

github-actions Bot assigned galipremsagar Jul 10, 2026

GPUtester moved this to In Progress in cuDF Python Jul 10, 2026

mroeschke reviewed Jul 10, 2026

View reviewed changes

Comment thread python/cudf/cudf/core/column/string.py Outdated

galipremsagar requested a review from mroeschke July 13, 2026 20:20

galipremsagar mentioned this pull request Jul 13, 2026

[FEA] cudf::strings::to_timestamps should not silently wrap on overflow of the target unit #23247

Closed

galipremsagar added 2 commits July 13, 2026 20:24

Reference issue NVIDIA#23247 in the to_timestamps overflow workaround…

03c9059

… comment

Merge branch 'main' into datetime-out-of-bounds-raise

48cfadd

mroeschke approved these changes Jul 13, 2026

View reviewed changes

rapids-bot Bot merged commit 5360c61 into NVIDIA:main Jul 14, 2026
126 checks passed

github-project-automation Bot moved this from In Progress to Done in cuDF Python Jul 14, 2026

Navigation Menu

Uh oh!

Raise OutOfBoundsDatetime for datetime conversions that exceed the target unit's range - #23216

Raise OutOfBoundsDatetime for datetime conversions that exceed the target unit's range#23216
rapids-bot[bot] merged 3 commits into
NVIDIA:mainfrom
galipremsagar:datetime-out-of-bounds-raise

galipremsagar commented Jul 10, 2026

Uh oh!

copy-pr-bot Bot commented Jul 10, 2026

Uh oh!

galipremsagar commented Jul 10, 2026

Uh oh!

coderabbitai Bot commented Jul 10, 2026 •

edited

Loading

Summary by CodeRabbit

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

Uh oh!

galipremsagar commented Jul 13, 2026

Uh oh!

galipremsagar commented Jul 14, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

galipremsagar commented Jul 10, 2026

Description

Checklist

Uh oh!

copy-pr-bot Bot commented Jul 10, 2026

Uh oh!

galipremsagar commented Jul 10, 2026

Uh oh!

coderabbitai Bot commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

Uh oh!

galipremsagar commented Jul 13, 2026

Uh oh!

galipremsagar commented Jul 14, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

coderabbitai Bot commented Jul 10, 2026 •

edited

Loading