Skip to content

Python: Fix date/time transforms #5667

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Fokko merged 1 commit into from
Aug 29, 2022
Merged

Python: Fix date/time transforms #5667

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 9 additions & 13 deletions python/pyiceberg/utils/datetime.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@

def micros_to_days(timestamp: int) -> int:
"""Converts a timestamp in microseconds to a date in days"""
return (datetime.fromtimestamp(timestamp / 1_000_000) - EPOCH_TIMESTAMP).days
return timedelta(microseconds=timestamp).days
Copy link
Contributor Author

@rdblue rdblue Aug 29, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, we can go directly to delta rather than going to date and subtracting.



def micros_to_time(micros: int) -> time:
Expand Down Expand Up @@ -133,28 +133,24 @@ def to_human_timestamp(timestamp_micros: int) -> str:
return (EPOCH_TIMESTAMP + timedelta(microseconds=timestamp_micros)).isoformat()


def micros_to_hours(timestamp: int) -> int:
"""Converts a timestamp in microseconds to a date in hours"""
return int((datetime.utcfromtimestamp(timestamp // 1_000_000) - EPOCH_TIMESTAMP).total_seconds() / 3600)
def micros_to_hours(micros: int) -> int:
"""Converts a timestamp in microseconds to hours from 1970-01-01T00:00"""
return micros // 3_600_000_000
Copy link
Contributor Author

@rdblue rdblue Aug 29, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This converts to a timestamp only to get back to a delta, and then get the seconds. Instead, we can convert directly.



def days_to_months(days: int) -> int:
"""Creates a date from the number of days from 1970-01-01"""
d = days_to_date(days)
return (d.year - EPOCH_DATE.year) * 12 + (d.month - EPOCH_DATE.month)


def micros_to_months(timestamp: int) -> int:
dt = micros_to_timestamp(timestamp)
return (dt.year - EPOCH_TIMESTAMP.year) * 12 + (dt.month - EPOCH_TIMESTAMP.month) - (1 if dt.day < EPOCH_TIMESTAMP.day else 0)
def micros_to_months(micros: int) -> int:
dt = micros_to_timestamp(micros)
return (dt.year - EPOCH_TIMESTAMP.year) * 12 + (dt.month - EPOCH_TIMESTAMP.month)


def days_to_years(days: int) -> int:
return days_to_date(days).year - EPOCH_DATE.year


def micros_to_years(timestamp: int) -> int:
dt = micros_to_timestamp(timestamp)
return (dt.year - EPOCH_TIMESTAMP.year) - (
1 if dt.month < EPOCH_TIMESTAMP.month or (dt.month == EPOCH_TIMESTAMP.month and dt.day < EPOCH_TIMESTAMP.day) else 0
Copy link
Contributor Author

@rdblue rdblue Aug 29, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Month and day are never 0.

)
def micros_to_years(micros: int) -> int:
return micros_to_timestamp(micros).year - EPOCH_TIMESTAMP.year