Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support jiff integration for postgresql #3511

Closed
wants to merge 7 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,7 @@ bigdecimal = ["sqlx-core/bigdecimal", "sqlx-macros?/bigdecimal", "sqlx-mysql?/bi
bit-vec = ["sqlx-core/bit-vec", "sqlx-macros?/bit-vec", "sqlx-postgres?/bit-vec"]
chrono = ["sqlx-core/chrono", "sqlx-macros?/chrono", "sqlx-mysql?/chrono", "sqlx-postgres?/chrono", "sqlx-sqlite?/chrono"]
ipnetwork = ["sqlx-core/ipnetwork", "sqlx-macros?/ipnetwork", "sqlx-postgres?/ipnetwork"]
jiff = ["sqlx-core/jiff", "sqlx-postgres?/jiff"]
mac_address = ["sqlx-core/mac_address", "sqlx-macros?/mac_address", "sqlx-postgres?/mac_address"]
rust_decimal = ["sqlx-core/rust_decimal", "sqlx-macros?/rust_decimal", "sqlx-mysql?/rust_decimal", "sqlx-postgres?/rust_decimal"]
time = ["sqlx-core/time", "sqlx-macros?/time", "sqlx-mysql?/time", "sqlx-postgres?/time", "sqlx-sqlite?/time"]
Expand All @@ -138,6 +139,7 @@ bigdecimal = "0.4.0"
bit-vec = "0.6.3"
chrono = { version = "0.4.34", default-features = false, features = ["std", "clock"] }
ipnetwork = "0.20.0"
jiff = { version = "0.1.13" }
mac_address = "1.1.5"
rust_decimal = { version = "1.26.1", default-features = false, features = ["std"] }
time = { version = "0.3.36", features = ["formatting", "parsing", "macros"] }
Expand Down
1 change: 1 addition & 0 deletions sqlx-core/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ bit-vec = { workspace = true, optional = true }
bigdecimal = { workspace = true, optional = true }
rust_decimal = { workspace = true, optional = true }
time = { workspace = true, optional = true }
jiff = { workspace = true, optional = true }
ipnetwork = { workspace = true, optional = true }
mac_address = { workspace = true, optional = true }
uuid = { workspace = true, optional = true }
Expand Down
7 changes: 7 additions & 0 deletions sqlx-core/src/types/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,13 @@ pub mod chrono {
};
}

#[cfg(feature = "jiff")]
#[cfg_attr(docsrs, doc(cfg(feature = "jiff")))]
pub mod jiff {
#[doc(no_inline)]
pub use jiff::{civil::Date, civil::DateTime, civil::Time, Timestamp};
}

#[cfg(feature = "bit-vec")]
#[cfg_attr(docsrs, doc(cfg(feature = "bit-vec")))]
#[doc(no_inline)]
Expand Down
4 changes: 3 additions & 1 deletion sqlx-postgres/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ bigdecimal = ["dep:bigdecimal", "dep:num-bigint", "sqlx-core/bigdecimal"]
bit-vec = ["dep:bit-vec", "sqlx-core/bit-vec"]
chrono = ["dep:chrono", "sqlx-core/chrono"]
ipnetwork = ["dep:ipnetwork", "sqlx-core/ipnetwork"]
jiff = ["dep:jiff", "sqlx-core/jiff"]
mac_address = ["dep:mac_address", "sqlx-core/mac_address"]
rust_decimal = ["dep:rust_decimal", "rust_decimal/maths", "sqlx-core/rust_decimal"]
time = ["dep:time", "sqlx-core/time"]
Expand All @@ -35,7 +36,7 @@ futures-util = { version = "0.3.19", default-features = false, features = ["allo
# Cryptographic Primitives
crc = "3.0.0"
hkdf = "0.12.0"
hmac = { version = "0.12.0", default-features = false, features = ["reset"]}
hmac = { version = "0.12.0", default-features = false, features = ["reset"] }
md-5 = { version = "0.10.0", default-features = false }
rand = { version = "0.8.4", default-features = false, features = ["std", "std_rng"] }
sha2 = { version = "0.10.0", default-features = false }
Expand All @@ -45,6 +46,7 @@ bigdecimal = { workspace = true, optional = true }
bit-vec = { workspace = true, optional = true }
chrono = { workspace = true, optional = true }
ipnetwork = { workspace = true, optional = true }
jiff = { workspace = true, optional = true }
mac_address = { workspace = true, optional = true }
rust_decimal = { workspace = true, optional = true }
time = { workspace = true, optional = true }
Expand Down
87 changes: 87 additions & 0 deletions sqlx-postgres/src/types/interval.rs
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,58 @@ impl TryFrom<std::time::Duration> for PgInterval {
}
}

#[cfg(feature = "jiff")]
impl Type<Postgres> for jiff::SignedDuration {
fn type_info() -> PgTypeInfo {
PgTypeInfo::INTERVAL
}
}

#[cfg(feature = "jiff")]
impl PgHasArrayType for jiff::SignedDuration {
fn array_type_info() -> PgTypeInfo {
PgTypeInfo::INTERVAL_ARRAY
}
}

#[cfg(feature = "jiff")]
impl Encode<'_, Postgres> for jiff::SignedDuration {
fn encode_by_ref(&self, buf: &mut PgArgumentBuffer) -> Result<IsNull, BoxDynError> {
let pg_interval = PgInterval::try_from(*self)?;
pg_interval.encode_by_ref(buf)
}

fn size_hint(&self) -> usize {
2 * mem::size_of::<i64>()
}
}

#[cfg(feature = "jiff")]
impl TryFrom<jiff::SignedDuration> for PgInterval {
type Error = BoxDynError;

/// Convert a `jiff::SignedDuration` to a `PgInterval`.
///
/// This returns an error if there is a loss of precision using nanoseconds or if there is a
/// microseconds overflow.
fn try_from(value: jiff::SignedDuration) -> Result<Self, BoxDynError> {
if value.subsec_nanos() % 1000 != 0 {
return Err("PostgreSQL `INTERVAL` does not support nanoseconds precision".into());
}

let micros = value.as_micros();
if micros >= i64::MIN as i128 && micros <= i64::MAX as i128 {
Ok(Self {
months: 0,
days: 0,
microseconds: micros as i64,
})
} else {
Err("Overflow has occurred for PostgreSQL `INTERVAL`".into())
}
}
}

#[cfg(feature = "chrono")]
impl Type<Postgres> for chrono::Duration {
fn type_info() -> PgTypeInfo {
Expand Down Expand Up @@ -330,6 +382,41 @@ fn test_pginterval_std() {
assert!(PgInterval::try_from(std::time::Duration::from_secs(20_000_000_000_000)).is_err());
}

#[test]
#[cfg(feature = "jiff")]
fn test_pginterval_jiff() {
// Case for positive duration
let interval = PgInterval {
days: 0,
months: 0,
microseconds: 27_000,
};
assert_eq!(
&PgInterval::try_from(jiff::SignedDuration::from_micros(27_000)).unwrap(),
&interval
);

// Case for negative duration
let interval = PgInterval {
days: 0,
months: 0,
microseconds: -27_000,
};
assert_eq!(
&PgInterval::try_from(jiff::SignedDuration::from_micros(-27_000)).unwrap(),
&interval
);

// Case when precision loss occurs
assert!(PgInterval::try_from(jiff::SignedDuration::from_nanos(27_000_001)).is_err());
assert!(PgInterval::try_from(jiff::SignedDuration::from_nanos(-27_000_001)).is_err());

// Case when microseconds overflow occurs
assert!(PgInterval::try_from(jiff::SignedDuration::from_secs(10_000_000_000_000)).is_err());
assert!(PgInterval::try_from(jiff::SignedDuration::from_secs(-10_000_000_000_000)).is_err());
}


#[test]
#[cfg(feature = "chrono")]
fn test_pginterval_chrono() {
Expand Down
52 changes: 52 additions & 0 deletions sqlx-postgres/src/types/jiff/date.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
use std::mem;

use crate::decode::Decode;
use crate::encode::{Encode, IsNull};
use crate::error::BoxDynError;
use crate::types::Type;
use crate::{PgArgumentBuffer, PgHasArrayType, PgTypeInfo, PgValueFormat, PgValueRef, Postgres};
use jiff::civil::Date;

impl Type<Postgres> for Date {
fn type_info() -> PgTypeInfo {
PgTypeInfo::DATE
}
}

impl PgHasArrayType for Date {
fn array_type_info() -> PgTypeInfo {
PgTypeInfo::DATE_ARRAY
}
}

impl Encode<'_, Postgres> for Date {
fn encode_by_ref(&self, buf: &mut PgArgumentBuffer) -> Result<IsNull, BoxDynError> {
// DATE is encoded as the days since epoch
let days = (*self - postgres_epoch_date()).get_days();
Encode::<Postgres>::encode(days, buf)
}

fn size_hint(&self) -> usize {
mem::size_of::<i32>()
}
}

impl<'r> Decode<'r, Postgres> for Date {
fn decode(value: PgValueRef<'r>) -> Result<Self, BoxDynError> {
Ok(match value.format() {
PgValueFormat::Binary => {
// DATE is encoded as the days since epoch
let days: i32 = Decode::<Postgres>::decode(value)?;
let days = jiff::Span::new()
.try_days(days)
.map_err(|err| format!("value {days} overflow Postgres DATE: {err:?}"))?;
postgres_epoch_date() + days
}
PgValueFormat::Text => Date::strptime("%Y-%m-%d", value.as_str()?)?,
})
}
}

const fn postgres_epoch_date() -> Date {
Date::constant(2000, 1, 1)
}
104 changes: 104 additions & 0 deletions sqlx-postgres/src/types/jiff/datetime.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
use crate::decode::Decode;
use crate::encode::{Encode, IsNull};
use crate::error::BoxDynError;
use crate::types::Type;
use crate::{PgArgumentBuffer, PgHasArrayType, PgTypeInfo, PgValueFormat, PgValueRef, Postgres};
use jiff::civil::DateTime;
use jiff::tz::{Offset, TimeZone};
use jiff::{SignedDuration, Timestamp};
use std::mem;
use std::str::FromStr;

impl Type<Postgres> for DateTime {
fn type_info() -> PgTypeInfo {
PgTypeInfo::TIMESTAMP
}
}

impl Type<Postgres> for Timestamp {
fn type_info() -> PgTypeInfo {
PgTypeInfo::TIMESTAMPTZ
}
}

impl PgHasArrayType for DateTime {
fn array_type_info() -> PgTypeInfo {
PgTypeInfo::TIMESTAMP_ARRAY
}
}

impl PgHasArrayType for Timestamp {
fn array_type_info() -> PgTypeInfo {
PgTypeInfo::TIMESTAMPTZ_ARRAY
}
}

impl Encode<'_, Postgres> for DateTime {
fn encode_by_ref(&self, buf: &mut PgArgumentBuffer) -> Result<IsNull, BoxDynError> {
// TIMESTAMP is encoded as the microseconds since the epoch
let micros = (*self - postgres_epoch_datetime()).get_microseconds();
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorrect arith, should probably use:

        let micros = self
            .0
            .duration_since(postgres_epoch_timestamp())
            .as_micros();
        if micros > i64::MAX as i128 || micros < i64::MIN as i128 {
            return Err(format!("Timestamp {} out of range for Postgres: {micros}", self.0).into());
        }
        Encode::<Postgres>::encode(micros as i64, buf)

Encode::<Postgres>::encode(micros, buf)
}

fn size_hint(&self) -> usize {
mem::size_of::<i64>()
}
}

#[derive(Debug, thiserror::Error)]
#[error("error parsing datetime {squashed:?}")]
struct ParseError {
squashed: Vec<jiff::Error>,
}

impl<'r> Decode<'r, Postgres> for DateTime {
fn decode(value: PgValueRef<'r>) -> Result<Self, BoxDynError> {
match value.format() {
PgValueFormat::Binary => {
// TIMESTAMP is encoded as the microseconds since the epoch
let us = Decode::<Postgres>::decode(value)?;
Ok(postgres_epoch_datetime() + SignedDuration::from_micros(us))
}
PgValueFormat::Text => {
let input = value.as_str()?;
let mut squashed = vec![];
match DateTime::strptime("%Y-%m-%d %H:%M:%S%.f", input) {
Ok(datetime) => return Ok(datetime),
Err(err) => squashed.push(err),
}
match DateTime::strptime("%Y-%m-%d %H:%M:%S%.f%#z", input) {
Ok(datetime) => return Ok(datetime),
Err(err) => squashed.push(err),
}
Err(Box::new(ParseError { squashed }))
}
}
}
}

impl Encode<'_, Postgres> for Timestamp {
fn encode_by_ref(&self, buf: &mut PgArgumentBuffer) -> Result<IsNull, BoxDynError> {
let datetime = Offset::UTC.to_datetime(*self);
Encode::<Postgres>::encode(datetime, buf)
}

fn size_hint(&self) -> usize {
mem::size_of::<i64>()
}
}

impl<'r> Decode<'r, Postgres> for Timestamp {
fn decode(value: PgValueRef<'r>) -> Result<Self, BoxDynError> {
Ok(match value.format() {
PgValueFormat::Binary => {
let naive = <DateTime as Decode<Postgres>>::decode(value)?;
naive.to_zoned(TimeZone::UTC)?.timestamp()
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BurntSushi Is this the proper way to convert DateTime to Timestamp? I don't find a direct way though.

Copy link

@BurntSushi BurntSushi Sep 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no direct way because there is no such thing as a general way of converting a civil datetime to a timestamp. The former is naive or civil or local time, where as the latter is a precise instant in time. The only way to convert the former to the latter is with an offset. In this particular case, you've chose an offset of 0 or UTC. Whether that's correct or not depends on the meaning of naive from PostgreSQL.

I think you have things backwards here. postgres_epoch_datetime should probably return a Timestamp. And then you should use that to implement Decode<'r, Postgres> for Timestamp, and not involve civil times at all.

Then have a different function that returns the base for civil time.

See https://github.com/sfackler/rust-postgres/pull/1164/files for how it's done there. You should be able to copy that same conceptual approach.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes correct. Thank you.

}
PgValueFormat::Text => Timestamp::from_str(value.as_str()?)?,
})
}
}

const fn postgres_epoch_datetime() -> DateTime {
DateTime::constant(2000, 1, 1, 0, 0, 0, 0)
}
3 changes: 3 additions & 0 deletions sqlx-postgres/src/types/jiff/mod.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
mod date;
mod datetime;
mod time;
Loading