Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: correct unaligned slice access #827

Merged
merged 1 commit into from
Nov 13, 2024

Conversation

JamesGuthrie
Copy link
Member

@JamesGuthrie JamesGuthrie commented Nov 13, 2024

Tests started failing for an unknown reason. It turns out that we were attempting to build a slice with an unaligned pointer, and slice::from_raw_parts was panicking.

The reason the memory address is unaligned is that the Rust type has an 8-byte alignment requirement, but the Postgres type does not have a configured alignment, so it has the default 4-byte alignment. It is unclear why the type alignment was not configured in Postgres, presumably an oversight.

It appears as though we only end up with unaligned addresses when the types are stored on disk.

The affected code is doing zero-copy deserialization of Rust structs with Postgres-allocated memory. The deserialization code takes care of all internal alignment requirements, and assumes that the memory to be deserialized is aligned. Not all types are affected, only those which contain a slice of 8-byte aligned elements, because the slice is reconstructed with a (potentially 4-byte aligned) address.

This fix detects when we would attempt to deserialize data stored at an un-aligned address, and copies the data to a new (8-byte aligned) memory location, allowing for safe processing. It does not constrain this detection to only types with 8-byte aligned slices.

@JamesGuthrie JamesGuthrie force-pushed the jg/fix-unaligned-slice-accesses branch from 0ea02c8 to 1785bdb Compare November 13, 2024 09:11
@JamesGuthrie
Copy link
Member Author

Note: We should also follow up by setting the postgres type alignment to 'double' for all varlena types which require 8-byte alignment.

@JamesGuthrie JamesGuthrie requested review from zilder and syvb November 13, 2024 10:57
@JamesGuthrie JamesGuthrie force-pushed the jg/fix-unaligned-slice-accesses branch from 1785bdb to cb5c279 Compare November 13, 2024 11:15
Copy link
Contributor

@zilder zilder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work, thank you!

@JamesGuthrie JamesGuthrie force-pushed the jg/fix-unaligned-slice-accesses branch from cb5c279 to d7f9372 Compare November 13, 2024 15:05
Tests started failing for an unknown reason. It turns out that we were
attempting to build a slice with an unaligned pointer, and
`slice::from_raw_parts` was panicking.

The reason the memory address is unaligned is that the Rust type has
an 8-byte alignment requirement, but the Postgres type does not have a
configured alignment, so it has the default 4-byte alignment. It is
unclear why the type alignment was not configured in Postgres,
presumably an oversight.

It appears as though we only end up with unaligned addresses when the
types are stored on disk.

The affected code is doing zero-copy deserialization of Rust structs
with Postgres-allocated memory. The deserialization code takes care of
all internal alignment requirements, and assumes that the memory to be
deserialized is aligned. Not all types are affected, only those which
contain a slice of 8-byte aligned elements, because the slice is
reconstructed with a (potentially 4-byte aligned) address.

This fix detects when we would attempt to deserialize data stored at an
un-aligned address, and copies the data to a new (8-byte aligned) memory
location, allowing for safe processing. It does not constrain this
detection to only types with 8-byte aligned slices.
@JamesGuthrie JamesGuthrie force-pushed the jg/fix-unaligned-slice-accesses branch from d7f9372 to 5bace43 Compare November 13, 2024 15:08
@JamesGuthrie JamesGuthrie merged commit a36e670 into main Nov 13, 2024
6 of 13 checks passed
@JamesGuthrie JamesGuthrie deleted the jg/fix-unaligned-slice-accesses branch November 13, 2024 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants