Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementation of #[repr(packed(n))] RFC 1399. #48528

Merged
merged 1 commit into from
Apr 12, 2018

Conversation

bitshifter
Copy link
Contributor

Tracking issue #33158.

@rust-highfive
Copy link
Collaborator

r? @petrochenkov

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Feb 25, 2018
@petrochenkov
Copy link
Contributor

r? @eddyb

@rust-highfive rust-highfive assigned eddyb and unassigned petrochenkov Feb 25, 2018
@gnzlbg
Copy link
Contributor

gnzlbg commented Feb 26, 2018

There are no tests mixing packed + repr(C).
EDIT: there is one test mixing these.

@bitshifter
Copy link
Contributor Author

bitshifter commented Feb 26, 2018

You found the one in print-size-types :) Do you think I should add more though? It's not very thoroughly tested.

@retep998
Copy link
Member

It's always better to err on the side of too many tests.

@gnzlbg
Copy link
Contributor

gnzlbg commented Feb 26, 2018

@bitshifter #[repr(C)] + #[repr(packed(...))] is going to be heavily used by fundamental C-FFI libraries (winapi, libc, mach, ...). If this breaks, one gets ABI incompatibilities that are very hard to debug (they are essentially undefined behavior), and a lot of crates depend on these crates.

I think it would be a good idea for the tests to do their best to try to break the repr(C) + repr(packed) combination. I don't know about winapi, but at least in libc and mach, repr(C) + repr(packed(4)) is common. I don't know if there are some repr(C) + repr(packed(..)) + repr(align(...)) somewhere there, but there might be.


FWIW I'd like repr(packed) to be on a fast track to stabilization. MacOSX kernel headers rely heavily on #pragma pack(4) and without something like packed nothing really works.

@retep998
Copy link
Member

I don't know if there are some repr(C) + repr(packed(..)) + repr(align(...)) somewhere there, but there might be.

Currently repr(packed) and repr(align) cannot be specified on the same type, nor can a repr(packed) type transitively contain repr(align) types so the only tests for this combination would be compile fail tests.

@bitshifter
Copy link
Contributor Author

bitshifter commented Feb 26, 2018

You can put a [repr(packed)] struct inside a [repr(align(...))]. There is a test for that, but not for the new [repr(packed(n))]. I will add some more tests combining [repr(C)]. I am a bit worried there will be a combinatorial explosion of tests combining repr C, packed and packed(n).

@@ -974,7 +979,8 @@ impl<'a, 'tcx> LayoutCx<'tcx, TyCtxt<'a, 'tcx, 'tcx>> {
let mut inverse_memory_index: Vec<u32> = (0..fields.len() as u32).collect();

// Anything with repr(C) or repr(packed) doesn't optimize.
let mut optimize = (repr.flags & ReprFlags::IS_UNOPTIMISABLE).is_empty();
let mut optimize = (repr.flags & ReprFlags::IS_UNOPTIMISABLE).is_empty()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that this is a bit more complex, I think I'll pull this out to a method on ReprOptions, something like inhibit_field_reordering_opt.

@bitshifter bitshifter force-pushed the repr_packed branch 3 times, most recently from bf94f97 to c5af68a Compare March 1, 2018 22:10
// rustc::ty::layout::Align restricts align to <= 2147483647
if align <= 2147483647 {
acc.push(ReprAlign(align as u32));
if *literal <= 2147483647 {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to LLVM docs the maximum alignment on alloca, load, store etc is 1 << 29 (e.g. https://llvm.org/docs/LangRef.html#id185). Perhaps we should enforce that restriction here in case anyone tries to use alignments that LLVM doesn't support.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eddyb do think it would be a good idea to change this max value to 1 << 29? I'll do it as a separate PR if so - don't want to jeopardise my position in the homu queue!

@bitshifter
Copy link
Contributor Author

@gnzlbg I've added a bunch more tests combining repr C and packed. Let me know if there's anything specific you think needs better test coverage.

@bors
Copy link
Contributor

bors commented Mar 2, 2018

☔ The latest upstream changes (presumably #48653) made this pull request unmergeable. Please resolve the merge conflicts.

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 2, 2018
let mut align_error = None;
if let ast::LitKind::Int(align, ast::LitIntType::Unsuffixed) = value.node {
if align.is_power_of_two() {
let parse_int_literal = |node: &ast::LitKind| -> Result<u32, &'static str> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be parse_alignment or similar.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@gnzlbg
Copy link
Contributor

gnzlbg commented Mar 2, 2018

@bitshifter thanks! the new tests cover the cases that are relevant for libc and mach :)

@bors
Copy link
Contributor

bors commented Mar 9, 2018

☔ The latest upstream changes (presumably #48860) made this pull request unmergeable. Please resolve the merge conflicts.

@bors
Copy link
Contributor

bors commented Mar 16, 2018

☔ The latest upstream changes (presumably #49051) made this pull request unmergeable. Please resolve the merge conflicts.

@kennytm kennytm added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 27, 2018
@bitshifter
Copy link
Contributor Author

I couldn't repro it building my branch via msys2 and msvc, will try again after merging latest master.

@bitshifter
Copy link
Contributor Author

bitshifter commented Mar 27, 2018

Still didn't get this failure after merging latest master. I have no idea why this test has failed. It appears to have just exited with a non-zero code (due to a panic) but nothing in stdout or stderr. I'll follow up on IRC when I have time, probably over the long weekend here.

@shepmaster
Copy link
Member

Ping from triage, @bitshifter ! Will you have time to work on this soon?

@bitshifter
Copy link
Contributor Author

bitshifter commented Apr 7, 2018

Current status is I'm stuck on this. I haven't had the time and energy to proactively ask for help on IRC and a lot of people were at the all hands or in transit last weekend anyway.

I have not been able to reproduce this test locally after rebuilding about half a dozen times. I am also unable to see any reason why my changes would break mir-opt. My gut feeling is this is a spurious test fail unrelated to this PR and I'd like to see if I second run through Homu gets the same failure.

In the process of chasing this I've been failing to get my PR or even master to complete a full test pass under msys2/msvc. I had a few fails due to path length and after moving my check out even master always fails on run-make-fulldeps/target-specs. So perhaps there is some difference in my msys2 environment and what is being used on homu to cause this and the other fail that I can't reproduce? My msys2 is basically up to date. My MSVC is the latest 2017, running Windows 10 home edition. Anyway I still intend to chase this up on IRC, hopefully sometime over the next week. If someone wanted to try running it through homu again that would be great.

@gnzlbg
Copy link
Contributor

gnzlbg commented Apr 10, 2018

@bitshifter which arch is exactly failing here? Maybe we could involve someone interested / experienced with this architecture to help fix this one last issue?


cc @alexcrichton ? repr(packed) would significantly improve the way structs are passed to/from C via FFI. The current alternative is to just use a bag of bytes + getters/setters on the Rust side, but this is so painful that at least libc and mach (and probably winapi as well) just prefer to invoke undefined behavior here. . .

@bitshifter
Copy link
Contributor Author

bitshifter commented Apr 10, 2018

@gnzlbg the failing arch is x86_64-pc-windows-msvc. I'm going to download the mingw that's being used by the rust appveyor.yml and see if that makes a difference.

edit: it's using mingw-w64 instead of msys2 so perhaps that will make a difference.

@alexcrichton
Copy link
Member

I checked out this branch locally and it passed on x86_64-pc-windows-msvc (the mir-opt tests), so I'm not sure :(

@bitshifter
Copy link
Contributor Author

I tried building in a mingw-w64 environment like what is being used on appveyor but the test still passed without issue, so still no idea what's going on here.

@bitshifter
Copy link
Contributor Author

@eddyb could you +r this again please to see if it will pass the mir-opt/issue-38669.rs test.

@eddyb
Copy link
Member

eddyb commented Apr 11, 2018

@bors r+

@bors
Copy link
Contributor

bors commented Apr 11, 2018

📌 Commit 15d1c4d has been approved by eddyb

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Apr 11, 2018
@bors
Copy link
Contributor

bors commented Apr 12, 2018

⌛ Testing commit 15d1c4d with merge 4777881...

bors added a commit that referenced this pull request Apr 12, 2018
Implementation of `#[repr(packed(n))]` RFC 1399.

Tracking issue #33158.
@bors
Copy link
Contributor

bors commented Apr 12, 2018

☀️ Test successful - status-appveyor, status-travis
Approved by: eddyb
Pushing 4777881 to master...

@bors bors merged commit 15d1c4d into rust-lang:master Apr 12, 2018
@bors bors mentioned this pull request Apr 12, 2018
8 tasks
@bitshifter bitshifter deleted the repr_packed branch April 12, 2018 08:32
bors added a commit to rust-lang-ci/rust that referenced this pull request May 29, 2024
don't inhibit random field reordering on repr(packed(1))

`inhibit_struct_field_reordering_opt` being false means we exclude this type from random field shuffling. However, `packed(1)` types can still be shuffled! The logic was added in rust-lang#48528 since it's pointless to reorder fields in packed(1) types (there's no padding that could be saved) -- but that shouldn't inhibit `-Zrandomize-layout` (which did not exist at the time).

We could add an optimization elsewhere to not bother sorting the fields for `repr(packed)` types, but I don't think that's worth the effort.

This *does* change the behavior in that we may now reorder fields of `packed(1)` structs (e.g. if there are niches, we'll try to move them to the start/end, according to `NicheBias`).  We were always allowed to do that but so far we didn't. Quoting the [reference](https://doc.rust-lang.org/reference/type-layout.html):

> On their own, align and packed do not provide guarantees about the order of fields in the layout of a struct or the layout of an enum variant, although they may be combined with representations (such as C) which do provide such guarantees.
github-actions bot pushed a commit to rust-lang/miri that referenced this pull request May 30, 2024
don't inhibit random field reordering on repr(packed(1))

`inhibit_struct_field_reordering_opt` being false means we exclude this type from random field shuffling. However, `packed(1)` types can still be shuffled! The logic was added in rust-lang/rust#48528 since it's pointless to reorder fields in packed(1) types (there's no padding that could be saved) -- but that shouldn't inhibit `-Zrandomize-layout` (which did not exist at the time).

We could add an optimization elsewhere to not bother sorting the fields for `repr(packed)` types, but I don't think that's worth the effort.

This *does* change the behavior in that we may now reorder fields of `packed(1)` structs (e.g. if there are niches, we'll try to move them to the start/end, according to `NicheBias`).  We were always allowed to do that but so far we didn't. Quoting the [reference](https://doc.rust-lang.org/reference/type-layout.html):

> On their own, align and packed do not provide guarantees about the order of fields in the layout of a struct or the layout of an enum variant, although they may be combined with representations (such as C) which do provide such guarantees.
flip1995 pushed a commit to flip1995/rust-clippy that referenced this pull request May 30, 2024
don't inhibit random field reordering on repr(packed(1))

`inhibit_struct_field_reordering_opt` being false means we exclude this type from random field shuffling. However, `packed(1)` types can still be shuffled! The logic was added in rust-lang/rust#48528 since it's pointless to reorder fields in packed(1) types (there's no padding that could be saved) -- but that shouldn't inhibit `-Zrandomize-layout` (which did not exist at the time).

We could add an optimization elsewhere to not bother sorting the fields for `repr(packed)` types, but I don't think that's worth the effort.

This *does* change the behavior in that we may now reorder fields of `packed(1)` structs (e.g. if there are niches, we'll try to move them to the start/end, according to `NicheBias`).  We were always allowed to do that but so far we didn't. Quoting the [reference](https://doc.rust-lang.org/reference/type-layout.html):

> On their own, align and packed do not provide guarantees about the order of fields in the layout of a struct or the layout of an enum variant, although they may be combined with representations (such as C) which do provide such guarantees.
bors added a commit to rust-lang/rust-analyzer that referenced this pull request Jun 20, 2024
don't inhibit random field reordering on repr(packed(1))

`inhibit_struct_field_reordering_opt` being false means we exclude this type from random field shuffling. However, `packed(1)` types can still be shuffled! The logic was added in rust-lang/rust#48528 since it's pointless to reorder fields in packed(1) types (there's no padding that could be saved) -- but that shouldn't inhibit `-Zrandomize-layout` (which did not exist at the time).

We could add an optimization elsewhere to not bother sorting the fields for `repr(packed)` types, but I don't think that's worth the effort.

This *does* change the behavior in that we may now reorder fields of `packed(1)` structs (e.g. if there are niches, we'll try to move them to the start/end, according to `NicheBias`).  We were always allowed to do that but so far we didn't. Quoting the [reference](https://doc.rust-lang.org/reference/type-layout.html):

> On their own, align and packed do not provide guarantees about the order of fields in the layout of a struct or the layout of an enum variant, although they may be combined with representations (such as C) which do provide such guarantees.
flip1995 pushed a commit to flip1995/rust-clippy that referenced this pull request Jun 28, 2024
don't inhibit random field reordering on repr(packed(1))

`inhibit_struct_field_reordering_opt` being false means we exclude this type from random field shuffling. However, `packed(1)` types can still be shuffled! The logic was added in rust-lang/rust#48528 since it's pointless to reorder fields in packed(1) types (there's no padding that could be saved) -- but that shouldn't inhibit `-Zrandomize-layout` (which did not exist at the time).

We could add an optimization elsewhere to not bother sorting the fields for `repr(packed)` types, but I don't think that's worth the effort.

This *does* change the behavior in that we may now reorder fields of `packed(1)` structs (e.g. if there are niches, we'll try to move them to the start/end, according to `NicheBias`).  We were always allowed to do that but so far we didn't. Quoting the [reference](https://doc.rust-lang.org/reference/type-layout.html):

> On their own, align and packed do not provide guarantees about the order of fields in the layout of a struct or the layout of an enum variant, although they may be combined with representations (such as C) which do provide such guarantees.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants