Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ensure JSON-defined targets are consistent #133409

Merged
merged 2 commits into from
Nov 29, 2024

Conversation

RalfJung
Copy link
Member

@RalfJung RalfJung commented Nov 24, 2024

We have a check_consistency check that ensures some invariants which (presumably) the rest of the compiler relies on. However, JSON targets can easily be written in a way that violates those invariants. So this PR applies the same consistency check to JSON targets that we already enforce for built-in targets.

I have converted many of the assertions in that function to new macros that show a nice error instead of a panic; if people are okay with the general approach here, I can do that for the rest of the checks as well.

@rustbot
Copy link
Collaborator

rustbot commented Nov 24, 2024

r? @lcnr

rustbot has assigned @lcnr.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot
Copy link
Collaborator

rustbot commented Nov 24, 2024

These commits modify compiler targets.
(See the Target Tier Policy.)

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Nov 24, 2024
@RalfJung RalfJung changed the title Target consistency ensure JSON-defined targets are consistent Nov 24, 2024
@rust-log-analyzer

This comment has been minimized.

@RalfJung RalfJung force-pushed the target-consistency branch 3 times, most recently from 3be78e5 to 1628e60 Compare November 24, 2024 08:12
@rust-log-analyzer

This comment has been minimized.

Copy link
Contributor

@lcnr lcnr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r=me after nits

@RalfJung RalfJung force-pushed the target-consistency branch 2 times, most recently from ac22231 to 5dbca06 Compare November 25, 2024 15:44
@RalfJung
Copy link
Member Author

This will conflict with #133411 so let's let that one land first.

@rust-log-analyzer

This comment has been minimized.

@petrochenkov
Copy link
Contributor

We have a check_consistency check that ensures some invariants which (presumably) the rest of the compiler relies on.

Not really, it's a sort of linting to avoid accidental mistakes in target specs.

However, JSON targets can easily be written in a way that violates those invariants.

And that's entirely fine, the check was intended for built-in targets, and all the exception cases also taken only from built-in targets (and documented so we know why we diverge from the default in some specific cases).
I don't think we should report errors if some json spec doesn't pass the check.

Although, not all of the checks are the same.
For some it's hard to imagine a target that would break them, but many are just default guidelines.

@RalfJung
Copy link
Member Author

#133410 is something I'd definitely like to rely on in other parts of the compiler.

It's hard to say which of the checks here are load-bearing vs just indicating something "unusual". It's also easy to imagine parts of the compiler seeing these invariants enshrined in the consistency checks, and beginning to rely on them.

@bors
Copy link
Contributor

bors commented Nov 26, 2024

☔ The latest upstream changes (presumably #133500) made this pull request unmergeable. Please resolve the merge conflicts.

@RalfJung RalfJung force-pushed the target-consistency branch 2 times, most recently from ab740ea to 3ef83e2 Compare November 26, 2024 22:34
@RalfJung
Copy link
Member Author

I've now made it so that all the dynamic linker checks are skipped for JSON targets. Many of them were anyway already skipped on some targets, so it seems like those are more of the form "this makes an odd target" than "this makes an invalid target".

@lcnr
Copy link
Contributor

lcnr commented Nov 27, 2024

maybe move the consistency tests into a consistency_checks submodule 🤔

IMO if we want to split up this file we should move the JSON parsing, that's a lot bigger.

also fine with me, just don't like big files :<

@RalfJung
Copy link
Member Author

Yeah me neither. But the file is already huge before this PR...

@RalfJung
Copy link
Member Author

Well I did the move in this PR. The thumbs-up didn't trigger a notification so I figured maybe you want me to do it in this PR.

@rustbot ready

@lcnr
Copy link
Contributor

lcnr commented Nov 28, 2024

ah, damn, I didn't actually send a previous comment here 😅 well, paraphrasing

fn check_consistency is supposed to do two things:

  • check that our builtin targets don't have any weird settings, which is pretty much just a lint, which has been expressed by @petrochenkov
  • check settings for consistency which may be relied on by the compiler, these should hard error

I am not sure whether the linker stuff is the only thing which doesn't belong in the second category, but given that this is unstable we can move anything else that's actually just "weird, not wrong" to also only be checked for builtin targets later. With this

@bors r+ rollup=iffy

@bors
Copy link
Contributor

bors commented Nov 28, 2024

📌 Commit 77080d8 has been approved by lcnr

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 28, 2024
@bors
Copy link
Contributor

bors commented Nov 28, 2024

⌛ Testing commit 77080d8 with merge 188f378...

bors added a commit to rust-lang-ci/rust that referenced this pull request Nov 28, 2024
ensure JSON-defined targets are consistent

We have a `check_consistency` check that ensures some invariants which (presumably) the rest of the compiler relies on. However, JSON targets can easily be written in a way that violates those invariants. So this PR applies the same consistency check to JSON targets that we already enforce for built-in targets.

I have converted many of the assertions in that function to new macros that show a nice error instead of a panic; if people are okay with the general approach here, I can do that for the rest of the checks as well.
@rust-log-analyzer
Copy link
Collaborator

The job dist-x86_64-msvc failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
   Compiling rustc_driver v0.0.0 (C:\a\rust\rust\compiler\rustc_driver)
[RUSTC-TIMING] rustc_driver test:false 4.669
error: linking with `link.exe` failed: exit code: 1104
  |
  = note: "C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.42.34433\\bin\\HostX64\\x64\\link.exe" "/NOLOGO" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\symbols.o" "C:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage1-rustc\\x86_64-pc-windows-msvc\\release\\deps\\rustc_main-dd45d38ca289c647.rustc_main.c54ffc24bc16dfbe-cgu.0.rcgu.o" "C:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage1-rustc\\x86_64-pc-windows-msvc\\release\\deps\\rustc_driver-1034d7feaa0a4c5d.dll.lib" "C:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage1\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcompiler_builtins-c25bba97e86b102c.rlib" "psapi.lib" "shell32.lib" "ole32.lib" "uuid.lib" "advapi32.lib" "ws2_32.lib" "ntdll.lib" "kernel32.lib" "advapi32.lib" "ole32.lib" "oleaut32.lib" "advapi32.lib" "cfgmgr32.lib" "gdi32.lib" "kernel32.lib" "msimg32.lib" "opengl32.lib" "user32.lib" "winspool.lib" "bcrypt.lib" "advapi32.lib" "legacy_stdio_definitions.lib" "kernel32.lib" "kernel32.lib" "advapi32.lib" "ntdll.lib" "userenv.lib" "ws2_32.lib" "dbghelp.lib" "/defaultlib:libcmt" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\advapi32.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\api-ms-win-core-errorhandling-l1-1-3.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\api-ms-win-core-file-fromapp-l1-1-0.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\api-ms-win-core-handle-l1-1-0.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\api-ms-win-core-ioring-l1-1-0.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\api-ms-win-core-memory-l1-1-3.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\api-ms-win-core-memory-l1-1-4.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\api-ms-win-core-memory-l1-1-5.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\api-ms-win-core-memory-l1-1-6.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\api-ms-win-core-memory-l1-1-7.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\api-ms-win-core-memory-l1-1-8.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\api-ms-win-core-synch-l1-2-0.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\api-ms-win-core-sysinfo-l1-2-0.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\api-ms-win-core-sysinfo-l1-2-3.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\api-ms-win-core-sysinfo-l1-2-4.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\api-ms-win-core-sysinfo-l1-2-6.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\api-ms-win-core-util-l1-1-1.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\api-ms-win-core-winrt-error-l1-1-0.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\api-ms-win-core-winrt-l1-1-0.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\api-ms-win-core-wow64-l1-1-1.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\api-ms-win-security-base-l1-2-2.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\avrt.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\bcp47mrm.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\bcryptprimitives.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\clfsw32.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\dbghelp.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\elscore.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\gdi32.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\icu.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\imagehlp.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\kernel32.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\ktmw32.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\netapi32.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\normaliz.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\ntdll.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\ntdllk.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\ole32.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\oleacc.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\oleaut32.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\propsys.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\psapi.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\rtworkq.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\txfw32.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\user32.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\usp10.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\version.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustchdrOV3\\wofutil.dll_imports_indirect.lib" "/NXCOMPAT" "/LIBPATH:C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.42.34433\\atlmfc\\lib\\x64" "/LIBPATH:C:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage1-rustc\\x86_64-pc-windows-msvc\\release\\build\\stacker-51665ada3368c7c5\\out" "/LIBPATH:C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.42.34433\\atlmfc\\lib\\x64" "/LIBPATH:C:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage1-rustc\\x86_64-pc-windows-msvc\\release\\build\\psm-18bbb9b2c1f4a85c\\out" "/LIBPATH:C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.42.34433\\atlmfc\\lib\\x64" "/LIBPATH:C:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage1-rustc\\x86_64-pc-windows-msvc\\release\\build\\blake3-40658902d69c9924\\out" "/LIBPATH:C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.42.34433\\atlmfc\\lib\\x64" "/LIBPATH:C:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage1-rustc\\x86_64-pc-windows-msvc\\release\\build\\blake3-40658902d69c9924\\out" "/LIBPATH:C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.42.34433\\atlmfc\\lib\\x64" "/LIBPATH:C:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage1-rustc\\x86_64-pc-windows-msvc\\release\\build\\rustc_llvm-2e0139d76762918c\\out" "/LIBPATH:C:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\llvm\\lib" "/OUT:C:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage1-rustc\\x86_64-pc-windows-msvc\\release\\deps\\rustc_main-dd45d38ca289c647.exe" "/OPT:REF,ICF" "/DEBUG" "/PDBALTPATH:%_PDB%" "/MANIFEST:EMBED" "/MANIFESTINPUT:C:\\a\\rust\\rust\\compiler\\rustc\\Windows Manifest.xml" "/WX"
  = note: LINK : fatal error LNK1104: cannot open file 'C:\a\rust\rust\build\x86_64-pc-windows-msvc\stage1-rustc\x86_64-pc-windows-msvc\release\deps\rustc_main-dd45d38ca289c647.exe'␍

[RUSTC-TIMING] rustc_main test:false 0.650
error: could not compile `rustc-main` (bin "rustc-main") due to 1 previous error
Build completed unsuccessfully in 0:07:53

@bors
Copy link
Contributor

bors commented Nov 28, 2024

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Nov 28, 2024
@RalfJung
Copy link
Member Author

RalfJung commented Nov 28, 2024 via email

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 28, 2024
@RalfJung
Copy link
Member Author

That was probably #127883.

@jieyouxu jieyouxu added the CI-spurious-fail-msvc CI spurious failure: target env msvc label Nov 29, 2024
@bors
Copy link
Contributor

bors commented Nov 29, 2024

⌛ Testing commit 77080d8 with merge d6f8829...

@bors
Copy link
Contributor

bors commented Nov 29, 2024

☀️ Test successful - checks-actions
Approved by: lcnr
Pushing d6f8829 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Nov 29, 2024
@bors bors merged commit d6f8829 into rust-lang:master Nov 29, 2024
7 checks passed
@rustbot rustbot added this to the 1.85.0 milestone Nov 29, 2024
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (d6f8829): comparison URL.

Overall result: ✅ improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-0.2% [-0.2%, -0.2%] 2
All ❌✅ (primary) - - 0

Max RSS (memory usage)

Results (primary -2.2%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-2.2% [-2.2%, -2.2%] 1
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) -2.2% [-2.2%, -2.2%] 1

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 774.051s -> 773.667s (-0.05%)
Artifact size: 331.96 MiB -> 331.97 MiB (0.00%)

@RalfJung RalfJung deleted the target-consistency branch November 29, 2024 06:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI-spurious-fail-msvc CI spurious failure: target env msvc merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants