Skip to content

Commit

Permalink
Rollup merge of rust-lang#82047 - the8472:fast-rename, r=davidtwco
Browse files Browse the repository at this point in the history
bypass auto_da_alloc for metadata files

This saves about 0.7% when rerunning the UI test suite. I.e. when the metadata files exist and will be overwritten. No improvements expected for a clean build. So it might show up in incr-patched perf results.
```
regular rename:

Benchmark rust-lang#1: touch src/tools/compiletest/src/main.rs ; RUSTC_WRAPPER="" schedtool -B -e ./x.py test src/test/ui
  Time (mean ± σ):     47.305 s ±  0.170 s    [User: 1631.540 s, System: 412.648 s]
  Range (min … max):   47.125 s … 47.856 s    20 runs

non-durable rename:

Benchmark rust-lang#1: touch src/tools/compiletest/src/main.rs ; RUSTC_WRAPPER="" schedtool -B -e ./x.py test src/test/ui
  Time (mean ± σ):     46.930 s ±  0.064 s    [User: 1634.344 s, System: 396.038 s]
  Range (min … max):   46.759 s … 47.043 s    20 runs
```

There are more places that trigger auto_da_alloc behavior by overwriting existing files with O_TRUNC, but those are much harder to locate because `O_TRUNC` is set on `open()` but the writeback is triggered on `close()`. The latter is the part which shows up in profiles.
  • Loading branch information
Dylan-DPC authored Mar 8, 2021
2 parents 27885a9 + 6a679ff commit f311750
Show file tree
Hide file tree
Showing 2 changed files with 19 additions and 1 deletion.
2 changes: 1 addition & 1 deletion compiler/rustc_interface/src/passes.rs
Original file line number Diff line number Diff line change
Expand Up @@ -988,7 +988,7 @@ fn encode_and_write_metadata(
.unwrap_or_else(|err| tcx.sess.fatal(&format!("couldn't create a temp dir: {}", err)));
let metadata_tmpdir = MaybeTempDir::new(metadata_tmpdir, tcx.sess.opts.cg.save_temps);
let metadata_filename = emit_metadata(tcx.sess, &metadata, &metadata_tmpdir);
if let Err(e) = fs::rename(&metadata_filename, &out_filename) {
if let Err(e) = util::non_durable_rename(&metadata_filename, &out_filename) {
tcx.sess.fatal(&format!("failed to write {}: {}", out_filename.display(), e));
}
if tcx.sess.opts.json_artifact_notifications {
Expand Down
18 changes: 18 additions & 0 deletions compiler/rustc_interface/src/util.rs
Original file line number Diff line number Diff line change
Expand Up @@ -694,6 +694,24 @@ pub fn build_output_filenames(
}
}

#[cfg(not(target_os = "linux"))]
pub fn non_durable_rename(src: &Path, dst: &Path) -> std::io::Result<()> {
std::fs::rename(src, dst)
}

/// This function attempts to bypass the auto_da_alloc heuristic implemented by some filesystems
/// such as btrfs and ext4. When renaming over a file that already exists then they will "helpfully"
/// write back the source file before committing the rename in case a developer forgot some of
/// the fsyncs in the open/write/fsync(file)/rename/fsync(dir) dance for atomic file updates.
///
/// To avoid triggering this heuristic we delete the destination first, if it exists.
/// The cost of an extra syscall is much lower than getting descheduled for the sync IO.
#[cfg(target_os = "linux")]
pub fn non_durable_rename(src: &Path, dst: &Path) -> std::io::Result<()> {
let _ = std::fs::remove_file(dst);
std::fs::rename(src, dst)
}

// Note: Also used by librustdoc, see PR #43348. Consider moving this struct elsewhere.
//
// FIXME: Currently the `everybody_loops` transformation is not applied to:
Expand Down

0 comments on commit f311750

Please sign in to comment.