Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remote-test: use u64 to represent file size #119999

Merged
merged 1 commit into from
Jan 22, 2024

Conversation

onur-ozkan
Copy link
Member

@onur-ozkan onur-ozkan commented Jan 15, 2024

Currently, triggering a transfer of data exceeding the size of 4294967295 bytes results in a panic on the remote-test-server as io::copy(&mut file, dst) failed with Connection reset by peer (os error 104). This issue happens because the size is transmitted as u32 to remote-test-server.

First commit increases the supported file size. But I am not sure about its necessity — can we realistically encounter file sizes exceeding 4GB in builds, perhaps through some complicated configurations?

The second commit adds a sanity check to avoid encountering the error io::copy(&mut file, dst) failed with Connection reset by peer (os error 104) on the remote-test-server side.

@rustbot
Copy link
Collaborator

rustbot commented Jan 15, 2024

r? @Mark-Simulacrum

(rustbot has picked a reviewer for you, use r? to override)

@rustbot rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 15, 2024
@onur-ozkan onur-ozkan changed the title improvements remote-test improvements Jan 15, 2024
@onur-ozkan onur-ozkan marked this pull request as ready for review January 15, 2024 17:52
@onur-ozkan
Copy link
Member Author

First commit increases the supported file size. But I am not sure about its necessity — can we realistically encounter file sizes exceeding 4GB in builds, perhaps through some complicated configurations?

Let me know if this isn't needed (so I can revert).

@Mark-Simulacrum
Copy link
Member

I don't think we need to support large files, though I also don't mind supporting them.

That said, if we're touching this code I think rather than manually bit twiddling using https://doc.rust-lang.org/nightly/std/primitive.u32.html#method.to_le_bytes and correspondign from_le_bytes feels much better to me.

@Mark-Simulacrum Mark-Simulacrum added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 20, 2024
@onur-ozkan onur-ozkan force-pushed the remote-test-tools branch 2 times, most recently from 9ae2ecf to 84fe1eb Compare January 21, 2024 09:07
@onur-ozkan
Copy link
Member Author

The second commit adds a sanity check to avoid encountering the error io::copy(&mut file, dst) failed with Connection reset by peer (os error 104) on the remote-test-server side.

Just realized that there is no need for a sanity check on the size anymore since we transmit the size without any conversion.

Updated the manual bit-twiddling part to use to_be_bytes on the client side (server side already uses from_be_bytes).

@rustbot ready

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jan 21, 2024
@onur-ozkan onur-ozkan changed the title remote-test improvements remote-test: use u64 to represent file size Jan 21, 2024
let mut amt = 0u64;
for (i, &byte) in header[1..=8].iter().enumerate() {
amt |= (byte as u64) << (56 - 8 * i);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

u64::from_be_bytes(header[1..9].try_into().unwrap())

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 21, 2024
This increases the maximum supported file size (previously limited to 4GB)
and avoids potential issues with u32 to u64 conversions, which are no longer
needed.

Signed-off-by: onur-ozkan <[email protected]>
@onur-ozkan onur-ozkan added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jan 21, 2024
@Mark-Simulacrum
Copy link
Member

@bors r+

@bors
Copy link
Contributor

bors commented Jan 21, 2024

📌 Commit 1afd216 has been approved by Mark-Simulacrum

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 21, 2024
TaKO8Ki added a commit to TaKO8Ki/rust that referenced this pull request Jan 21, 2024
…rk-Simulacrum

remote-test: use u64 to represent file size

Currently, triggering a transfer of data exceeding the size of 4294967295 bytes results in a panic on the `remote-test-server` as `io::copy(&mut file, dst) failed with Connection reset by peer (os error 104)`. This issue happens because the size is transmitted as u32 to `remote-test-server`.

First commit increases the supported file size. But I am not sure about its necessity — can we realistically encounter file sizes exceeding 4GB in builds, perhaps through some complicated configurations?

~The second commit adds a sanity check to avoid encountering the error `io::copy(&mut file, dst) failed with Connection reset by peer (os error 104)` on the `remote-test-server` side.~
bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 21, 2024
Rollup of 12 pull requests

Successful merges:

 - rust-lang#118639 (Undeprecate lint `unstable_features` and make use of it in the compiler)
 - rust-lang#118714 ( Explanation that fields are being used when deriving `(Partial)Ord` on enums)
 - rust-lang#119801 (Fix deallocation with wrong allocator in (A)Rc::from_box_in)
 - rust-lang#119948 (Make `unsafe_op_in_unsafe_fn` migrated in edition 2024)
 - rust-lang#119999 (remote-test: use u64 to represent file size)
 - rust-lang#120058 (bootstrap: improvements for compiler builds)
 - rust-lang#120160 (Manually implement derived `NonZero` traits.)
 - rust-lang#120177 (Remove duplicate dependencies for rustc)
 - rust-lang#120185 (coverage: Don't instrument `#[automatically_derived]` functions)
 - rust-lang#120194 (Shorten `#[must_use]` Diagnostic Message for `Option::is_none`)
 - rust-lang#120200 (Correct the anchor of an URL in an error message)
 - rust-lang#120203 (Replace `#!/bin/bash` with `#!/usr/bin/env bash` in rust-installer tests)

r? `@ghost`
`@rustbot` modify labels: rollup
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Jan 22, 2024
…rk-Simulacrum

remote-test: use u64 to represent file size

Currently, triggering a transfer of data exceeding the size of 4294967295 bytes results in a panic on the `remote-test-server` as `io::copy(&mut file, dst) failed with Connection reset by peer (os error 104)`. This issue happens because the size is transmitted as u32 to `remote-test-server`.

First commit increases the supported file size. But I am not sure about its necessity — can we realistically encounter file sizes exceeding 4GB in builds, perhaps through some complicated configurations?

~The second commit adds a sanity check to avoid encountering the error `io::copy(&mut file, dst) failed with Connection reset by peer (os error 104)` on the `remote-test-server` side.~
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Jan 22, 2024
…rk-Simulacrum

remote-test: use u64 to represent file size

Currently, triggering a transfer of data exceeding the size of 4294967295 bytes results in a panic on the `remote-test-server` as `io::copy(&mut file, dst) failed with Connection reset by peer (os error 104)`. This issue happens because the size is transmitted as u32 to `remote-test-server`.

First commit increases the supported file size. But I am not sure about its necessity — can we realistically encounter file sizes exceeding 4GB in builds, perhaps through some complicated configurations?

~The second commit adds a sanity check to avoid encountering the error `io::copy(&mut file, dst) failed with Connection reset by peer (os error 104)` on the `remote-test-server` side.~
bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 22, 2024
…iaskrgr

Rollup of 9 pull requests

Successful merges:

 - rust-lang#118578 (core: introduce split_at{,_mut}_checked)
 - rust-lang#119369 (exclude unexported macro bindings from extern crate)
 - rust-lang#119408 (xous: misc fixes + add network support)
 - rust-lang#119943 (std::net: bind update for using backlog as `-1` too.)
 - rust-lang#119948 (Make `unsafe_op_in_unsafe_fn` migrated in edition 2024)
 - rust-lang#119999 (remote-test: use u64 to represent file size)
 - rust-lang#120152 (add help message for `exclusive_range_pattern` error)
 - rust-lang#120213 (Don't actually make bound ty/const for RTN)
 - rust-lang#120225 (Fix -Zremap-path-scope typo)

Failed merges:

 - rust-lang#119972 (Add `ErrCode`)

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit 9e896f4 into rust-lang:master Jan 22, 2024
11 checks passed
@rustbot rustbot added this to the 1.77.0 milestone Jan 22, 2024
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Jan 22, 2024
Rollup merge of rust-lang#119999 - onur-ozkan:remote-test-tools, r=Mark-Simulacrum

remote-test: use u64 to represent file size

Currently, triggering a transfer of data exceeding the size of 4294967295 bytes results in a panic on the `remote-test-server` as `io::copy(&mut file, dst) failed with Connection reset by peer (os error 104)`. This issue happens because the size is transmitted as u32 to `remote-test-server`.

First commit increases the supported file size. But I am not sure about its necessity — can we realistically encounter file sizes exceeding 4GB in builds, perhaps through some complicated configurations?

~The second commit adds a sanity check to avoid encountering the error `io::copy(&mut file, dst) failed with Connection reset by peer (os error 104)` on the `remote-test-server` side.~
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants