Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ExactChunks::remainder and ExactChunks::into_remainder #51339

Merged
merged 1 commit into from
Jul 12, 2018

Conversation

sdroege
Copy link
Contributor

@sdroege sdroege commented Jun 4, 2018

These allow to get the leftover items of the slice that are not being
iterated as part of the iterator due to not filling a complete chunk.

The mutable version consumes the slice because otherwise we would either
a) have to borrow the iterator instead of taking the lifetime of
the underlying slice, which is not what any of the other iterator
functions is doing, or
b) would allow returning multiple mutable references to the same data

The current behaviour of consuming the iterator is consistent with
IterMut::into_slice for the normal iterator.


This is related to #47115 (comment) and the following comments.

While there the discussion was first about a way to get the "tail" of the iterator (everything from the slice that is still not iterated yet), this gives kind of unintuitive behaviour and is inconsistent with how the other slice iterators work.

Unintuitive because the next_back would have no effect on the tail (or otherwise the tail could not include the remainder items), inconsistent because a) generally the idea of the slice iterators seems to be to only ever return items that were not iterated yet (and don't provide a way to access the same item twice) and b) we would return a "flat" &[T] slice but the iterator's shape is &[[T]] instead, c) the mutable variant would have to borrow from the iterator instead of the underlying slice (all other iterator functions borrow from the underlying slice!)

As such, I've only implemented functions to get the remainder. This also allows the implementation to be completely safe still (and around slices instead of raw pointers), while getting the tail would either be inefficient or would have to be implemented around raw pointers.

CC @Kerollmops

@rust-highfive
Copy link
Collaborator

r? @alexcrichton

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 4, 2018
@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-3.9 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
[00:39:49]            
[00:39:49]            [1] Florian Loitsch. 2010. Printing floating-point numbers quickly and
[00:39:49]             ^
[00:39:49] 
[00:39:50] warning: [into_remainder] cannot be resolved, ignoring it...
[00:39:50]     |
[00:39:50]     |
[00:39:50] 732 | /     /// Returns an iterator over `chunk_size` elements of the slice at a time.
[00:39:50] 733 | |     /// The chunks are mutable slices, and do not overlap. If `chunk_size` does
[00:39:50] 734 | |     /// not divide the length of the slice, then the last chunk will not
[00:39:50] 735 | |     /// have length `chunk_size` and can be retrieved from the [`into_remainder`]
[00:39:50] 759 | |     ///
[00:39:50] 759 | |     ///
[00:39:50] 760 | |     /// [`exact_chunks_mut`]: #method.exact_chunks_mut
[00:39:50]     |
[00:39:50]     = note: the link appears in this line:
[00:39:50]             
[00:39:50]             
[00:39:50]              have length `chunk_size` and can be retrieved from the [`into_remainder`]
[00:39:50] 
[00:39:50] warning: [x] cannot be resolved, ignoring it...
[00:39:50]   --> libcore/../stdsimd/coresimd/ppsv/api/mod.rs:1:1
[00:39:50]    |
---
[00:40:00]     Checking panic_unwind v0.0.0 (file:///checkout/src/libpanic_unwind)
[00:40:00]     Checking rustc_msan v0.0.0 (file:///checkout/src/librustc_msan)
[00:40:00]     Checking rustc_tsan v0.0.0 (file:///checkout/src/librustc_tsan)
[00:40:00]  Documenting std v0.0.0 (file:///checkout/src/libstd)
[00:40:02] warning: [into_remainder] cannot be resolved, ignoring it...
[00:40:02]     |
[00:40:02]     |
[00:40:02] 732 | /     /// Returns an iterator over `chunk_size` elements of the slice at a time.
[00:40:02] 733 | |     /// The chunks are mutable slices, and do not overlap. If `chunk_size` does
[00:40:02] 734 | |     /// not divide the length of the slice, then the last chunk will not
[00:40:02] 735 | |     /// have length `chunk_size` and can be retrieved from the [`into_remainder`]
[00:40:02] 759 | |     ///
[00:40:02] 759 | |     ///
[00:40:02] 760 | |     /// [`exact_chunks_mut`]: #method.exact_chunks_mut
[00:40:02]     |
[00:40:02]     = note: the link appears in this line:
[00:40:02]             
[00:40:02]             
[00:40:02]              have length `chunk_size` and can be retrieved from the [`into_remainder`]
[00:40:02] 
[00:40:02] 
[00:40:09] warning: [into_remainder] cannot be resolved, ignoring it...
[00:40:09]     |
[00:40:09]     |
[00:40:09] 732 | /     /// Returns an iterator over `chunk_size` elements of the slice at a time.
[00:40:09] 733 | |     /// The chunks are mutable slices, and do not overlap. If `chunk_size` does
[00:40:09] 734 | |     /// not divide the length of the slice, then the last chunk will not
[00:40:09] 735 | |     /// have length `chunk_size` and can be retrieved from the [`into_remainder`]
[00:40:09] 759 | |     ///
[00:40:09] 759 | |     ///
[00:40:09] 760 | |     /// [`exact_chunks_mut`]: #method.exact_chunks_mut
[00:40:09]     |
[00:40:09]     = note: the link appears in this line:
[00:40:09]             
[00:40:09]             
[00:40:09]              have length `chunk_size` and can be retrieved from the [`into_remainder`]
[00:40:09] 
[00:40:16]     Finished release [optimized] target(s) in 1m 11.42s
[00:40:16] Documenting stage2 test (x86_64-unknown-linux-gnu)
[00:40:17]     Checking getopts v0.2.17
---
[00:43:03] ...........................................................................i........................
[00:43:08] ....................................................................................................
[00:43:13] ....................................................................................................
[00:43:19] ....................................................................................................
[00:43:24] ........i.................iiiiiiiii...................................................
[00:43:24] 
[00:43:24] travis_fold:start:test_ui_nll
travis_time:start:test_ui_nll
Check compiletest suite=ui mode=ui compare_mode=nll (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
---
[00:44:12] ...........................................................................i........................
[00:44:17] ....................................................................................................
[00:44:21] ....................................................................................................
[00:44:27] ....................................................................................................
[00:44:31] ........i.................iiiiiiiii...................................................
[00:44:31] 
[00:44:31]  finished in 67.394
[00:44:31] travis_fold:end:test_ui_nll

---
[01:20:16] travis_fold:end:stage0-linkchecker

[01:20:16] travis_time:end:stage0-linkchecker:start=1528103521715620406,finish=1528103524480752033,duration=2765131627

[01:20:24] std/vec/struct.Vec.html:985: broken link - std/std/slice/struct.ExactChunks.html
[01:20:24] std/vec/struct.Vec.html:1025: broken link - std/std/slice/struct.ExactChunksMut.html
[01:20:24] std/slice/index.html:193: broken link fragment `#method.into_remainder` pointing to `std/slice/index.html`
[01:20:28] core/slice/index.html:147: broken link fragment `#method.into_remainder` pointing to `core/slice/index.html`
[01:20:33] alloc/slice/index.html:193: broken link fragment `#method.into_remainder` pointing to `alloc/slice/index.html`
[01:20:33] thread 'main' panicked at 'found some broken links', tools/linkchecker/main.rs:49:9
[01:20:33] 
[01:20:33] 
[01:20:33] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/linkchecker" "/checkout/obj/build/x86_64-unknown-linux-gnu/doc"
[01:20:33] expected success, got: exit code: 101
[01:20:33] expected success, got: exit code: 101
[01:20:33] 
[01:20:33] 
[01:20:33] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
[01:20:33] Build completed unsuccessfully in 0:39:38
[01:20:33] Makefile:58: recipe for target 'check' failed
[01:20:33] make: *** [check] Error 1
34916 ./obj/build/x86_64-unknown-linux-gnu/native/jemalloc/src
34816 ./obj/build/x86_64-unknown-linux-gnu/test/run-pass
34588 ./obj/build/x86_64-unknown-linux-gnu/native/jemalloc/lib
34372 ./obj/build/x86_64-unknown-linux-gnu/doc/core/arch

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@sdroege sdroege force-pushed the exact-chunks-remainder branch 2 times, most recently from 62eae2a to 73d609a Compare June 4, 2018 10:02
@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-3.9 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
[00:47:12] ...........................................................................i........................
[00:47:16] ....................................................................................................
[00:47:22] ....................................................................................................
[00:47:28] ....................................................................................................
[00:47:33] ........i.................iiiiiiiii...................................................
[00:47:33] 
[00:47:33] travis_fold:start:test_ui_nll
travis_time:start:test_ui_nll
Check compiletest suite=ui mode=ui compare_mode=nll (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
---
[00:48:24] ...........................................................................i........................
[00:48:28] ....................................................................................................
[00:48:33] ....................................................................................................
[00:48:39] ....................................................................................................
[00:48:43] ........i.................iiiiiiiii...................................................
[00:48:43] 
[00:48:43]  finished in 70.545
[00:48:43] travis_fold:end:test_ui_nll

---
[01:27:12] travis_fold:end:stage0-linkchecker

[01:27:12] travis_time:end:stage0-linkchecker:start=1528111839626278601,finish=1528111842602995470,duration=2976716869

[01:27:20] std/vec/struct.Vec.html:1005: broken link - std/std/slice/struct.ExactChunks.html
[01:27:20] std/vec/struct.Vec.html:1024: broken link - std/std/slice/struct.ExactChunksMut.html
[01:27:21] std/slice/index.html:193: broken link fragment `#method.into_remainder` pointing to `std/slice/index.html`
[01:27:25] core/slice/index.html:147: broken link fragment `#method.into_remainder` pointing to `core/slice/index.html`
[01:27:30] alloc/slice/index.html:193: broken link fragment `#method.into_remainder` pointing to `alloc/slice/index.html`
[01:27:31] thread 'main' panicked at 'found some broken links', tools/linkchecker/main.rs:49:9
[01:27:31] 
[01:27:31] 
[01:27:31] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/linkchecker" "/checkout/obj/build/x86_64-unknown-linux-gnu/doc"
[01:27:31] expected success, got: exit code: 101
[01:27:31] expected success, got: exit code: 101
[01:27:31] 
[01:27:31] 
[01:27:31] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
[01:27:31] Build completed unsuccessfully in 0:42:38
[01:27:31] make: *** [check] Error 1
[01:27:31] Makefile:58: recipe for target 'check' failed

The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:0e2e2a60
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@sdroege
Copy link
Contributor Author

sdroege commented Jun 4, 2018

If someone has suggestions how to make rustdoc happy, please tell me :) In the meantime I'll try different things. Locally the docs links seem fine when rendered, but the linkchecker still complains

These allow to get the leftover items of the slice that are not being
iterated as part of the iterator due to not filling a complete chunk.

The mutable version consumes the slice because otherwise we would either
a) have to borrow the iterator instead of taking the lifetime of
the underlying slice, which is not what *any* of the other iterator
functions is doing, or
b) would allow returning multiple mutable references to the same data

The current behaviour of consuming the iterator is consistent with
IterMut::into_slice for the normal iterator.
@sdroege sdroege force-pushed the exact-chunks-remainder branch from 73d609a to 903624f Compare June 4, 2018 13:09
@@ -729,7 +729,8 @@ impl<T> [T] {
/// Returns an iterator over `chunk_size` elements of the slice at a
/// time. The chunks are slices and do not overlap. If `chunk_size` does
/// not divide the length of the slice, then the last up to `chunk_size-1`
/// elements will be omitted.
/// elements will be omitted and can be retrieved from the `remainder`
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This and the same for exact_chunks_mut can't be linked to the documentation of the function. It would have to be ../std/slice/etc for the direct documentation here and ../../std/slice/etc for the "copied" documentation in e.g. Vec (for its AsRef<_> impl)

Just omitting the link for now

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#32130 Is the relevant issue for this AFAIU

@sdroege
Copy link
Contributor Author

sdroege commented Jun 7, 2018

Once this is in it would be possible to implement the other chunks iterators on top of this, which is probably faster than the current ones still. I'll play with that at some point in the next days.

@pietroalbini
Copy link
Member

The reviewer is away, maybe someone else from @rust-lang/libs can review this?

1 similar comment
@pietroalbini
Copy link
Member

The reviewer is away, maybe someone else from @rust-lang/libs can review this?

@alexcrichton alexcrichton added T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. S-waiting-on-team Status: Awaiting decision from the relevant subteam (see the T-<team> label). labels Jun 27, 2018
@TimNN TimNN removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 3, 2018
@alexcrichton
Copy link
Member

Egad sorry for the delay!

@bors: r+

@bors
Copy link
Contributor

bors commented Jul 12, 2018

📌 Commit 903624f has been approved by alexcrichton

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-team Status: Awaiting decision from the relevant subteam (see the T-<team> label). labels Jul 12, 2018
@sdroege
Copy link
Contributor Author

sdroege commented Jul 12, 2018

Egad sorry for the delay

No worries! Thanks for the review and merging :)

@bors
Copy link
Contributor

bors commented Jul 12, 2018

⌛ Testing commit 903624f with merge 64f7de9...

bors added a commit that referenced this pull request Jul 12, 2018
Add ExactChunks::remainder and ExactChunks::into_remainder

These allow to get the leftover items of the slice that are not being
iterated as part of the iterator due to not filling a complete chunk.

The mutable version consumes the slice because otherwise we would either
a) have to borrow the iterator instead of taking the lifetime of
the underlying slice, which is not what *any* of the other iterator
functions is doing, or
b) would allow returning multiple mutable references to the same data

The current behaviour of consuming the iterator is consistent with
IterMut::into_slice for the normal iterator.

----

This is related to #47115 (comment) and the following comments.

While there the discussion was first about a way to get the "tail" of the iterator (everything from the slice that is still not iterated yet), this gives kind of unintuitive behaviour and is inconsistent with how the other slice iterators work.

Unintuitive because the `next_back` would have no effect on the tail (or otherwise the tail could not include the remainder items), inconsistent because a) generally the idea of the slice iterators seems to be to only ever return items that were not iterated yet (and don't provide a way to access the same item twice) and b) we would return a "flat" `&[T]` slice but the iterator's shape is `&[[T]]` instead, c) the mutable variant would have to borrow from the iterator instead of the underlying slice (all other iterator functions borrow from the underlying slice!)

As such, I've only implemented functions to get the remainder. This also allows the implementation to be completely safe still (and around slices instead of raw pointers), while getting the tail would either be inefficient or would have to be implemented around raw pointers.

CC @Kerollmops
@bors
Copy link
Contributor

bors commented Jul 12, 2018

☀️ Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing 64f7de9 to master...

@bors bors merged commit 903624f into rust-lang:master Jul 12, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants