Tracking Issue for Iterator::intersperse

[camelid]

This is a tracking issue for the Iterator::intersperse and intersperse_with methods. It is based on itertools::intersperse and is the iterator equivalent of Vec::join.

The feature gate for the issue is #![feature(iter_intersperse)].

Public API

// core::iter

pub trait Iterator {
    fn intersperse(self, separator: Self::Item) -> Intersperse<Self>
    where
        Self: Sized,
        Self::Item: Clone;

    fn intersperse_with<G>(self, separator: G) -> IntersperseWith<Self, G>
    where
        Self: Sized,
        G: FnMut() -> Self::Item;
}

#[derive(Debug, Clone)]
pub struct Intersperse<I: Iterator> where I::Item: Clone {}

impl<I> Iterator for Intersperse<I> where I: Iterator, I::Item: Clone {
    type Item = I::Item;
}

pub struct IntersperseWith<I, G> where I: Iterator {}

impl<I, G> Debug for IntersperseWith<I, G> where I: Iterator + Debug, I::Item: Debug, G: Debug {}

impl<I, G> Clone for IntersperseWith<I, G> where I: Iterator + Clone, I::Item: Clone, G: Clone {}

impl<I, G> Iterator for IntersperseWith<I, G> where I: Iterator, G: FnMut() -> I::Item {
    type Item = I::Item;
}

Steps / History

• Implement intersperse: Add Iterator::intersperse #79479
• Add intersperse_with: Add Iterator::intersperse_with #80567
• FCP
• Stabilization PR: Stabilize Iterator::intersperse() #88548
  (UPDATE: stabilization was reverted because there was a lot of breakage: Stabilizing Iterator::intersperse breaks a large number of crates #88967)
• Determine how to avoid breakage and un-revert
• Add impl<I: FusedIterator> FusedIterator for Intersperse<I>? (Can be added after stabilization, so not urgent.)

Unresolved Questions

None.

[m-ou-se]

#80567 is adding intersperse_with to this feature.

[cyqsimon]

I would like to suggest a simpler, alternative name for this method - weave. At the time of comment, the word weave is not yet used in Rust std.

Intersperse is quite verbose (especially when compared to Vec::join), but more importantly rarely used in common English. weave is much more common and concise in comparison.

[ibraheemdev]

Scala and haskell both use intersperse. Scalaz calls it intercalate.

[tmplt]

I would like to suggest a simpler, alternative name for this method - weave.

weave is not on the list of terms I would search for when looking for the expected behavior. I'd even consider it vanity.

Intersperse is quite verbose (especially when compared to Vec::join), but more importantly rarely used in common English. weave is much more common and concise in comparison.

A verbose function name describes its effect better. More so if the term is uncommon; there is then less ambiguity. intersperse does exactly what is says: intersperse elements with a common separator.

[arnabanimesh]

why not use join and join_with for iterators too?

[fosskers]

Scala and haskell both use intersperse. Scalaz calls it intercalate...

intercalate is slightly different.

intersperse :: a -> [a] -> [a] 

intercalate :: [a] -> [[a]] -> [a] 

[redgoldlace]

why not use join and join_with for iterators too?

I feel like join implies the creation of something monolithic from a sequence and a separator, i.e

let items = vec!["foo", "bar", "blah"];
let result = items.join(" ");

will create the string foo bar blah, and I don't have any issue with that.

But in my mind intersperse better communicates the idea of placing separators in between a sequence, without creating something monolithic using the results of that.

I suppose I mean that if I intersperse an iterator of &strs with " ", it makes sense to me that I get an iterator back.
But if I have an iterator of &strs, and the method is named join, I feel like it's a little less clear. Without looking further into it, it might seem like that I'd get a concatenated String back, or similar.
For other types it may be less unclear but I'm still unsure on whether calling the method join is a good fit.

[arnabanimesh]

I feel like join implies the creation of something monolithic from a sequence and a separator

That's true, but we are using vec::join or slice::join and that respects the pattern matching and it is not monolithic. Then vec::join should be changed too. If Rust already acknowledges that join be non monolithic, then I suggest we stick with the same ideology.

But if I have an iterator of &strs, and the method is named join, I feel like it's a little less clear. Without looking further into it, it might seem like that I'd get a concatenated String back, or similar.

I'm always collecting to String after intersperse. 😅

My personal preference aside, vec::join creates a vec if the initial data type is not &str. So it is just intersperse which happens to create a String in a specific scenario.

Personally, if it were not as verbose I would have gone for intersperse. If join was not used in vec or slice, I would not root for it. But weave doesn't sound good IMO.

My main point is that let's not have two different terms do the same thing based on datatype.

[pickfire]

Personally, if it were not as verbose I would have gone for intersperse. If join was not used in vec or slice, I would not root for it. But weave doesn't sound good IMO.

I did join for Iterator in the past #75738 which accepts the current Join that does both but it wasn't accepted because it requires allocation or Iterator is in std so it couldn't be implemented, splitting it as two functions works but may not be that discoverable.

In a sample code in helix,

        let res = sel
            .ranges
            .into_iter()
            .map(|range| format!("{}/{}", range.anchor, range.head))
            .collect::<Vec<String>>()
            .join(",");

How to join String or &String easily with &str and such? I think cases more than that we need some intervention which the API will not be ergonomic. I think maybe it only works well with &'static str.

In the current example, I believe it's too basic and limited, requires &'static str and copied.

let hello = ["Hello", "World", "!"].iter().copied().intersperse(" ").collect::<String>();

I think we need to think more about the ergonomics of the API before stabilization.

[m-ou-se]

@rfcbot merge

[rfcbot]

Team member @m-ou-se has proposed to merge this. The next step is review by the rest of the tagged team members:

• @Amanieu
• @BurntSushi
• @dtolnay
• @joshtriplett
• @m-ou-se
• @yaahc

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[SoniEx2]

could call it "sep" for separator?

[c410-f3r]

What is the status of this feature?

[Fishrock123]

@c410-f3r as I mentioned above it seems to be blocked on requiring #89151

[c410-f3r]

Thank you @Fishrock123

The plan we arrived upon is to first revert the stabilization and then plan on stabilizing the feature in a future release, @Mark-Simulacrum suggested 1.65.

Leaving aside the above initial plan described in #88967 (comment) and taking into consideration that #89151 appears to be stalled, this feature will likely take some years to be stabilized, if ever.

[Fishrock123]

Yes, it looks like someone interested would need to take up working on #89151

[kornelski]

This has been open for years. The nice solution of changing trait resolution had no progress for years either, and may be blocked by other refactorings too.

Could we reconsider renaming the method?

• Rust has the doc(alias) feature, so it can still be findable by this term.

• "intersperse" seems to be an uncommon word. English is not my native language, and I don't recall ever hearing that word in a conversation. It's not a word I'd use to find this function, and the name to me doesn't explain what the function does, so personally I don't care for that term.

How about separate_by or insert_between? I think these are simpler names that accurately explain this function.

Or lace, interlace, weave, interweave, intertwine.

[lukaslueg]

My personal bikeshed-report: I'd suggest interweave. Unlike most alternatives, the verb addresses an action done to the iterator (as opposed to an action done to the elements, e.g. seperated_by), and doesn't sound unnecessarily mathematical (e.g. interpolate) or skeuomorphic (e.g. sprinkle, interlace, amalgamate, commingle)

[jplatte]

I think separate(d)_by is a really good name. Unlike intersperse, interlace and interweave it's actually obvious that it takes a separator, not a second iterator like Itertools::interleave.

[camelid]

We could also consider calling it join, like the similar method Vec::join. It's straightforward and matches names in other languages (like Python). The only downside could be confusion with Vec::join, but there are already a couple of existing methods that have the same name between Iterator and Vec (e.g., first and last).

[lukaslueg]

We could also consider calling it join, like the similar method Vec::join. It's straightforward and matches names in other languages (like Python). The only downside could be confusion with Vec::join, but there are already a couple of existing methods that have the same name between Iterator and Vec (e.g., first and last).

.join() is a misnomer on Vec imho, as the term alludes to an action done to the elements ("the elements are joined using the argument"), as opposed to the Vec ("the Vec is joined with a second Vec; which join() doesn't actually do).

If someone can make a decision, I'm willing to change the Stone Of Shame around my neck for the Stone Of Triumph and submit a PR.

[camelid]

I agree that Vec::join has a confusing name... however, I'm unclear if that means you agree that Iterator::join is a good name. It's obviously too late to change Vec::join, so if Iterator::join would match it and be no worse a name, it seems like a potentially good option. Do you mind clarifying?

[kornelski]

It can't be join, because that's already taken by a slightly different operation that also collects. Vec::join("") returns a single joined String. So does Itertools::join.

Both join functions call their argument a separator. Therefore I think Iter::separate(_by) makes the most sense for the iterator operation that happens under the hood of join.

[camelid]

I see, that makes sense. It does seem like Iter::separate or separate_by are best then. Perhaps we do intersperse -> separate and intersperse_with -> separate_with?

[senekor]

My personal favorite would be separate_with and nothing else. The version accepting a closure is more powerful and those couple characters won't hurt anybody. I look at it like this:

.separate_with(|| ",")
.separate(",")

The only purpose of having a separate version seems to me to save a couple (8) characters. Not worth it to clutter the standard library documentation in my opinion. But it doesn't matter that much either. Having both is fine too.

[lukaslueg]

@camelid I was arguing for not using .join(), my bad.

fn separate() -> Separate and fn separate_with() -> SeparateWith it is, then?

[workingjubilee]

I think fn insert_between works best, particularly if we only have the closure form, but I don't mind separate_with. I agree that we only really need one form.

[camelid]

@workingjubilee my disagreement with wording like insert_between is that it sounds very imperative, whereas Iterator methods (almost) always have declarative wording in line with their behavior.

[senekor]

I think separate_with fits better than insert_between with the grammatical pattern of existing iterator methods. Maybe other people turn the method names into sentences differently, but here's what my brain does:

method	mental expansion
verb_adverb	verb "items" adverb (argument)
step_by	step (over) items by
take_while	take items while
collect_into	collect the items into
separate_with	separate the items with ✅
insert_between	insert (separator) between the items 🤔

[lukaslueg]

I love a good bikeshedding just as everyone, yet before this Tracking Issue is overwhelmed with it, I suggest to wait and see on the merit of #79524 (comment) in it's own right. My guess is that the PR to simply evade blockers now being on I-libs-api-nominated will yield some insights on that.

[jadebenn]

I just encountered a situation where this method would have been an excellent solution to a problem I was facing, but I was restricted to stable Rust and couldn't use another crate. It's a shame that it's been marked as unstable for so long given that it only seems to be blocked via a naming conflict with itertools.

Renaming seems like a reasonable solution to me, but I would also note that the corresponding depreciation request on the itertools github seemed to point to there not being a full understanding on their part that this was not being marked as stable because of the naming conflict. They wanted to wait for stability to deprecate their own method, leading to the current state of things.

[farazshaikh]

better to have intersperse(seprator, interval) where the marker needs to spaced every iterval.
Only problem i see is reconciling the meaning of iterval == 0. It could be

• idx becomes NonZero(usize)
• return the array unchanged
• returns a new array just wit markers (this is the worst , )

Examples: [1,1,1,1].iter().intersperse(0,2) -> [1, 1, 0, 1,1, 0]
Also would need a to capture the meaning for some output like intersperse(??) -> [0, 1, 1, 0 , 1, 1]

I specifically ran into this while looking for solution for KZG encoding that requrires a tranfroming a byte stream into 32 byte symbol with the first (big edian) byte to be zero.

https://docs.eigenlayer.xyz/eigenda/rollup-guides/blob-encoding

As it stands , this API looks like a "alternate" implementation of zip() with infinite iterator over the second iterator.

[farazshaikh]

:( looks like a direct transation of this
https://hackage.haskell.org/package/base-4.19.1.0/docs/Data-List.html#v:intersperse

Would be gereat to know how to solve this usecase
[elt elt elt sep elt elt elt sep]
[sep elt elt elt sep elt elt elt]

[Amanieu]

To give an update, we're currently trying to work around the naming conflict with a language-level feature: #89151 (comment)

[bew]

Will there be a way to access the current separator index when using intersperse_with ?