IEnumerable should have a extension for creating fixed size chunks

[inputfalken]

Background and Motivation

There's currently no method available to create chunks from an IEnumerable<T>. It's a popular functionality as this comment shows.
I believe most projects in need of this functionality have their own extension to IEnumerable<T> and the goal of this proposal is to have System.Linq offer this functionality

Proposed API

namespace System.Linq
{
    public static class Enumerable
    {
+        public static IEnumerable<T[]> Chunk(this IEnumerable<T> source, int size);
    }

    public static class Queryable
    {
+        public static IQueryable<T[]> Chunk(this IQueryable<T> source, int size);
    }
}

Usage Examples

int[] numbers = new int[]{1 ,2 ,3 ,4 ,5 ,6 ,7 ,8 ,9, 10, 11, 12}
IEnumerable<int[]> numberChunks = numbers.Chunk(5) // {[1, 2, 3, 4, 5], [6, 7, 8, 9, 10], [11, 12]}

Alternative Designs

Another way to create chunks would be to specify the amount of chunks you would like to have from an IEnumerable<T> but I think it would be more desirable to have it as a separate method.
like the sample below.

// Where size determines the split amount.
public static IEnumerable<T[]> SplitBy(this IEnumerable<T> source, int size)
{
    ...
}

int[] numbers = new int[]{1 ,2 ,3 ,4 ,5 ,6 ,7 ,8 ,9, 10}
IEnumerable<int[]> numberChunks = numbers.SplitBy(2) // {[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]}

Risks

This could potentially be a breaking change for users who have their own extension methods due to naming collisions.

[benaadams]

GroupBy?

[svick]

@inputfalken If you want to add a new API to .Net Core, you should follow the API review process. That means not opening a PR before the API is accepted and also including a speclet in this issue.

[inputfalken]

@benaadams Maybe this is redundant.

But I think it would be nice to have the option to easily create fixed size sub-sequences by just providing an integer.

[svick]

Also note that there already was a proposal for this in the past, which was closed: https://github.com/dotnet/corefx/issues/2146.

[inputfalken]

@svick Oh i didn't see that one 😐, should I remove the PR then?

[svick]

@inputfalken Yes, I think the PR should be closed for now.

[karelz]

Is there anything left here or is it just duplicate of #14753?

[svick]

@karelz That part of https://github.com/dotnet/corefx/issues/2146 was closed by the question "Is this really common?", with no response given to the people (including me) who think it is common. So maybe its closing could be reconsidered?

(Though that issue is also a mix of unrelated proposals. Keeping this issue open instead might be the better choice.)

[karelz]

Alright, I let the new area owners to comment hen they get to it during their incomming triage - @cston @333fred (cc @jaredpar)

[eiriktsarpalis]

Chunking is a useful feature which we should consider adding to System.Linq. My preferred signature would be the following:

public static class Enumerable
{
     public static IEnumerable<T[]> ChunkBy(this IEnumerable<T> source, int size);
}

F# core already has a corresponding chunkBySize function.

[GSPP]

There has been some discussion about chunking in #14753 (search to find the relevant places). There are a few use cases detailed there.

Stack Overflow shows high demand for this API:

Link	Votes
https://stackoverflow.com/questions/11463734/split-a-list-into-smaller-lists-of-n-size	237
https://stackoverflow.com/questions/438188/split-a-collection-into-n-parts-with-linq	122
https://stackoverflow.com/questions/13709626/split-an-ienumerablet-into-fixed-sized-chunks-return-an-ienumerableienumerab	51
https://stackoverflow.com/questions/419019/split-list-into-sublists-with-linq	393
https://stackoverflow.com/questions/1450774/splitting-a-string-into-chunks-of-a-certain-size	227
https://stackoverflow.com/questions/13731796/create-batches-in-linq	114
https://stackoverflow.com/questions/15414347/how-to-loop-through-ienumerable-in-batches	83
https://stackoverflow.com/questions/12260166/how-to-consume-a-blockingcollectiont-in-batches	14

Revisit for .NET 6? @karelz

[eiriktsarpalis]

Revisit for .NET 6?

We could take a look. @inputfalken would you be able to update the original post to match the API review template?

[inputfalken]

Revisit for .NET 6?

We could take a look. @inputfalken would you be able to update the original post to match the API review template?

@eiriktsarpalis ✔

[MgSam]

I think Chunk is a better name than ChunkBy. Especially if the proposed LINQ *By methods get accepted. Those all take a function, whereas this just accepts an integer size.

[inputfalken]

I think Chunk is a better name than ChunkBy. Especially if the proposed LINQ *By methods get accepted. Those all take a function, whereas this just accepts an integer size.

Sounds fine to me, it would be unified with the Take method.

[R2D221]

MoreLINQ includes this method named as Batch:
https://github.com/morelinq/MoreLINQ/blob/master/MoreLinq/Batch.cs

[terrajobst]

Video

• Looks good as proposed, but: does this need to be on IQuerable<T>?

namespace System.Linq
{
    public static class Enumerable
    {
        public static IEnumerable<T[]> Chunk(this IEnumerable<T> source, int size);
    }
}

[eiriktsarpalis]

This should have a queryable counterpart, I've updated the proposed API.

[terrajobst]

Video

• Looks good as proposed.
• Note: Every time we add a member to Enumerable that returns IEnumerable<T> we need to add a corresponding member on Queryable that returns IQueryable<T> (and we do have a test for that)

namespace System.Linq
{
    public static class Enumerable
    {
        public static IEnumerable<T[]> Chunk(this IEnumerable<T> source, int size);
    }
    public static class Queryable
    {
        public static IQueryable<T[]> Chunk(this IQueryable<T> source, int size);
    }
}

[inputfalken]

@eiriktsarpalis I would like to grab this if possible. 🙂

Would this be the phase where a pull request is created?

[eiriktsarpalis]

@inputfalken absolutely, happy to accept PRs for this work ☺️