Potential Error in Generate Rolling Dataset

[aamrb96]

Hi everyone,

I am working with the generate_rolling_dataset function in order to cross-validate my results. Given the documentation I would have expected NumSplitsStrategy to yield "chunks of equal size" given my parameters. However, the function returns chunks of different size and works very similar to StepStrategy.

Here is some reproducible code:

ds = [{

    "target": np.array([9,67,45,345,88,64,45,3,23,11]),

    "start": pd.Timestamp('2000-1-1-01', freq='1H')

}]

rolled = generate_rolling_dataset(

    dataset=ds,

    strategy = NumSplitsStrategy(prediction_length=3, num_splits=3),

    start_time = pd.Timestamp('2000-1-1-06', '1H'),

    end_time = pd.Timestamp('2000-1-1-10', '1H')

)

# Print chunks created by generate_rolling_dataset
for i in rolled:
    print(i["target"])

# Out
# [  9  67  45 345  88  64  45   3]
# [  9  67  45 345  88  64  45   3  23]
# [  9  67  45 345  88  64  45   3  23  11]

I would have expected the output chunks to be of equal size as indicated in the documentation.

Maybe I am missing something but this seems unintuitive. I would have expected this to implement a rolling cross validation with a fixed window.

This article summarizes the idea under "Blocked Cross-Validation":