Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slow iteration because of IxDyn #1340

Open
RunDevelopment opened this issue Dec 9, 2023 · 1 comment
Open

Slow iteration because of IxDyn #1340

RunDevelopment opened this issue Dec 9, 2023 · 1 comment

Comments

@RunDevelopment
Copy link

As I described in #1339, an array with IxDyn has 10x slower iteration performance than an equivalent array using a fixed-size index. This has wide-reaching implications, as this means that many pixel-wise operations are substantially slower.

Example:
Let n be an ndarray with the shape (4320, 8468, 4).

let n: ArrayViewD<f32>; // uses IxDyn 

// iter()

// slow: takes 3sec on my machine
let _: Vec<f32> = n.iter().cloned().collect();

// fast: takes 0.4sec on my machine
let n3: ArrayView3<f32> = n.into_dimensionality().unwrap();
let _: Vec<f32> = n3.iter().cloned().collect();

// to_owned()

// slow: takes 0.95sec on my machine
let _ = n.to_owned();

// fast: takes 0.25sec on my machine
let n3: ArrayView3<f32> = n.into_dimensionality().unwrap();
let _ = n3.to_owned();

To improve the performance of arrays using IxDyn, I suggest optimizing iteration for these arrays. Since we can see that using fixed-sized indexes is substantially faster, I suggest internally "casting" the array to a fixed-size index (or similar) before iteration when possible.

@bluss
Copy link
Member

bluss commented Mar 31, 2024

#979 helps for this case, even if it was developed with the general case (any iterator) in mind. ndarray does also already in several cases have fast paths for contiguous arrays, also if they are dynamic dimensional.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants