Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Grab keys via callback iteration rather than many absolute calls #787

Merged
merged 3 commits into from
Dec 28, 2020

Commits on Dec 23, 2020

  1. Grab keys via callback iteration rather than many absolute calls

    It appears there's significant overhead in directly obtaining the name
    of every sub-element using a loop of `h5?_get_name_by_idx` calls.
    For instance, a v7.3-formatted Matlab save file I have access to has
    deeply nested and large struct which are serialized into a huge number
    of datasets within a `#refs#` group.
    
    ```julia
    julia> Int(length(fid["#refs#"]))
    36236
    ```
    
    Before this change, enumerating the keys takes almost a minute:
    ```julia
    julia> @time keys(fid["#refs#"]);
     54.193914 seconds (72.47 k allocations: 3.594 MiB)
    ```
    but with this change, that's reduced to a fraction of a second:
    ```julia
    julia> @time keys(fid["#refs#"]);
      0.013353 seconds (36.24 k allocations: 1.383 MiB)
    ```
    jmert committed Dec 23, 2020
    Configuration menu
    Copy the full SHA
    04a9ccf View commit details
    Browse the repository at this point in the history

Commits on Dec 27, 2020

  1. Configuration menu
    Copy the full SHA
    6065167 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    73e62e4 View commit details
    Browse the repository at this point in the history