Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unnecessary(?) allocations when hashing immutable types #10208

Closed
mlubin opened this issue Feb 15, 2015 · 2 comments
Closed

unnecessary(?) allocations when hashing immutable types #10208

mlubin opened this issue Feb 15, 2015 · 2 comments
Labels
help wanted Indicates that a maintainer wants help on an issue or pull request performance Must go faster

Comments

@mlubin
Copy link
Member

mlubin commented Feb 15, 2015

I traced an allocation bottleneck in my code to the following issue. See the snippet below:

        - immutable Foo{T}
        -     x::T
        - end
        - 
        - function f()
      672     d = Dict{Foo{Int},Int}()
      480     d2 = Dict{Int,Int}()
        - 
   557200     sizehint!(d,20000)
   557200     sizehint!(d2,20000)
        0     for i in 1:10000
   160000         d[Foo(i)] = i+1
        0         d2[i] = i+1
        -     end
    80048     x = Array(Float64,10000)
        - 
        0     for i in 1:10000
   160000         x[i] = d[Foo(i)]
        0         x[i] += d2[i]
        -     end
        0     return x
        - end
        - 
        - f()
        - Profile.clear_malloc_data()
        - f()

It looks like the issue traces to allocations when calling object_id. Are these necessary, especially for bitstypes?

@mlubin
Copy link
Member Author

mlubin commented Feb 15, 2015

FWIW, implementing

Base.hash(f::Foo{Int},h::UInt) = hash(f.x,h)

solves the allocation issue. Using this workaround gave me an order of magnitude drop in memory allocation in my code.

@JeffBezanson
Copy link
Member

I think we are boxing the Foo in order to call the builtin jl_object_id. We should rewrite the relevant part of that function to accept a type tag and data address, so it can hash objects that are on the stack.

@JeffBezanson JeffBezanson added the performance Must go faster label Feb 16, 2015
@StefanKarpinski StefanKarpinski added this to the 0.5.x milestone Sep 13, 2016
@StefanKarpinski StefanKarpinski added help wanted Indicates that a maintainer wants help on an issue or pull request and removed help wanted Indicates that a maintainer wants help on an issue or pull request labels Oct 27, 2016
@Keno Keno modified the milestones: 1.x, 0.5.x Sep 3, 2017
Keno added a commit that referenced this issue Apr 19, 2019
Introduce a fast path that passes the type and the data separately
to jl_object_id_. Fixes the allocation performance problems noted
in the issue, though the `Foo` version is still approx 4x slower,
since the Int version doesn't have to go through a call to compute
its hash. Fixing that is future work.
Keno added a commit that referenced this issue Apr 20, 2019
Introduce a fast path that passes the type and the data separately
to jl_object_id_. Fixes the allocation performance problems noted
in the issue, though the `Foo` version is still approx 4x slower,
since the Int version doesn't have to go through a call to compute
its hash. Fixing that is future work.
Keno added a commit that referenced this issue Apr 20, 2019
Introduce a fast path that passes the type and the data separately
to jl_object_id_. Fixes the allocation performance problems noted
in the issue, though the `Foo` version is still approx 4x slower,
since the Int version doesn't have to go through a call to compute
its hash. Fixing that is future work.
Keno added a commit that referenced this issue Apr 20, 2019
Introduce a fast path that passes the type and the data separately
to jl_object_id_. Fixes the allocation performance problems noted
in the issue, though the `Foo` version is still approx 4x slower,
since the Int version doesn't have to go through a call to compute
its hash. Fixing that is future work.
@Keno Keno closed this as completed in b8b6c20 Apr 22, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Indicates that a maintainer wants help on an issue or pull request performance Must go faster
Projects
None yet
Development

No branches or pull requests

4 participants