Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use standard merge algorithm when merging Values #8131

Merged
merged 2 commits into from
Mar 15, 2017
Merged

Conversation

jwilder
Copy link
Contributor

@jwilder jwilder commented Mar 14, 2017

While investigating a runtime/GC stall, @aclements noticed that Values.Merge was using a sub-optimal algorithm and that a traditional merge algorithm would likely perform better in general. He also discovered that the benchmark we were using had a bug which caused the current implementation to appear better when it was actually quite worse in all but one case.

This switches the Values.Merge to use a standard merge algorithm.

benchmark                          old ns/op     new ns/op     delta
BenchmarkValues_Merge-8            729849        47429         -93.50%
BenchmarkValues_MergeSame-8        22544         51528         +128.57%
BenchmarkValues_MergeSimilar-8     247516        49819         -79.87%

benchmark                          old allocs     new allocs     delta
BenchmarkValues_Merge-8            1              1              +0.00%
BenchmarkValues_MergeSame-8        0              1              +Inf%
BenchmarkValues_MergeSimilar-8     1              1              +0.00%

benchmark                          old bytes     new bytes     delta
BenchmarkValues_Merge-8            32768         32768         +0.00%
BenchmarkValues_MergeSame-8        0             32768         +Inf%
BenchmarkValues_MergeSimilar-8     32768         32768         +0.00%

@jwilder jwilder added this to the 1.3.0 milestone Mar 14, 2017
@jwilder jwilder requested a review from e-dard March 14, 2017 15:26
@e-dard
Copy link
Contributor

e-dard commented Mar 14, 2017

@jwilder two-way merges works best when the two lists are around the same size. Would it be unusual to merge very differently sized lists? If so, could we add a benchmark for that?

@jwilder
Copy link
Contributor Author

jwilder commented Mar 14, 2017

@e-dard I'll add those. Sizes of both lists will usually be different.

Merge had the side effect of modifying the original values so
the results are wrong because they always hit the fast path
after the first run.
@jwilder
Copy link
Contributor Author

jwilder commented Mar 14, 2017

@e-dard Added some other benchmarks:

benchmark                           old ns/op     new ns/op     delta
BenchmarkValues_Merge-8             725067        46914         -93.53%
BenchmarkValues_MergeDisjoint-8     23278         24094         +3.51%
BenchmarkValues_MergeSame-8         22181         51152         +130.61%
BenchmarkValues_MergeSimilar-8      250766        49441         -80.28%
BenchmarkValues_MergeUnevenA-8      5111          7141          +39.72%
BenchmarkValues_MergeUnevenB-8      7302          7265          -0.51%

benchmark                           old allocs     new allocs     delta
BenchmarkValues_Merge-8             1              1              +0.00%
BenchmarkValues_MergeDisjoint-8     1              1              +0.00%
BenchmarkValues_MergeSame-8         0              1              +Inf%
BenchmarkValues_MergeSimilar-8      1              1              +0.00%
BenchmarkValues_MergeUnevenA-8      0              1              +Inf%
BenchmarkValues_MergeUnevenB-8      1              1              +0.00%

benchmark                           old bytes     new bytes     delta
BenchmarkValues_Merge-8             32768         32768         +0.00%
BenchmarkValues_MergeDisjoint-8     32768         32768         +0.00%
BenchmarkValues_MergeSame-8         0             32768         +Inf%
BenchmarkValues_MergeSimilar-8      32768         32768         +0.00%
BenchmarkValues_MergeUnevenA-8      0             12288         +Inf%
BenchmarkValues_MergeUnevenB-8      12288         12288         +0.00%

if a[0].UnixNano() < b[0].UnixNano() {
out, a = append(out, a[0]), a[1:]
} else {
if len(b) > 0 && a[0].UnixNano() == b[0].UnixNano() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can move line 145 up to make 144 an else if.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 7561dea

out, a = append(out, a[0]), a[1:]
} else {
if len(b) > 0 && a[0].UnixNano() == b[0].UnixNano() {
a = a[1:]
} else {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This else can then be an else on the outer if block, removing some indentation.

out, b = append(out, b[0]), b[1:]
}
}
if len(a) > 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be simplified to:

if len(a) > 0 {
    return append(out, a...)
}
return append(out, b...)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in a4cfeac

The previous version was very innefficient due to the benchmarks used
to optimize it having a bug.  This version always allocates a new
slice, but is O(n).
@jwilder jwilder merged commit 65464ea into master Mar 15, 2017
@jwilder jwilder deleted the jw-values-merge branch March 15, 2017 15:51
@jwilder jwilder mentioned this pull request Mar 23, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants