Improve performance2: reduce unneeded allocation #16

haya14busa · 2016-09-19T18:27:20Z

ref: #4

…rovement $ go test -v -run="^$" -bench="." -benchtime=5s -cpuprofile=prof.cpu -memprofile=prof.mem | tee prof.0 BenchmarkParseFile-4 30 245095530 ns/op $ go tool pprof --alloc_space go-vimlparser.test prof.mem Entering interactive mode (type "help" for commands) (pprof) top 1401.64MB of 1678.59MB total (83.50%) Dropped 27 nodes (cum <= 8.39MB) Showing top 10 nodes out of 56 (cum >= 143.54MB) flat flat% sum% cum cum% 297.09MB 17.70% 17.70% 345.10MB 20.56% github.com/haya14busa/go-vimlparser/go.NewExportNode 254.71MB 15.17% 32.87% 254.71MB 15.17% github.com/haya14busa/go-vimlparser/go.(*StringReader).__init__ 195.13MB 11.62% 44.50% 352.14MB 20.98% github.com/haya14busa/go-vimlparser/go.(*ExprTokenizer).get 193.57MB 11.53% 56.03% 468.70MB 27.92% github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_expr9 99.08MB 5.90% 61.93% 166.59MB 9.92% github.com/haya14busa/go-vimlparser/internal/exporter.NewNode 83MB 4.94% 66.88% 83MB 4.94% reflect.unsafe_New 75.52MB 4.50% 71.38% 78.02MB 4.65% github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_curly_parts 71.52MB 4.26% 75.64% 77.02MB 4.59% github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_dot 66.50MB 3.96% 79.60% 74MB 4.41% github.com/haya14busa/go-vimlparser/go.(*ExprTokenizer).get2 65.52MB 3.90% 83.50% 143.54MB 8.55% github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_identifier (pprof) list NewExportNode Total: 1.64GB ROUTINE ======================== github.com/haya14busa/go-vimlparser/go.NewExportNode in /home/haya14busa/src/github.com/haya14busa/go-vimlparser/go/export.go 297.09MB 1.10GB (flat, cum) 66.83% of Total . . 132: if n == nil { . . 133: return nil . . 134: } . . 135: list := make([]*ExportNode, 0) . . 136: for _, n := range n.list { . 1.50MB 137: list = append(list, NewExportNode(n)) . . 138: } . . 139: rlist := make([]*ExportNode, 0) . . 140: for _, n := range n.rlist { . 34.51MB 141: rlist = append(rlist, NewExportNode(n)) . . 142: } . . 143: body := make([]*ExportNode, 0) . . 144: for _, n := range n.body { 2.01MB 294.08MB 145: body = append(body, NewExportNode(n)) . . 146: } . . 147: elseif := make([]*ExportNode, 0) . . 148: for _, n := range n.elseif { . 114.53MB 149: elseif = append(elseif, NewExportNode(n)) . . 150: } . . 151: catch := make([]*ExportNode, 0) . . 152: for _, n := range n.catch { . . 153: catch = append(catch, NewExportNode(n)) . . 154: } . . 155: return &ExportNode{ . . 156: Type: n.type_, 18MB 18MB 157: Pos: NewExportPos(n.pos), . 150.54MB 158: Left: NewExportNode(n.left), . 86.53MB 159: Right: NewExportNode(n.right), . 54.02MB 160: Cond: NewExportNode(n.cond), . . 161: Rest: NewExportNode(n.rest), . . 162: List: list, . . 163: Rlist: rlist, . . 164: Body: body, . . 165: Op: n.op, . . 166: Str: n.str, . . 167: Depth: n.depth, . . 168: Value: n.value, . . 169: . 48.01MB 170: Ea: NewExportExArg(n.ea), 6MB 6MB 171: Attr: NewExportFuncAttr(n.attr), . . 172: . 8MB 173: Endfunction: NewExportNode(n.endfunction), . . 174: Elseif: elseif, . 28.01MB 175: Else: NewExportNode(n.else_), . 6.50MB 176: Endif: NewExportNode(n.endif), . 512.16kB 177: Endwhile: NewExportNode(n.endwhile), . . 178: Endfor: NewExportNode(n.endfor), . . 179: Endtry: NewExportNode(n.endtry), . . 180: . . 181: Catch: catch, . . 182: Finally: NewExportNode(n.finally), . . 183: . . 184: Pattern: n.pattern, 271.08MB 271.08MB 185: Curly: n.curly, . . 186: } . . 187:} . . 188: . . 189:type ExportFuncAttr struct { . . 190: Range bool $ go tool pprof go-vimlparser.test prof.cpu Entering interactive mode (type "help" for commands) (pprof) top 4020ms of 9040ms total (44.47%) Dropped 108 nodes (cum <= 45.20ms) Showing top 10 nodes out of 151 (cum >= 420ms) flat flat% sum% cum cum% 1020ms 11.28% 11.28% 1710ms 18.92% runtime.scanobject 750ms 8.30% 19.58% 1430ms 15.82% runtime.mallocgc 330ms 3.65% 23.23% 760ms 8.41% regexp.(*machine).onepass 320ms 3.54% 26.77% 320ms 3.54% sync/atomic.CompareAndSwapUint32 290ms 3.21% 29.98% 290ms 3.21% runtime.heapBitsForObject 290ms 3.21% 33.19% 430ms 4.76% runtime.mapaccess2_faststr 270ms 2.99% 36.17% 270ms 2.99% sync/atomic.AddUint32 260ms 2.88% 39.05% 620ms 6.86% github.com/haya14busa/go-vimlparser/go.NewExportNode 260ms 2.88% 41.92% 540ms 5.97% sync.(*Mutex).Unlock 230ms 2.54% 44.47% 420ms 4.65% runtime.greyobject

$ benchcmp prof.0 prof.1 benchmark old ns/op new ns/op delta BenchmarkParseFile-4 245095530 244779103 -0.13%

$ benchcmp prof.1 prof.2 benchmark old ns/op new ns/op delta BenchmarkParseFile-4 244779103 225677916 -7.80% $ go tool pprof --alloc_space go-vimlparser.test prof.mem Entering interactive mode (type "help" for commands) (pprof) top 1153.55MB of 1363.78MB total (84.58%) Dropped 30 nodes (cum <= 6.82MB) Showing top 10 nodes out of 56 (cum >= 814.41MB) flat flat% sum% cum cum% 261.21MB 19.15% 19.15% 261.21MB 19.15% github.com/haya14busa/go-vimlparser/go.(*StringReader).__init__ 194.56MB 14.27% 33.42% 480.71MB 35.25% github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_expr9 185.66MB 13.61% 47.03% 348.66MB 25.57% github.com/haya14busa/go-vimlparser/go.(*ExprTokenizer).get 98.50MB 7.22% 54.26% 98.50MB 7.22% reflect.unsafe_New 88.56MB 6.49% 60.75% 112.06MB 8.22% github.com/haya14busa/go-vimlparser/go.NewNode 77.52MB 5.68% 66.43% 84.02MB 6.16% github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_curly_parts 70.52MB 5.17% 71.60% 154.54MB 11.33% github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_identifier 69.52MB 5.10% 76.70% 74.02MB 5.43% github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_dot 59MB 4.33% 81.03% 64.50MB 4.73% github.com/haya14busa/go-vimlparser/go.(*ExprTokenizer).get2 48.51MB 3.56% 84.58% 814.41MB 59.72% github.com/haya14busa/go-vimlparser/go.(*VimLParser).parse_one_cmd

coveralls · 2016-09-19T18:31:26Z

Coverage increased (+2.0%) to 83.751% when pulling 64a3a04 on performance-export into a41844f on master.

haya14busa · 2016-09-19T18:35:00Z

Benchmarks

ref: #4 (comment)

$ pwd
/home/haya14busa/src/github.com/ynkdir/vim-vimlparser

$ git rev-parse HEAD
2fff43c58968a18bc01bc8304df68bde01af04d9

$ wc -l < autoload/vimlparser.vim
5073

$ go get github.com/haya14busa/go-vimlparser/cmd/vimlparser

$ time vimlparser autoload/vimlparser.vim > /dev/null
vimlparser autoload/vimlparser.vim > /dev/null  0.25s user 0.03s system 114% cpu 0.244 total

(autoload/vimlparser.vim is the same as #4 (comment))

Before: 0.36s
After: 0.25s

coveralls · 2016-09-19T18:40:50Z

Coverage increased (+2.0%) to 83.751% when pulling 3a2f2be on performance-export into a41844f on master.

$ benchcmp prof.3 prof.4 benchmark old ns/op new ns/op delta BenchmarkParseFile-4 227103681 221732445 -2.37%

Before: . . 375: . . 376: case NODE_DICT: . . 377: var kvs []ast.KeyValue . . 378: for _, nn := range n.value.([]interface{}) { . . 379: kv := nn.([]interface{}) . 3MB 380: k := newAstNode(kv[0].(*VimNode)) . 2.50MB 381: v := newAstNode(kv[1].(*VimNode)) 5MB 5MB 382: kvs = append(kvs, ast.KeyValue{Key: k, Value: v}) . . 383: } . . 384: return &ast.Dict{ . . 385: Lcurlybrace: pos, . . 386: Entries: kvs, . . 387: } . . 388: After: . . 375: . . 376: case NODE_DICT: . . 377: entries := n.value.([]interface{}) 2MB 2MB 378: kvs := make([]ast.KeyValue, 0, len(entries)) . . 379: for _, nn := range entries { . . 380: kv := nn.([]interface{}) . 3.50MB 381: k := newAstNode(kv[0].(*VimNode)) . 3.50MB 382: v := newAstNode(kv[1].(*VimNode)) . . 383: kvs = append(kvs, ast.KeyValue{Key: k, Value: v}) . . 384: } . . 385: return &ast.Dict{ . . 386: Lcurlybrace: pos, 1MB 1MB 387: Entries: kvs, . . 388: }

coveralls · 2016-09-19T19:06:09Z

Coverage increased (+2.0%) to 83.763% when pulling 47a8076 on performance-export into a41844f on master.

haya14busa added 5 commits September 20, 2016 02:18

slice allocation

1bae2fd

$ benchcmp prof.0 prof.1 benchmark old ns/op new ns/op delta BenchmarkParseFile-4 245095530 244779103 -0.13%

s/NewNode/newAstNode

b12279c

reduce unused Regexp compile

64a3a04

update benchmarks result in README

3a2f2be

haya14busa added 2 commits September 20, 2016 03:51

return nil for Node(-1)

2ff688a

$ benchcmp prof.3 prof.4 benchmark old ns/op new ns/op delta BenchmarkParseFile-4 227103681 221732445 -2.37%

haya14busa merged commit 30ddaee into master Sep 19, 2016

haya14busa deleted the performance-export branch September 19, 2016 19:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve performance2: reduce unneeded allocation #16

Improve performance2: reduce unneeded allocation #16

haya14busa commented Sep 19, 2016 •

edited

Loading

coveralls commented Sep 19, 2016 •

edited

Loading

haya14busa commented Sep 19, 2016

coveralls commented Sep 19, 2016 •

edited

Loading

coveralls commented Sep 19, 2016 •

edited

Loading

Improve performance2: reduce unneeded allocation #16

Improve performance2: reduce unneeded allocation #16

Conversation

haya14busa commented Sep 19, 2016 • edited Loading

coveralls commented Sep 19, 2016 • edited Loading

haya14busa commented Sep 19, 2016

Benchmarks

coveralls commented Sep 19, 2016 • edited Loading

coveralls commented Sep 19, 2016 • edited Loading

haya14busa commented Sep 19, 2016 •

edited

Loading

coveralls commented Sep 19, 2016 •

edited

Loading

coveralls commented Sep 19, 2016 •

edited

Loading

coveralls commented Sep 19, 2016 •

edited

Loading