-
Notifications
You must be signed in to change notification settings - Fork 198
Merge protobuf_benchmarks and query_benchmark #698
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 15 commits
Commits
Show all changes
63 commits
Select commit
Hold shift + click to select a range
3775eb2
WIP
osa1 00a46ea
Fix compile_protos.sh
osa1 3401c33
Copy common code
osa1 6afbd6f
Add query_benchmark encode/decode benchmarks
osa1 a7a8061
Add query_benchmark set_nested_value
osa1 8acb3cc
Add protobuf_benchmarks decode benchmark
osa1 5532e13
Fixups
osa1 b8eeb2e
Add another bench
osa1 d2598b1
Another bench
osa1 9aa9e59
More benches
osa1 5398c17
Up
osa1 b4e0a77
Add hash code bench
osa1 76ecf29
Remove unused file
osa1 e177125
Update bench names
osa1 e45f7c8
Remove old benchmarks
osa1 e11e12a
Merge remote-tracking branch 'origin/master' into refactor_benchmarks
osa1 db19e75
Update workflow
osa1 b212ffc
Remove late vars
osa1 b17ecfd
Move scripts to tool/, add mono_repo
osa1 51e9fad
Add a tool to compile benchmarks
osa1 61fa212
Add script to compile benchmarks faster
osa1 3d81be5
Divide query_benchmark results by 10 to be compatible with old results
osa1 ce60860
Fix warnings
osa1 1902aca
Add README
osa1 1b567e9
Fix readme formatting
osa1 5256dad
Commit pubspec.yaml
osa1 ee2320a
Update `QueryBenchmark` documentation
osa1 97b137b
Remove compile_benchmark.sh, update README
osa1 0075b98
Report accurate results in all benchmarks, enable linter
osa1 fdd1c7e
Update copyright years in tool/
osa1 d058236
compile_benchmarks.py: Do not spawn more processes than number of CPUs
osa1 9c1cf13
compile_benchamrks.py: Update comments
osa1 eae565b
compile_benchmarks.py: Exit with 1 when a subprocess fails
osa1 83c3f5f
Implement benchmark builder in Dart
osa1 1f627fd
Remove compile_benchmarks.py, update Dart script
osa1 6e78ad8
Update README
osa1 9e37586
Update an error message
osa1 98800c8
Make some vars final
osa1 0c1a928
Remove a dynamic cast
osa1 2ef63b4
Print process stdout and stderr on failure
osa1 adf1109
Remove redundant `async`
osa1 424ac75
Rename aot -> exe
osa1 df3ae16
Create 'out' directory before compilation
osa1 da2f87a
Add aot target
osa1 3790230
Only try to compile Dart files
osa1 39da206
Merge remote-tracking branch 'origin/master' into refactor_benchmarks
osa1 2f19d56
Update dill file name
osa1 5156dd2
Add js-production mode
osa1 933086b
Add proto3 JSON object parsing benchmark
osa1 de6a05f
Fix warnings
osa1 847124a
Update benchmark names
osa1 f6c00c3
Remove silly comment
osa1 f5f3344
Refactor file list initialization
osa1 751e661
Use -O4 in JS production mode
osa1 98b0e84
Fix type error
osa1 f2e0fe7
Show failing command when a process fails
osa1 2d14348
Merge remote-tracking branch 'origin/master' into refactor_benchmarks
osa1 c427802
Report file system errors when creating out dir
osa1 a01c866
Add protoc_version file for Golem
osa1 ed89a19
Remove _benchmark_ in query benchmark names
osa1 5e9862b
Report benchmark results as RunTimeRaw instead of RunTime
osa1 aecc575
Add "protobuf_" prefix to test names
osa1 665d291
Merge remote-tracking branch 'origin/master' into refactor_benchmarks
osa1 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| // Copyright (c) 2018, the Dart project authors. Please see the AUTHORS file | ||
| // for details. All rights reserved. Use of this source code is governed by a | ||
| // BSD-style license that can be found in the LICENSE file. | ||
|
|
||
| import 'package:protobuf_benchmarks/readfile.dart'; | ||
| import 'package:protobuf_benchmarks/generated/google_message1_proto2.pb.dart' | ||
| as p2; | ||
| import 'package:protobuf_benchmarks/generated/google_message1_proto3.pb.dart' | ||
| as p3; | ||
| import 'package:protobuf_benchmarks/generated/google_message2.pb.dart'; | ||
|
|
||
| import 'dart:typed_data'; | ||
|
|
||
| import 'package:benchmark_harness/benchmark_harness.dart'; | ||
|
|
||
| class Benchmark extends BenchmarkBase { | ||
| late final Uint8List _message1Proto2Input; | ||
| late final Uint8List _message1Proto3Input; | ||
| late final Uint8List _message2Input; | ||
|
|
||
| Benchmark(String name, List<int> message1Proto2Input, | ||
| List<int> message1Proto3Input, List<int> message2Input) | ||
| : super(name) { | ||
| _message1Proto2Input = Uint8List.fromList(message1Proto2Input); | ||
| _message1Proto3Input = Uint8List.fromList(message1Proto3Input); | ||
| _message2Input = Uint8List.fromList(message2Input); | ||
| } | ||
|
|
||
| @override | ||
| void run() { | ||
| p2.GoogleMessage1.fromBuffer(_message1Proto2Input); | ||
| p3.GoogleMessage1.fromBuffer(_message1Proto3Input); | ||
| GoogleMessage2.fromBuffer(_message2Input); | ||
| } | ||
| } | ||
|
|
||
| void main() { | ||
| List<int> message1Proto2Input = | ||
| readfile('datasets/google_message1_proto2.pb'); | ||
| List<int> message1Proto3Input = | ||
| readfile('datasets/google_message1_proto3.pb'); | ||
| List<int> message2Input = readfile('datasets/google_message2.pb'); | ||
| Benchmark('protobuf_from_binary', message1Proto2Input, message1Proto3Input, | ||
| message2Input) | ||
| .report(); | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| // Copyright (c) 2018, the Dart project authors. Please see the AUTHORS file | ||
| // for details. All rights reserved. Use of this source code is governed by a | ||
| // BSD-style license that can be found in the LICENSE file. | ||
|
|
||
| import 'package:protobuf_benchmarks/readfile.dart'; | ||
| import 'package:protobuf_benchmarks/generated/google_message1_proto2.pb.dart' | ||
| as p2; | ||
| import 'package:protobuf_benchmarks/generated/google_message1_proto3.pb.dart' | ||
| as p3; | ||
| import 'package:protobuf_benchmarks/generated/google_message2.pb.dart'; | ||
|
|
||
| import 'package:benchmark_harness/benchmark_harness.dart'; | ||
|
|
||
| class Benchmark extends BenchmarkBase { | ||
| late final String _message1Proto2JsonString; | ||
| late final String _message1Proto3JsonString; | ||
| late final String _message2JsonString; | ||
|
|
||
| Benchmark(String name, List<int> message1Proto2Input, | ||
| List<int> message1Proto3Input, List<int> message2Input) | ||
| : super(name) { | ||
| _message1Proto2JsonString = | ||
| p2.GoogleMessage1.fromBuffer(message1Proto2Input).writeToJson(); | ||
| _message1Proto3JsonString = | ||
| p3.GoogleMessage1.fromBuffer(message1Proto3Input).writeToJson(); | ||
| _message2JsonString = | ||
| GoogleMessage2.fromBuffer(message2Input).writeToJson(); | ||
| } | ||
|
|
||
| @override | ||
| void run() { | ||
| p2.GoogleMessage1.fromJson(_message1Proto2JsonString); | ||
| p3.GoogleMessage1.fromJson(_message1Proto3JsonString); | ||
| GoogleMessage2.fromJson(_message2JsonString); | ||
| } | ||
| } | ||
|
|
||
| void main() { | ||
| List<int> message1Proto2Input = | ||
| readfile('datasets/google_message1_proto2.pb'); | ||
| List<int> message1Proto3Input = | ||
| readfile('datasets/google_message1_proto3.pb'); | ||
| List<int> message2Input = readfile('datasets/google_message2.pb'); | ||
| Benchmark('protobuf_from_json_string', message1Proto2Input, | ||
| message1Proto3Input, message2Input) | ||
| .report(); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| // Copyright (c) 2018, the Dart project authors. Please see the AUTHORS file | ||
| // for details. All rights reserved. Use of this source code is governed by a | ||
| // BSD-style license that can be found in the LICENSE file. | ||
|
|
||
| import 'package:protobuf_benchmarks/readfile.dart'; | ||
| import 'package:protobuf_benchmarks/generated/google_message1_proto2.pb.dart' | ||
| as p2; | ||
| import 'package:protobuf_benchmarks/generated/google_message1_proto3.pb.dart' | ||
| as p3; | ||
| import 'package:protobuf_benchmarks/generated/google_message2.pb.dart'; | ||
|
|
||
| import 'dart:convert' show jsonDecode, jsonEncode; | ||
|
|
||
| import 'package:benchmark_harness/benchmark_harness.dart'; | ||
|
|
||
| class Benchmark extends BenchmarkBase { | ||
| late final String _message1Proto2Proto3JsonString; | ||
| late final String _message1Proto3Proto3JsonString; | ||
| late final String _message2Proto3JsonString; | ||
|
|
||
| Benchmark(String name, List<int> message1Proto2Input, | ||
| List<int> message1Proto3Input, List<int> message2Input) | ||
| : super(name) { | ||
| _message1Proto2Proto3JsonString = jsonEncode( | ||
| p2.GoogleMessage1.fromBuffer(message1Proto2Input).toProto3Json()); | ||
| _message1Proto3Proto3JsonString = jsonEncode( | ||
| p3.GoogleMessage1.fromBuffer(message1Proto3Input).toProto3Json()); | ||
| _message2Proto3JsonString = | ||
| jsonEncode(GoogleMessage2.fromBuffer(message2Input).toProto3Json()); | ||
| } | ||
|
|
||
| @override | ||
| void run() { | ||
| p2.GoogleMessage1.create() | ||
| ..mergeFromProto3Json(jsonDecode(_message1Proto2Proto3JsonString)); | ||
| p3.GoogleMessage1.create() | ||
| ..mergeFromProto3Json(jsonDecode(_message1Proto3Proto3JsonString)); | ||
| GoogleMessage2.create() | ||
| ..mergeFromProto3Json(jsonDecode(_message2Proto3JsonString)); | ||
| } | ||
| } | ||
|
|
||
| void main() { | ||
| List<int> message1Proto2Input = | ||
| readfile('datasets/google_message1_proto2.pb'); | ||
| List<int> message1Proto3Input = | ||
| readfile('datasets/google_message1_proto3.pb'); | ||
| List<int> message2Input = readfile('datasets/google_message2.pb'); | ||
| Benchmark('protobuf_from_proto3_json_string', message1Proto2Input, | ||
| message1Proto3Input, message2Input) | ||
| .report(); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| // Copyright (c) 2018, the Dart project authors. Please see the AUTHORS file | ||
| // for details. All rights reserved. Use of this source code is governed by a | ||
| // BSD-style license that can be found in the LICENSE file. | ||
|
|
||
| import 'package:protobuf_benchmarks/readfile.dart'; | ||
| import 'package:protobuf_benchmarks/generated/google_message1_proto2.pb.dart' | ||
| as p2; | ||
| import 'package:protobuf_benchmarks/generated/google_message1_proto3.pb.dart' | ||
| as p3; | ||
| import 'package:protobuf_benchmarks/generated/google_message2.pb.dart'; | ||
|
|
||
| import 'dart:convert' show jsonEncode; | ||
|
|
||
| import 'package:benchmark_harness/benchmark_harness.dart'; | ||
|
|
||
| class Benchmark extends BenchmarkBase { | ||
| late final p2.GoogleMessage1 _message1Proto2; | ||
| late final p3.GoogleMessage1 _message1Proto3; | ||
| late final GoogleMessage2 _message2; | ||
|
|
||
| Benchmark(String name, List<int> message1Proto2Input, | ||
| List<int> message1Proto3Input, List<int> message2Input) | ||
| : super(name) { | ||
| _message1Proto2 = p2.GoogleMessage1.fromBuffer(message1Proto2Input); | ||
| _message1Proto3 = p3.GoogleMessage1.fromBuffer(message1Proto3Input); | ||
| _message2 = GoogleMessage2.fromBuffer(message2Input); | ||
| } | ||
|
|
||
| @override | ||
| void run() { | ||
| _message1Proto2.hashCode; | ||
| _message1Proto3.hashCode; | ||
| _message2.hashCode; | ||
| } | ||
| } | ||
|
|
||
| void main() { | ||
| List<int> message1Proto2Input = | ||
| readfile('datasets/google_message1_proto2.pb'); | ||
| List<int> message1Proto3Input = | ||
| readfile('datasets/google_message1_proto3.pb'); | ||
| List<int> message2Input = readfile('datasets/google_message2.pb'); | ||
| Benchmark('protobuf_hashCode', message1Proto2Input, message1Proto3Input, | ||
| message2Input) | ||
| .report(); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| // Copyright (c) 2018, the Dart project authors. Please see the AUTHORS file | ||
| // for details. All rights reserved. Use of this source code is governed by a | ||
| // BSD-style license that can be found in the LICENSE file. | ||
|
|
||
| import 'package:protobuf_benchmarks/generated/f0.pb.dart' as f0; | ||
| import 'package:protobuf_benchmarks/readfile.dart'; | ||
|
|
||
| import 'package:benchmark_harness/benchmark_harness.dart'; | ||
|
|
||
| class Benchmark extends BenchmarkBase { | ||
| final List<int> _input; | ||
|
|
||
| Benchmark(String name, this._input) : super(name); | ||
|
|
||
| @override | ||
| void run() { | ||
| f0.A0.fromBuffer(this._input); | ||
| } | ||
| } | ||
|
|
||
| void main() { | ||
| List<int> encoded = readfile('datasets/query_benchmark.pb'); | ||
| Benchmark('protobuf_decode', encoded).report(); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| // Copyright (c) 2018, the Dart project authors. Please see the AUTHORS file | ||
| // for details. All rights reserved. Use of this source code is governed by a | ||
| // BSD-style license that can be found in the LICENSE file. | ||
|
|
||
| import 'package:protobuf_benchmarks/generated/f0.pb.dart' as f0; | ||
| import 'package:protobuf_benchmarks/readfile.dart'; | ||
|
|
||
| import 'package:benchmark_harness/benchmark_harness.dart'; | ||
|
|
||
| class Benchmark extends BenchmarkBase { | ||
| late final String _input; | ||
|
|
||
| Benchmark(String name, List<int> input) : super(name) { | ||
| f0.A0 a = f0.A0.fromBuffer(input); | ||
| _input = a.writeToJson(); | ||
| } | ||
|
|
||
| @override | ||
| void run() { | ||
| f0.A0.fromJson(this._input); | ||
| } | ||
| } | ||
|
|
||
| void main() { | ||
| List<int> encoded = readfile('datasets/query_benchmark.pb'); | ||
| Benchmark('protobuf_decode_json', encoded).report(); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| // Copyright (c) 2018, the Dart project authors. Please see the AUTHORS file | ||
| // for details. All rights reserved. Use of this source code is governed by a | ||
| // BSD-style license that can be found in the LICENSE file. | ||
|
|
||
| import 'package:protobuf_benchmarks/generated/f0.pb.dart' as f0; | ||
| import 'package:protobuf_benchmarks/readfile.dart'; | ||
|
|
||
| import 'package:benchmark_harness/benchmark_harness.dart'; | ||
|
|
||
| class Benchmark extends BenchmarkBase { | ||
| late final f0.A0 _input; | ||
|
|
||
| Benchmark(String name, List<int> input) : super(name) { | ||
| _input = f0.A0.fromBuffer(input); | ||
| } | ||
|
|
||
| @override | ||
| void run() { | ||
| _input.writeToBuffer(); | ||
| } | ||
| } | ||
|
|
||
| void main() { | ||
| List<int> encoded = readfile('datasets/query_benchmark.pb'); | ||
| Benchmark('protobuf_encode', encoded).report(); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| // Copyright (c) 2018, the Dart project authors. Please see the AUTHORS file | ||
| // for details. All rights reserved. Use of this source code is governed by a | ||
| // BSD-style license that can be found in the LICENSE file. | ||
|
|
||
| import 'package:protobuf_benchmarks/generated/f0.pb.dart' as f0; | ||
| import 'package:protobuf_benchmarks/readfile.dart'; | ||
|
|
||
| import 'package:benchmark_harness/benchmark_harness.dart'; | ||
|
|
||
| class Benchmark extends BenchmarkBase { | ||
| late final f0.A0 _input; | ||
|
|
||
| Benchmark(String name, List<int> input) : super(name) { | ||
| _input = f0.A0.fromBuffer(input); | ||
| } | ||
|
|
||
| @override | ||
| void run() { | ||
| _input.writeToJson(); | ||
| } | ||
| } | ||
|
|
||
| void main() { | ||
| List<int> encoded = readfile('datasets/query_benchmark.pb'); | ||
| Benchmark('protobuf_encode_json', encoded).report(); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| // Copyright (c) 2018, the Dart project authors. Please see the AUTHORS file | ||
| // for details. All rights reserved. Use of this source code is governed by a | ||
| // BSD-style license that can be found in the LICENSE file. | ||
|
|
||
| import 'package:protobuf_benchmarks/readfile.dart'; | ||
| import 'package:protobuf_benchmarks/generated/f0.pb.dart' as f0; | ||
| import 'package:protobuf_benchmarks/generated/f19.pb.dart' as f19; | ||
| import 'package:protobuf_benchmarks/generated/f2.pb.dart' as f2; | ||
|
|
||
| import 'package:benchmark_harness/benchmark_harness.dart'; | ||
| import 'package:protobuf/protobuf.dart'; | ||
|
|
||
| class Benchmark extends BenchmarkBase { | ||
| late final f0.A0 _input; | ||
|
|
||
| Benchmark(String name, List<int> input) : super(name) { | ||
| _input = f0.A0.fromBuffer(input)..freeze(); | ||
| } | ||
|
|
||
| @override | ||
| void run() { | ||
| // ignore: unused_result | ||
| _input.rebuild((f0.A0 a0Builder) { | ||
| a0Builder.a4.last = a0Builder.a4.last.rebuild((f2.A1 a1builder) { | ||
| a1builder.a378 = a1builder.a378 | ||
| .rebuild((f19.A220 a220builder) => a220builder.a234 = 'new_value'); | ||
| }); | ||
| }); | ||
| } | ||
| } | ||
|
|
||
| void main() { | ||
| List<int> encoded = readfile('datasets/query_benchmark.pb'); | ||
| Benchmark('protobuf_set_nested_value', encoded).report(); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| // Copyright (c) 2018, the Dart project authors. Please see the AUTHORS file | ||
| // for details. All rights reserved. Use of this source code is governed by a | ||
| // BSD-style license that can be found in the LICENSE file. | ||
|
|
||
| import 'package:protobuf_benchmarks/readfile.dart'; | ||
| import 'package:protobuf_benchmarks/generated/google_message1_proto2.pb.dart' | ||
| as p2; | ||
| import 'package:protobuf_benchmarks/generated/google_message1_proto3.pb.dart' | ||
| as p3; | ||
| import 'package:protobuf_benchmarks/generated/google_message2.pb.dart'; | ||
|
|
||
| import 'package:benchmark_harness/benchmark_harness.dart'; | ||
|
|
||
| class Benchmark extends BenchmarkBase { | ||
| late final p2.GoogleMessage1 _message1Proto2; | ||
| late final p3.GoogleMessage1 _message1Proto3; | ||
| late final GoogleMessage2 _message2; | ||
|
|
||
| Benchmark(String name, List<int> message1Proto2Input, | ||
| List<int> message1Proto3Input, List<int> message2Input) | ||
| : super(name) { | ||
| _message1Proto2 = p2.GoogleMessage1.fromBuffer(message1Proto2Input); | ||
| _message1Proto3 = p3.GoogleMessage1.fromBuffer(message1Proto3Input); | ||
| _message2 = GoogleMessage2.fromBuffer(message2Input); | ||
| } | ||
|
|
||
| @override | ||
| void run() { | ||
| _message1Proto2.writeToBuffer(); | ||
| _message1Proto3.writeToBuffer(); | ||
| _message2.writeToBuffer(); | ||
| } | ||
| } | ||
|
|
||
| void main() { | ||
| List<int> message1Proto2Input = | ||
| readfile('datasets/google_message1_proto2.pb'); | ||
| List<int> message1Proto3Input = | ||
| readfile('datasets/google_message1_proto3.pb'); | ||
| List<int> message2Input = readfile('datasets/google_message2.pb'); | ||
| Benchmark('protobuf_to_binary', message1Proto2Input, message1Proto3Input, | ||
| message2Input) | ||
| .report(); | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.