Skip to content

Commit

Permalink
Improve the performance of TextSource by reducing how many byte[]s ar…
Browse files Browse the repository at this point in the history
…e copied (fixes apache#23193)

This makes TextSource take about 2.3x less CPU resources during decoding.

Before this change:
```
TextSourceBenchmark.benchmarkTextSource        thrpt    5  0.248 ± 0.029  ops/s
```

After this change:
```
TextSourceBenchmark.benchmarkHadoopLineReader  thrpt    5  0.465 ± 0.064  ops/s
TextSourceBenchmark.benchmarkTextSource        thrpt    5  0.575 ± 0.059  ops/s
```
  • Loading branch information
lukecwik committed Sep 12, 2022
1 parent f31b277 commit e9a7c11
Show file tree
Hide file tree
Showing 4 changed files with 292 additions and 133 deletions.
1 change: 1 addition & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@
## I/Os

* Support for X source added (Java/Python) ([#X](https://github.com/apache/beam/issues/X)).
* Decreased TextSource CPU utilization by 2.3x (Java) ([#23193](https://github.com/apache/beam/issues/23193)).

## New Features / Improvements

Expand Down
2 changes: 2 additions & 0 deletions sdks/java/core/jmh/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,11 @@ ext.summary = "This contains JMH benchmarks for the SDK Core for Beam Java"

dependencies {
implementation project(path: ":sdks:java:core", configuration: "shadow")
implementation project(path: ":sdks:java:core", configuration: "shadowTest")
implementation library.java.joda_time
implementation library.java.vendored_grpc_1_48_1
implementation library.java.vendored_guava_26_0_jre
implementation library.java.hadoop_common
runtimeOnly library.java.slf4j_jdk14
testImplementation library.java.junit
testImplementation library.java.hamcrest
Expand Down
Loading

0 comments on commit e9a7c11

Please sign in to comment.