Skip to content

Conversation

@avijayanhwx
Copy link
Contributor

In RocksDb 6.0.1, some useful tuning features were brought into the JNI API. We need to upgrade the version in ozone to pick those up.

facebook/rocksdb#4833

Copy link
Contributor

@bharatviswa504 bharatviswa504 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 LGTM, pending CI.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 67 Docker mode activated.
_ Prechecks _
+1 dupname 1 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 mvninstall 692 trunk passed
+1 compile 305 trunk passed
+1 mvnsite 0 trunk passed
+1 shadedclient 1874 branch has no errors when building and testing our client artifacts.
+1 javadoc 184 trunk passed
_ Patch Compile Tests _
+1 mvninstall 516 the patch passed
+1 compile 311 the patch passed
-1 javac 98 hadoop-hdds generated 3 new + 11 unchanged - 0 fixed = 14 total (was 11)
-1 javac 213 hadoop-ozone generated 1 new + 7 unchanged - 0 fixed = 8 total (was 7)
+1 mvnsite 0 the patch passed
+1 whitespace 1 The patch has no whitespace issues.
+1 xml 1 The patch has no ill-formed XML file.
+1 shadedclient 732 patch has no errors when building and testing our client artifacts.
+1 javadoc 172 the patch passed
_ Other Tests _
-1 unit 186 hadoop-hdds in the patch failed.
-1 unit 1387 hadoop-ozone in the patch failed.
+1 asflicense 53 The patch does not generate ASF License warnings.
5713
Reason Tests
Failed junit tests hadoop.ozone.container.common.impl.TestHddsDispatcher
hadoop.ozone.client.rpc.TestOzoneRpcClientWithRatis
hadoop.ozone.client.rpc.TestOzoneAtRestEncryption
hadoop.ozone.client.rpc.TestCommitWatcher
hadoop.ozone.client.rpc.TestOzoneRpcClient
hadoop.ozone.client.rpc.TestSecureOzoneRpcClient
hadoop.ozone.TestMiniOzoneCluster
hadoop.ozone.client.rpc.TestBCSID
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-980/1/artifact/out/Dockerfile
GITHUB PR #980
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml
uname Linux 80ae802a1ddd 4.4.0-143-generic #169~14.04.2-Ubuntu SMP Wed Feb 13 15:00:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 3d020e9
Default Java 1.8.0_212
javac https://builds.apache.org/job/hadoop-multibranch/job/PR-980/1/artifact/out/diff-compile-javac-hadoop-hdds.txt
javac https://builds.apache.org/job/hadoop-multibranch/job/PR-980/1/artifact/out/diff-compile-javac-hadoop-ozone.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-980/1/artifact/out/patch-unit-hadoop-hdds.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-980/1/artifact/out/patch-unit-hadoop-ozone.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-980/1/testReport/
Max. process+thread count 4643 (vs. ulimit of 5500)
modules C: hadoop-hdds/common U: hadoop-hdds/common
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-980/1/console
versions git=2.7.4 maven=3.3.9
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@bharatviswa504
Copy link
Contributor

bharatviswa504 commented Jun 18, 2019

Test failures look unrelated to this patch.
Thank You @avijayanhwx for the contribution. Is there a Jira associated with this, so that I can add it during commit?

@bharatviswa504 bharatviswa504 merged commit 71edfce into apache:trunk Jun 18, 2019
@bharatviswa504
Copy link
Contributor

I found the Jira for this.
Thank You @anuengineer for the review.
I have committed this to the trunk.

bshashikant pushed a commit to bshashikant/hadoop that referenced this pull request Jul 10, 2019
shanthoosh pushed a commit to shanthoosh/hadoop that referenced this pull request Oct 15, 2019
1. To disambiguate between YARN containers and Samza Processor.
2. Use consistent logging format. Now can search logs with strings "Container ID: <id>", "Processor ID: <id>", and "host: <hostname>" to find all relevant information for the container / processor / host.
3. Removed references to "preferred host" in logs when host affinity is disabled.

Left StandbyContainerMangager as is for now. Will update it in a separate PR.

Without Host Affinity
Before: https://0bin.net/paste/VwqKIK2ADNCrycb-#UiOWKLYwnphA72uwXcj4wQzSX3MVqM8+qT+FY+3-tQJ
After: https://0bin.net/paste/ts+fifnScrpKpIkU#nZ0RhSaqImzHNheYBbK4ROxICnehLPzPnwHFFY+Xj1Z

With Host Affinity:
Before: https://0bin.net/paste/6Z5914FWYWJuCGo3#76r4QO+K8SX9NBJyT7yfp9Mi25-N0pzmDmIxuknEipy
After: https://0bin.net/paste/fBKJyYqHTtn41CMZ#35MEGwpsgQjeoNf0U1lnIQqywlW2vm+qim3dI3mRxfb

Author: Prateek Maheshwari <[email protected]>

Reviewers: Jagadish Venkatraman <[email protected]>, Ray Matharu <[email protected]>

Closes apache#980 from prateekm/am-metrics
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants