Skip to content

HDDS-7309. Enable by default GRPC between S3G and OM #3820

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
adoroszlai merged 13 commits into from
Apr 19, 2023
Merged

HDDS-7309. Enable by default GRPC between S3G and OM #3820

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
7bba841
OM_S3_GRPC default flag set to true
xBis7 Oct 11, 2022
3987afb
OZONE_OM_TRANSPORT_CLASS_DEFAULT changed
xBis7 Oct 11, 2022
5d30464
space cleanup
xBis7 Oct 11, 2022
8907dbe
docker-config
xBis7 Oct 11, 2022
bdd1857
integration tests fixed
xBis7 Oct 12, 2022
000a8dd
integration tests fixes
xBis7 Oct 12, 2022
1fea125
smoketest docker-config changes
xBis7 Oct 12, 2022
5beac65
test fixes
xBis7 Oct 13, 2022
51ca10d
OmTransportFactory file under hadoop-ozone/tools/META-INF/services
xBis7 Oct 14, 2022
bbfd277
TestOzoneConfigurationFields cleanup
xBis7 Oct 31, 2022
cc337e5
Merge remote-tracking branch 'origin/master' into HDDS-7309
adoroszlai Mar 27, 2023
9c14895
remove META-INF/services directory under integration-test
xBis7 Mar 28, 2023
e73516e
Merge remote-tracking branch 'origin/master' into HDDS-7309
adoroszlai Apr 17, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions hadoop-hdds/common/src/main/resources/ozone-default.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2506,6 +2506,14 @@
for Ozone Manager Admin protocol.
</description>
</property>
<property>
<name>ozone.om.transport.class</name>
<value>org.apache.hadoop.ozone.om.protocolPB.GrpcOmTransportFactory</value>
<tag>OM, MANAGEMENT</tag>
<description>
Enable Ozone Manager S3G Grpc channel.
</description>
</property>
<property>
<name>ozone.recon.http.enabled</name>
<value>true</value>
Expand Down
2 changes: 1 addition & 1 deletion hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OMConfigKeys.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -325,7 +325,7 @@ private OMConfigKeys() {
public static final String OZONE_OM_S3_GPRC_SERVER_ENABLED =
"ozone.om.s3.grpc.server_enabled";
public static final boolean OZONE_OM_S3_GRPC_SERVER_ENABLED_DEFAULT =
false;
true;
/**
* Configuration properties for OMAdminProtcol service.
*/
Expand Down
1 change: 0 additions & 1 deletion hadoop-ozone/dist/src/main/compose/ozone/docker-config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,6 @@ OZONE-SITE.XML_ozone.recon.om.snapshot.task.interval.delay=1m
OZONE-SITE.XML_ozone.datanode.pipeline.limit=1
OZONE-SITE.XML_hdds.scmclient.max.retry.timeout=30s
OZONE-SITE.XML_hdds.container.report.interval=60s
OZONE-SITE.XML_ozone.om.s3.grpc.server_enabled=true
OZONE-SITE.XML_ozone.scm.stale.node.interval=30s
OZONE-SITE.XML_ozone.scm.dead.node.interval=45s
OZONE-SITE.XML_hdds.heartbeat.interval=5s
Expand Down
1 change: 0 additions & 1 deletion hadoop-ozone/dist/src/main/compose/ozonesecure-ha/docker-config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,6 @@ OZONE-SITE.XML_hdds.grpc.tls.enabled=true
OZONE-SITE.XML_ozone.replication=3
OZONE-SITE.XML_hdds.scmclient.max.retry.timeout=30s
OZONE-SITE.XML_hdds.container.report.interval=60s
OZONE-SITE.XML_ozone.om.s3.grpc.server_enabled=true

OZONE-SITE.XML_ozone.recon.om.snapshot.task.interval.delay=1m
OZONE-SITE.XML_ozone.recon.db.dir=/data/metadata/recon
Expand Down
1 change: 0 additions & 1 deletion hadoop-ozone/dist/src/main/compose/ozonesecure/docker-config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,6 @@ OZONE-SITE.XML_hdds.scm.replication.event.timeout=10s
OZONE-SITE.XML_ozone.scm.stale.node.interval=30s
OZONE-SITE.XML_ozone.scm.dead.node.interval=45s
OZONE-SITE.XML_hdds.container.report.interval=60s
OZONE-SITE.XML_ozone.om.s3.grpc.server_enabled=true

HDFS-SITE.XML_dfs.datanode.kerberos.principal=dn/[email protected]
HDFS-SITE.XML_dfs.datanode.kerberos.keytab.file=/etc/security/keytabs/dn.keytab
Expand Down
1 change: 1 addition & 0 deletions .../integration-test/src/test/java/org/apache/hadoop/ozone/TestOzoneConfigurationFields.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ public void initializeMemberVariables() {
xmlPrefixToSkipCompare.add("ipc.client.rpc-timeout.ms");
xmlPropsToSkipCompare.add("ozone.om.leader.election.minimum.timeout" +
".duration"); // Deprecated config
xmlPropsToSkipCompare.add("ozone.om.transport.class");
configurationPropsToSkipCompare
.add(ScmConfig.ConfigStrings.HDDS_SCM_INIT_DEFAULT_LAYOUT_VERSION);
// Currently replication and type configs moved to server side.
Expand Down
15 changes: 15 additions & 0 deletions ...test/resources/META-INF/services/org.apache.hadoop.ozone.om.protocolPB.OmTransportFactory
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
org.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactory
10 changes: 10 additions & 0 deletions hadoop-ozone/integration-test/src/test/resources/ozone-site.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,16 @@
<value>org.apache.hadoop.hdds.fs.MockSpaceUsageCheckFactory$None</value>
</property>

<property>
<name>ozone.om.transport.class</name>
<value>org.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactory</value>
</property>

<property>
<name>ozone.om.s3.grpc.server_enabled</name>
<value>false</value>
</property>

<property>
<name>dfs.container.ratis.num.write.chunk.threads.per.volume</name>
<value>4</value>
Expand Down