Skip to content

HDDS-1216. Change name of ozoneManager service in docker compose file… #553

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ajayydv wants to merge 4 commits into from
Closed

HDDS-1216. Change name of ozoneManager service in docker compose file… #553

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f848d83
HDDS-1216. Change name of ozoneManager service in docker compose file…
Mar 4, 2019
81e1e29
revert s3 related changes
Mar 5, 2019
d20eb98
remove s3 service from ozonesecure compose file
Mar 5, 2019
270c5af
remove s3 related change.
Mar 6, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions hadoop-ozone/dist/src/main/compose/ozone-hdfs/docker-compose.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,14 +36,14 @@ services:
HADOOP_CLASSPATH: /opt/ozone/share/hadoop/ozoneplugin/*.jar
env_file:
- ./docker-config
ozoneManager:
om:
image: apache/hadoop-runner
volumes:
- ../..:/opt/hadoop
ports:
- 9874:9874
environment:
ENSURE_OM_INITIALIZED: /data/metadata/ozoneManager/current/VERSION
ENSURE_OM_INITIALIZED: /data/metadata/om/current/VERSION
WAITFOR: scm:9876
env_file:
- ./docker-config
Expand Down
4 changes: 2 additions & 2 deletions hadoop-ozone/dist/src/main/compose/ozone-hdfs/docker-config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.

OZONE-SITE.XML_ozone.om.address=ozoneManager
OZONE-SITE.XML_ozone.om.http-address=ozoneManager:9874
OZONE-SITE.XML_ozone.om.address=om
OZONE-SITE.XML_ozone.om.http-address=om:9874
OZONE-SITE.XML_ozone.scm.names=scm
OZONE-SITE.XML_ozone.enabled=true
OZONE-SITE.XML_ozone.scm.datanode.id=/data/datanode.id
Expand Down
4 changes: 2 additions & 2 deletions hadoop-ozone/dist/src/main/compose/ozone/docker-compose.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,15 +26,15 @@ services:
command: ["/opt/hadoop/bin/ozone","datanode"]
env_file:
- ./docker-config
ozoneManager:
om:
image: apache/hadoop-runner
privileged: true #required by the profiler
volumes:
- ../..:/opt/hadoop
ports:
- 9874:9874
environment:
ENSURE_OM_INITIALIZED: /data/metadata/ozoneManager/current/VERSION
ENSURE_OM_INITIALIZED: /data/metadata/om/current/VERSION
Copy link
Contributor

@bharatviswa504 bharatviswa504 Mar 5, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we have changed to om, will be this be same as service name when path is created?

Copy link
Contributor Author

@ajayydv ajayydv Mar 5, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed this for consistency purpose. IMO path can be independent of service name.

WAITFOR: scm:9876
env_file:
- ./docker-config
Expand Down
4 changes: 2 additions & 2 deletions hadoop-ozone/dist/src/main/compose/ozone/docker-config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.

OZONE-SITE.XML_ozone.om.address=ozoneManager
OZONE-SITE.XML_ozone.om.http-address=ozoneManager:9874
OZONE-SITE.XML_ozone.om.address=om
OZONE-SITE.XML_ozone.om.http-address=om:9874
OZONE-SITE.XML_ozone.scm.names=scm
OZONE-SITE.XML_ozone.enabled=True
OZONE-SITE.XML_ozone.scm.datanode.id=/data/datanode.id
Expand Down
4 changes: 2 additions & 2 deletions hadoop-ozone/dist/src/main/compose/ozoneblockade/docker-compose.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,14 @@ services:
command: ["/opt/hadoop/bin/ozone","datanode"]
env_file:
- ./docker-config
ozoneManager:
om:
image: apache/hadoop-runner
volumes:
- ../..:/opt/hadoop
ports:
- 9874:9874
environment:
ENSURE_OM_INITIALIZED: /data/metadata/ozoneManager/current/VERSION
ENSURE_OM_INITIALIZED: /data/metadata/om/current/VERSION
env_file:
- ./docker-config
command: ["/opt/hadoop/bin/ozone","om"]
Expand Down
4 changes: 2 additions & 2 deletions hadoop-ozone/dist/src/main/compose/ozoneblockade/docker-config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.

OZONE-SITE.XML_ozone.om.address=ozoneManager
OZONE-SITE.XML_ozone.om.http-address=ozoneManager:9874
OZONE-SITE.XML_ozone.om.address=om
OZONE-SITE.XML_ozone.om.http-address=om:9874
OZONE-SITE.XML_ozone.scm.names=scm
OZONE-SITE.XML_ozone.enabled=True
OZONE-SITE.XML_ozone.scm.datanode.id=/data/datanode.id
Expand Down
6 changes: 3 additions & 3 deletions hadoop-ozone/dist/src/main/compose/ozonefs/docker-compose.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,15 +25,15 @@ services:
command: ["/opt/hadoop/bin/ozone","datanode"]
env_file:
- ./docker-config
ozoneManager:
om:
image: apache/hadoop-runner
hostname: ozoneManager
hostname: om
volumes:
- ../..:/opt/hadoop
ports:
- 9874
environment:
ENSURE_OM_INITIALIZED: /data/metadata/ozoneManager/current/VERSION
ENSURE_OM_INITIALIZED: /data/metadata/om/current/VERSION
WAITFOR: scm:9876
env_file:
- ./docker-config
Expand Down
4 changes: 2 additions & 2 deletions hadoop-ozone/dist/src/main/compose/ozonefs/docker-config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@
# limitations under the License.

CORE-SITE.XML_fs.o3fs.impl=org.apache.hadoop.fs.ozone.OzoneFileSystem
OZONE-SITE.XML_ozone.om.address=ozoneManager
OZONE-SITE.XML_ozone.om.http-address=ozoneManager:9874
OZONE-SITE.XML_ozone.om.address=om
OZONE-SITE.XML_ozone.om.http-address=om:9874
OZONE-SITE.XML_ozone.scm.names=scm
OZONE-SITE.XML_ozone.enabled=True
OZONE-SITE.XML_ozone.scm.datanode.id=/data/datanode.id
Expand Down
4 changes: 2 additions & 2 deletions hadoop-ozone/dist/src/main/compose/ozoneperf/docker-compose.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,14 @@ services:
command: ["ozone","datanode"]
env_file:
- ./docker-config
ozoneManager:
om:
image: apache/hadoop-runner
volumes:
- ../..:/opt/hadoop
ports:
- 9874:9874
environment:
ENSURE_OM_INITIALIZED: /data/metadata/ozoneManager/current/VERSION
ENSURE_OM_INITIALIZED: /data/metadata/om/current/VERSION
WAITFOR: scm:9876
env_file:
- ./docker-config
Expand Down
4 changes: 2 additions & 2 deletions hadoop-ozone/dist/src/main/compose/ozoneperf/docker-config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.

OZONE-SITE.XML_ozone.om.address=ozoneManager
OZONE-SITE.XML_ozone.om.http-address=ozoneManager:9874
OZONE-SITE.XML_ozone.om.address=om
OZONE-SITE.XML_ozone.om.http-address=om:9874
OZONE-SITE.XML_ozone.scm.names=scm
OZONE-SITE.XML_ozone.enabled=true
OZONE-SITE.XML_ozone.scm.datanode.id=/data/datanode.id
Expand Down
4 changes: 2 additions & 2 deletions hadoop-ozone/dist/src/main/compose/ozones3/docker-compose.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,14 @@ services:
command: ["ozone","datanode"]
env_file:
- ./docker-config
ozoneManager:
om:
image: apache/hadoop-runner
volumes:
- ../..:/opt/hadoop
ports:
- 9874:9874
environment:
ENSURE_OM_INITIALIZED: /data/metadata/ozoneManager/current/VERSION
ENSURE_OM_INITIALIZED: /data/metadata/om/current/VERSION
WAITFOR: scm:9876
env_file:
- ./docker-config
Expand Down
2 changes: 1 addition & 1 deletion hadoop-ozone/dist/src/main/compose/ozones3/docker-config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.

OZONE-SITE.XML_ozone.om.address=ozoneManager
OZONE-SITE.XML_ozone.om.address=om
OZONE-SITE.XML_ozone.scm.names=scm
OZONE-SITE.XML_ozone.enabled=true
OZONE-SITE.XML_ozone.scm.datanode.id=/data/datanode.id
Expand Down
2 changes: 1 addition & 1 deletion hadoop-ozone/dist/src/main/compose/ozonesecure/docker-compose.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ services:
command: ["/opt/hadoop/bin/ozone","datanode"]
env_file:
- docker-config
ozoneManager:
om:
image: apache/hadoop-runner
hostname: om
volumes:
Expand Down
2 changes: 1 addition & 1 deletion hadoop-ozone/dist/src/main/compose/ozonesecure/docker-config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ LOG4J2.PROPERTIES_rootLogger.appenderRef.stdout.ref=STDOUT

OZONE_DATANODE_SECURE_USER=root
KEYTAB_DIR=/etc/security/keytabs
KERBEROS_KEYTABS=dn om scm HTTP testuser
KERBEROS_KEYTABS=dn om scm HTTP testuser s3g
KERBEROS_KEYSTORES=hadoop
KERBEROS_SERVER=kdc
JAVA_HOME=/usr/lib/jvm/jre
Expand Down
4 changes: 2 additions & 2 deletions hadoop-ozone/dist/src/main/compose/ozonetrace/docker-compose.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,14 +31,14 @@ services:
command: ["ozone","datanode"]
env_file:
- ./docker-config
ozoneManager:
om:
image: apache/hadoop-runner
volumes:
- ../..:/opt/hadoop
ports:
- 9874:9874
environment:
ENSURE_OM_INITIALIZED: /data/metadata/ozoneManager/current/VERSION
ENSURE_OM_INITIALIZED: /data/metadata/om/current/VERSION
WAITFOR: scm:9876
env_file:
- ./docker-config
Expand Down
2 changes: 1 addition & 1 deletion hadoop-ozone/dist/src/main/compose/ozonetrace/docker-config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.

OZONE-SITE.XML_ozone.om.address=ozoneManager
OZONE-SITE.XML_ozone.om.address=om
OZONE-SITE.XML_ozone.scm.names=scm
OZONE-SITE.XML_ozone.enabled=true
OZONE-SITE.XML_ozone.scm.datanode.id=/data/datanode.id
Expand Down
2 changes: 1 addition & 1 deletion hadoop-ozone/dist/src/main/smoketest/basic/basic.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ ${DATANODE_HOST} datanode
Check webui static resources
${result} = Execute curl -s -I http://scm:9876/static/bootstrap-3.3.7/js/bootstrap.min.js
Should contain ${result} 200
${result} = Execute curl -s -I http://ozoneManager:9874/static/bootstrap-3.3.7/js/bootstrap.min.js
${result} = Execute curl -s -I http://om:9874/static/bootstrap-3.3.7/js/bootstrap.min.js
Should contain ${result} 200

Start freon testing
Expand Down
2 changes: 1 addition & 1 deletion hadoop-ozone/dist/src/main/smoketest/basic/ozone-shell.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Test Timeout 2 minute

*** Test Cases ***
RpcClient with port
Test ozone shell o3:// ozoneManager:9862 rpcwoport
Test ozone shell o3:// om:9862 rpcwoport

RpcClient without host
Test ozone shell o3:// ${EMPTY} rpcwport
Expand Down
30 changes: 15 additions & 15 deletions hadoop-ozone/dist/src/main/smoketest/ozonefs/ozonefs.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,39 +23,39 @@ Resource ../commonlib.robot

*** Test Cases ***
Create volume and bucket
Execute ozone sh volume create http://ozoneManager/fstest --user bilbo --quota 100TB --root
Execute ozone sh volume create http://ozoneManager/fstest2 --user bilbo --quota 100TB --root
Execute ozone sh bucket create http://ozoneManager/fstest/bucket1
Execute ozone sh bucket create http://ozoneManager/fstest/bucket2
Execute ozone sh bucket create http://ozoneManager/fstest2/bucket3
Execute ozone sh volume create http://om/fstest --user bilbo --quota 100TB --root
Execute ozone sh volume create http://om/fstest2 --user bilbo --quota 100TB --root
Execute ozone sh bucket create http://om/fstest/bucket1
Execute ozone sh bucket create http://om/fstest/bucket2
Execute ozone sh bucket create http://om/fstest2/bucket3

Check volume from ozonefs
${result} = Execute ozone fs -ls o3fs://bucket1.fstest/

Run ozoneFS tests
Execute ozone fs -mkdir -p o3fs://bucket1.fstest/testdir/deep
${result} = Execute ozone sh key list o3://ozoneManager/fstest/bucket1 | grep -v WARN | jq -r '.[].keyName'
${result} = Execute ozone sh key list o3://om/fstest/bucket1 | grep -v WARN | jq -r '.[].keyName'
Should contain ${result} testdir/deep
Execute ozone fs -copyFromLocal NOTICE.txt o3fs://bucket1.fstest/testdir/deep/
${result} = Execute ozone sh key list o3://ozoneManager/fstest/bucket1 | grep -v WARN | jq -r '.[].keyName'
${result} = Execute ozone sh key list o3://om/fstest/bucket1 | grep -v WARN | jq -r '.[].keyName'
Should contain ${result} NOTICE.txt

Execute ozone fs -put NOTICE.txt o3fs://bucket1.fstest/testdir/deep/PUTFILE.txt
${result} = Execute ozone sh key list o3://ozoneManager/fstest/bucket1 | grep -v WARN | jq -r '.[].keyName'
${result} = Execute ozone sh key list o3://om/fstest/bucket1 | grep -v WARN | jq -r '.[].keyName'
Should contain ${result} PUTFILE.txt

${result} = Execute ozone fs -ls o3fs://bucket1.fstest/testdir/deep/
Should contain ${result} NOTICE.txt
Should contain ${result} PUTFILE.txt

Execute ozone fs -mv o3fs://bucket1.fstest/testdir/deep/NOTICE.txt o3fs://bucket1.fstest/testdir/deep/MOVED.TXT
${result} = Execute ozone sh key list o3://ozoneManager/fstest/bucket1 | grep -v WARN | jq -r '.[].keyName'
${result} = Execute ozone sh key list o3://om/fstest/bucket1 | grep -v WARN | jq -r '.[].keyName'
Should contain ${result} MOVED.TXT
Should not contain ${result} NOTICE.txt

Execute ozone fs -mkdir -p o3fs://bucket1.fstest/testdir/deep/subdir1
Execute ozone fs -cp o3fs://bucket1.fstest/testdir/deep/MOVED.TXT o3fs://bucket1.fstest/testdir/deep/subdir1/NOTICE.txt
${result} = Execute ozone sh key list o3://ozoneManager/fstest/bucket1 | grep -v WARN | jq -r '.[].keyName'
${result} = Execute ozone sh key list o3://om/fstest/bucket1 | grep -v WARN | jq -r '.[].keyName'
Should contain ${result} subdir1/NOTICE.txt

${result} = Execute ozone fs -ls o3fs://bucket1.fstest/testdir/deep/subdir1/
Expand All @@ -65,19 +65,19 @@ Run ozoneFS tests
Should not contain ${result} Failed

Execute ozone fs -rm o3fs://bucket1.fstest/testdir/deep/subdir1/NOTICE.txt
${result} = Execute ozone sh key list o3://ozoneManager/fstest/bucket1 | grep -v WARN | jq -r '.[].keyName'
${result} = Execute ozone sh key list o3://om/fstest/bucket1 | grep -v WARN | jq -r '.[].keyName'
Should not contain ${result} NOTICE.txt

${result} = Execute ozone fs -rmdir o3fs://bucket1.fstest/testdir/deep/subdir1/
${result} = Execute ozone sh key list o3://ozoneManager/fstest/bucket1 | grep -v WARN | jq -r '.[].keyName'
${result} = Execute ozone sh key list o3://om/fstest/bucket1 | grep -v WARN | jq -r '.[].keyName'
Should not contain ${result} subdir1

Execute ozone fs -touch o3fs://bucket1.fstest/testdir/TOUCHFILE.txt
${result} = Execute ozone sh key list o3://ozoneManager/fstest/bucket1 | grep -v WARN | jq -r '.[].keyName'
${result} = Execute ozone sh key list o3://om/fstest/bucket1 | grep -v WARN | jq -r '.[].keyName'
Should contain ${result} TOUCHFILE.txt

Execute ozone fs -rm -r o3fs://bucket1.fstest/testdir/
${result} = Execute ozone sh key list o3://ozoneManager/fstest/bucket1 | grep -v WARN | jq -r '.[].keyName'
${result} = Execute ozone sh key list o3://om/fstest/bucket1 | grep -v WARN | jq -r '.[].keyName'
Should not contain ${result} testdir

Execute rm -Rf localdir1
Expand All @@ -98,7 +98,7 @@ Run ozoneFS tests

Execute ozone fs -cp o3fs://bucket1.fstest/testdir1/localdir1 o3fs://bucket3.fstest2/testdir3/

Execute ozone sh key put o3://ozoneManager/fstest/bucket1/KEY.txt NOTICE.txt
Execute ozone sh key put o3://om/fstest/bucket1/KEY.txt NOTICE.txt
${result} = Execute ozone fs -ls o3fs://bucket1.fstest/KEY.txt
Should contain ${result} KEY.txt
${rc} ${result} = Run And Return Rc And Output ozone fs -copyFromLocal NOTICE.txt o3fs://bucket1.fstest/KEY.txt
Expand Down
2 changes: 1 addition & 1 deletion hadoop-ozone/dist/src/main/smoketest/test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ execute_tests(){
TITLE="Ozone $TEST tests with $COMPOSE_DIR cluster"
set +e
OUTPUT_NAME="$COMPOSE_DIR-${TEST//\//_}"
docker-compose -f "$COMPOSE_FILE" exec -T ozoneManager python -m robot --log NONE --report NONE "${OZONE_ROBOT_OPTS[@]}" --output "smoketest/$RESULT_DIR/robot-$OUTPUT_NAME.xml" --logtitle "$TITLE" --reporttitle "$TITLE" "smoketest/$TEST"
docker-compose -f "$COMPOSE_FILE" exec -T om python -m robot --log NONE --report NONE "${OZONE_ROBOT_OPTS[@]}" --output "smoketest/$RESULT_DIR/robot-$OUTPUT_NAME.xml" --logtitle "$TITLE" --reporttitle "$TITLE" "smoketest/$TEST"
Copy link

@hadoop-yetus hadoop-yetus Mar 5, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whitespace:tabs in line

Copy link
Contributor Author

@ajayydv ajayydv Mar 6, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure what is the problem here, change involves just rename.

Copy link
Member

@elek elek Mar 6, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO, you modified a line which already contained a tab.

set -e
docker-compose -f "$COMPOSE_FILE" logs > "$DIR/$RESULT_DIR/docker-$OUTPUT_NAME.log"
done
Expand Down