diff --git a/hadoop-ozone/dist/src/main/compose/upgrade/README.md b/hadoop-ozone/dist/src/main/compose/upgrade/README.md index 49c453800e4..345b91d7bde 100644 --- a/hadoop-ozone/dist/src/main/compose/upgrade/README.md +++ b/hadoop-ozone/dist/src/main/compose/upgrade/README.md @@ -43,7 +43,7 @@ an older release of Ozone and a later release (which may be the local build). ## Supported Versions -Non-rolling upgrades and downgrades are supported from 1.1.0 to any later version. Note that 1.1.0 did not have the non-rolling upgrade framework, so things like preparing the OMs for upgrade and checking finalization status are not present in that version. Manual upgrade is the only supported upgrade option from 1.0.0 to 1.1.0. +Non-rolling upgrades and downgrades are supported from 1.1.0 to any later version. Note that 1.1.0 did not have the non-rolling upgrade framework, so things like preparing the OMs for upgrade and checking finalization status are not present in that version. ## Directory Layout @@ -72,23 +72,9 @@ Each type of upgrade has a subdirectory under the *upgrades* directory. 4. `with_this_version_finalized`: Run after ozone is stopped in the old version after donwgrade, started again in the new version pre-finalized, and then finalized. - The upgrade is complete when this callback runs. -#### manual-upgrade - -- This is a legacy option that was used before the upgrade framework was introduced in 1.2.0. This option is left as an example in case it needs to be used for some reason in the future. - -- Any necessary conversion of on disk structures from the old version to the new version must be done explicitly. - -- This is primarily for testing upgrades from versions before the non-rolling upgrade framework was introduced. - -- Supported Callbacks: - 1. `setup_old_version`: Run before ozone is started in the old version. - 3. `with_old_version`: Run while ozone is running in the old version. - 3. `setup_this_version`: Run after ozone is stopped in the old version, but before it is restarted in the new version. - 4. `with_this_version`: Run while ozone is running in the new version. - ### compose -Docker compose cluster definitions to be used in upgrade testing are defined in the *compose* directory. A compose cluster can be selected by specifying the name of its subdirectory as the first argument to `run_test`. `run_test` will then source the `load.sh` script in the cluster's directory so it is used during the test. For manual testing, docker compose can be used normally from the compose cluster directory. Note that some clusters may not work with older versions. Ozone 1.1.0, for example, does not support SCM HA. +Docker compose cluster definitions to be used in upgrade testing are defined in the *compose* directory. A compose cluster can be selected by specifying the name of its subdirectory as the first argument to `run_test`. `run_test` will then source the `load.sh` script in the cluster's directory so it is used during the test. ## Persisting Data diff --git a/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/.env b/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/.env deleted file mode 100644 index babe87a492a..00000000000 --- a/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/.env +++ /dev/null @@ -1,22 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -HDDS_VERSION=${hdds.version} -OZONE_RUNNER_VERSION=${docker.ozone-runner.version} -OZONE_RUNNER_IMAGE=apache/ozone-runner -OZONE_TEST_IMAGE=apache/ozone-runner:${docker.ozone-runner.version} -OZONE_DIR=/opt/hadoop -OZONE_VOLUME=./data diff --git a/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-compose.yaml b/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-compose.yaml deleted file mode 100644 index 7aea9af378e..00000000000 --- a/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-compose.yaml +++ /dev/null @@ -1,155 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# reusable fragments (see https://docs.docker.com/compose/compose-file/#extension-fields) -x-common-config: - &common-config - env_file: - - docker-config - image: ${OZONE_TEST_IMAGE} - -x-environment: - &environment - OZONE-SITE.XML_ozone.server.default.replication: ${OZONE_REPLICATION_FACTOR:-1} - OZONE_UPGRADE_TO: ${OZONE_UPGRADE_TO:-0} - OZONE_UPGRADE_FROM: ${OZONE_UPGRADE_FROM:-0} - -x-datanode: - &datanode - command: ["ozone","datanode"] - <<: *common-config - environment: - <<: *environment - ports: - - 19864 - - 9882 - -x-volumes: - - &ozone-dir ${TEST_DIR}/../..:${OZONE_DIR} - - &transformation ${TEST_DIR}/../../libexec/transformation.py:/opt/hadoop/libexec/transformation.py - -services: - om: - command: ["ozone","om"] - <<: *common-config - environment: - ENSURE_OM_INITIALIZED: /data/metadata/om/current/VERSION - <<: *environment - ports: - - 9862 - - 9874 - networks: - net: - ipv4_address: 10.9.0.11 - volumes: - - ${OZONE_VOLUME}/om:/data - - *ozone-dir - - *transformation - scm: - command: ["ozone","scm"] - <<: *common-config - environment: - ENSURE_SCM_INITIALIZED: /data/metadata/scm/current/VERSION - OZONE-SITE.XML_hdds.scm.safemode.min.datanode: ${OZONE_SAFEMODE_MIN_DATANODES:-1} - <<: *environment - networks: - net: - ipv4_address: 10.9.0.12 - ports: - - 9876:9876 - volumes: - - ${OZONE_VOLUME}/scm:/data - - *ozone-dir - - *transformation - dn1: - <<: *datanode - networks: - net: - ipv4_address: 10.9.0.13 - volumes: - - ${OZONE_VOLUME}/dn1:/data - - *ozone-dir - - *transformation - dn2: - <<: *datanode - networks: - net: - ipv4_address: 10.9.0.14 - volumes: - - ${OZONE_VOLUME}/dn2:/data - - *ozone-dir - - *transformation - dn3: - <<: *datanode - networks: - net: - ipv4_address: 10.9.0.15 - volumes: - - ${OZONE_VOLUME}/dn3:/data - - *ozone-dir - - *transformation - dn4: - <<: *datanode - networks: - net: - ipv4_address: 10.9.0.16 - volumes: - - ${OZONE_VOLUME}/dn4:/data - - *ozone-dir - - *transformation - dn5: - <<: *datanode - networks: - net: - ipv4_address: 10.9.0.17 - volumes: - - ${OZONE_VOLUME}/dn5:/data - - *ozone-dir - - *transformation - recon: - command: ["ozone","recon"] - <<: *common-config - environment: - <<: *environment - networks: - net: - ipv4_address: 10.9.0.18 - ports: - - 9888:9888 - volumes: - - ${OZONE_VOLUME}/recon:/data - - *ozone-dir - - *transformation - s3g: - command: ["ozone","s3g"] - <<: *common-config - environment: - <<: *environment - networks: - net: - ipv4_address: 10.9.0.19 - ports: - - 9878:9878 - volumes: - - ${OZONE_VOLUME}/s3g:/data - - *ozone-dir - - *transformation -networks: - net: - driver: bridge - ipam: - config: - - subnet: 10.9.0.0/16 diff --git a/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-config b/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-config deleted file mode 100644 index ce4a8807e54..00000000000 --- a/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-config +++ /dev/null @@ -1,49 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -OZONE-SITE.XML_ozone.metadata.dirs=/data/metadata - -OZONE-SITE.XML_ozone.client.failover.max.attempts=6 - -OZONE-SITE.XML_ozone.om.address=om -OZONE-SITE.XML_ozone.om.http-address=om:9874 -OZONE-SITE.XML_ozone.scm.container.size=1GB -OZONE-SITE.XML_ozone.scm.datanode.ratis.volume.free-space.min=10MB -OZONE-SITE.XML_ozone.scm.pipeline.creation.interval=30s -OZONE-SITE.XML_ozone.scm.pipeline.owner.container.count=1 -OZONE-SITE.XML_ozone.scm.names=scm -OZONE-SITE.XML_ozone.scm.datanode.id.dir=/data/metadata -OZONE-SITE.XML_ozone.scm.block.client.address=scm -OZONE-SITE.XML_ozone.scm.container.size=1GB -OZONE-SITE.XML_ozone.scm.client.address=scm - -OZONE-SITE.XML_hdds.datanode.dir=/data/hdds -OZONE-SITE.XML_hdds.datanode.volume.min.free.space=100MB - -OZONE-SITE.XML_ozone.recon.db.dir=/data/metadata/recon -OZONE-SITE.XML_ozone.recon.om.snapshot.task.interval.delay=1m -OZONE-SITE.XML_hdds.scmclient.max.retry.timeout=30s -OZONE-SITE.XML_ozone.http.basedir=/tmp/ozone_http -OZONE-SITE.XML_ozone.fs.hsync.enabled=true - -OZONE_CONF_DIR=/etc/hadoop -OZONE_LOG_DIR=/var/log/hadoop - -no_proxy=om,scm,s3g,kdc,localhost,127.0.0.1 - -# Explicitly enable filesystem snapshot feature for this Docker compose cluster -# Does not take effect on Ozone versions < 1.4.0 -OZONE-SITE.XML_ozone.filesystem.snapshot.enabled=true diff --git a/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/load.sh b/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/load.sh deleted file mode 100755 index c1bd0d3724a..00000000000 --- a/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/load.sh +++ /dev/null @@ -1,29 +0,0 @@ -#!/usr/bin/env bash -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# Fail if required variables are not set. -set -u -: "${OZONE_VOLUME}" -: "${TEST_DIR}" -set +u - -source "$TEST_DIR/testlib.sh" - -export COMPOSE_FILE="$TEST_DIR/compose/non-ha/docker-compose.yaml" -create_data_dirs "${OZONE_VOLUME}"/{om,dn1,dn2,dn3,dn4,dn5,recon,s3g,scm} - -echo "Using docker cluster defined in $COMPOSE_FILE" diff --git a/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/.env b/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/.env deleted file mode 100644 index 85c422b5ad7..00000000000 --- a/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/.env +++ /dev/null @@ -1,27 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -HDDS_VERSION=${hdds.version} -OZONE_RUNNER_VERSION=${docker.ozone-runner.version} -OZONE_RUNNER_IMAGE=apache/ozone-runner -OZONE_TEST_IMAGE=apache/ozone-runner:${docker.ozone-runner.version} -OZONE_DIR=/opt/hadoop -OZONE_VOLUME=./data -OM_SERVICE_ID=omservice -# Indicates no arguments to the OM. -# This variable must be set to some non-empty value, or docker compose will -# expand it to an empty string and pass that to the OM as an argument. -OM_HA_ARGS=-- diff --git a/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/docker-compose.yaml b/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/docker-compose.yaml deleted file mode 100644 index 880b36ff2b3..00000000000 --- a/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/docker-compose.yaml +++ /dev/null @@ -1,178 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# reusable fragments (see https://docs.docker.com/compose/compose-file/#extension-fields) -x-common-config: - &common-config - env_file: - - docker-config - image: ${OZONE_TEST_IMAGE} - -x-environment: - &environment - OZONE-SITE.XML_ozone.server.default.replication: ${OZONE_REPLICATION_FACTOR:-3} - OZONE_UPGRADE_TO: ${OZONE_UPGRADE_TO:-0} - OZONE_UPGRADE_FROM: ${OZONE_UPGRADE_FROM:-0} - -x-datanode: - &datanode - command: ["ozone","datanode"] - <<: *common-config - environment: - <<: *environment - ports: - - 19864 - - 9882 - -x-om: - &om - command: ["ozone","om","${OM_HA_ARGS}"] - <<: *common-config - environment: - ENSURE_OM_INITIALIZED: /data/metadata/om/current/VERSION - <<: *environment - ports: - - 9862 - - 9872 - -x-volumes: - - &ozone-dir ../../../..:${OZONE_DIR} - - &transformation ../../../../libexec/transformation.py:/opt/hadoop/libexec/transformation.py - -services: - om1: - <<: *om - networks: - net: - ipv4_address: 10.9.0.11 - volumes: - - ${OZONE_VOLUME}/om1:/data - - *ozone-dir - - *transformation - om2: - <<: *om - networks: - net: - ipv4_address: 10.9.0.12 - volumes: - - ${OZONE_VOLUME}/om2:/data - - *ozone-dir - - *transformation - om3: - <<: *om - networks: - net: - ipv4_address: 10.9.0.13 - volumes: - - ${OZONE_VOLUME}/om3:/data - - *ozone-dir - - *transformation - - scm: - command: ["ozone","scm"] - <<: *common-config - environment: - ENSURE_SCM_INITIALIZED: /data/metadata/scm/current/VERSION - OZONE-SITE.XML_hdds.scm.safemode.min.datanode: ${OZONE_SAFEMODE_MIN_DATANODES:-1} - <<: *environment - networks: - net: - ipv4_address: 10.9.0.14 - ports: - - 9876:9876 - volumes: - - ${OZONE_VOLUME}/scm:/data - - *ozone-dir - - *transformation - dn1: - <<: *datanode - networks: - net: - ipv4_address: 10.9.0.15 - volumes: - - ${OZONE_VOLUME}/dn1:/data - - *ozone-dir - - *transformation - dn2: - <<: *datanode - networks: - net: - ipv4_address: 10.9.0.16 - volumes: - - ${OZONE_VOLUME}/dn2:/data - - *ozone-dir - - *transformation - dn3: - <<: *datanode - networks: - net: - ipv4_address: 10.9.0.17 - volumes: - - ${OZONE_VOLUME}/dn3:/data - - *ozone-dir - - *transformation - dn4: - <<: *datanode - networks: - net: - ipv4_address: 10.9.0.18 - volumes: - - ${OZONE_VOLUME}/dn4:/data - - *ozone-dir - - *transformation - dn5: - <<: *datanode - networks: - net: - ipv4_address: 10.9.0.19 - volumes: - - ${OZONE_VOLUME}/dn5:/data - - *ozone-dir - - *transformation - recon: - command: ["ozone","recon"] - <<: *common-config - environment: - <<: *environment - networks: - net: - ipv4_address: 10.9.0.20 - ports: - - 9888:9888 - volumes: - - ${OZONE_VOLUME}/recon:/data - - *ozone-dir - - *transformation - s3g: - command: ["ozone","s3g"] - <<: *common-config - environment: - <<: *environment - networks: - net: - ipv4_address: 10.9.0.21 - ports: - - 9878:9878 - volumes: - - ${OZONE_VOLUME}/s3g:/data - - *ozone-dir - - *transformation -networks: - net: - driver: bridge - ipam: - config: - - subnet: 10.9.0.0/16 diff --git a/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/docker-config b/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/docker-config deleted file mode 100644 index a049ba5f012..00000000000 --- a/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/docker-config +++ /dev/null @@ -1,65 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -OZONE-SITE.XML_ozone.metadata.dirs=/data/metadata - -OZONE-SITE.XML_ozone.client.failover.max.attempts=6 - -OZONE-SITE.XML_ozone.om.service.ids=omservice -OZONE-SITE.XML_ozone.om.nodes.omservice=om1,om2,om3 -OZONE-SITE.XML_ozone.om.address.omservice.om1=om1 -OZONE-SITE.XML_ozone.om.address.omservice.om2=om2 -OZONE-SITE.XML_ozone.om.address.omservice.om3=om3 -OZONE-SITE.XML_ozone.om.ratis.enable=true -OZONE-SITE.XML_ozone.scm.pipeline.creation.interval=30s -OZONE-SITE.XML_ozone.scm.pipeline.owner.container.count=1 -OZONE-SITE.XML_ozone.scm.names=scm -OZONE-SITE.XML_ozone.scm.datanode.id.dir=/data/metadata -OZONE-SITE.XML_ozone.scm.block.client.address=scm -OZONE-SITE.XML_ozone.scm.container.size=1GB -OZONE-SITE.XML_ozone.scm.client.address=scm -OZONE-SITE.XML_ozone.http.basedir=/tmp/ozone_http -OZONE-SITE.XML_ozone.fs.hsync.enabled=true - -OZONE-SITE.XML_hdds.datanode.dir=/data/hdds -OZONE-SITE.XML_hdds.datanode.volume.min.free.space=100MB - -# If SCM sends container close commands as part of upgrade finalization while -# datanodes are doing a leader election, all 3 replicas may end up in the -# CLOSING state. The replication manager must be running to later move them to -# a CLOSED state so the datanodes can progress with finalization. -# -# This config sets the amount of time SCM will wait after safemode exit to -# start the replication manager and pipeline scrubber. The default of 5 minutes -# is fine in real clusters to prevent unnecessary over-replication, -# but it is too long for this test. -OZONE-SITE.XML_hdds.scm.wait.time.after.safemode.exit=5s -# If datanodes take too long to close pipelines during finalization, let the -# scrubber force close them to move the test forward. -OZONE-SITE.XML_ozone.scm.pipeline.scrub.interval=1m -OZONE-SITE.XML_ozone.scm.pipeline.allocated.timeout=2m - -OZONE-SITE.XML_ozone.recon.db.dir=/data/metadata/recon -OZONE-SITE.XML_ozone.recon.om.snapshot.task.interval.delay=1m -OZONE-SITE.XML_ozone.recon.address=recon:9891 - -no_proxy=om1,om2,om3,scm,s3g,kdc,localhost,127.0.0.1 - -OM_SERVICE_ID=omservice - -# Explicitly enable filesystem snapshot feature for this Docker compose cluster -# Does not take effect on Ozone versions < 1.4.0 -OZONE-SITE.XML_ozone.filesystem.snapshot.enabled=true diff --git a/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/load.sh b/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/load.sh deleted file mode 100755 index a2f2603414f..00000000000 --- a/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/load.sh +++ /dev/null @@ -1,30 +0,0 @@ -#!/usr/bin/env bash -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# Fail if required variables are not set. -set -u -: "${OZONE_VOLUME}" -: "${TEST_DIR}" -set +u - -source "$TEST_DIR/testlib.sh" - -export COMPOSE_FILE="$TEST_DIR/compose/om-ha/docker-compose.yaml" -export OM_SERVICE_ID=omservice -create_data_dirs "${OZONE_VOLUME}"/{om1,om2,om3,dn1,dn2,dn3,dn4,dn5,recon,s3g,scm} - -echo "Using docker cluster defined in $COMPOSE_FILE" diff --git a/hadoop-ozone/dist/src/main/compose/upgrade/upgrades/manual-upgrade/README.md b/hadoop-ozone/dist/src/main/compose/upgrade/upgrades/manual-upgrade/README.md deleted file mode 100644 index 20c1c3dfb23..00000000000 --- a/hadoop-ozone/dist/src/main/compose/upgrade/upgrades/manual-upgrade/README.md +++ /dev/null @@ -1,19 +0,0 @@ - - -# Manual Upgrade Testing - -- Since the release of Ozone's non-rolling upgrade framework in 1.2.0, no manual reformatting steps need to be run on nodes between upgrades. - -- The driver for testing this type of upgrade is left here as an example in case it needs to be used for some reason in the future. diff --git a/hadoop-ozone/dist/src/main/compose/upgrade/upgrades/manual-upgrade/driver.sh b/hadoop-ozone/dist/src/main/compose/upgrade/upgrades/manual-upgrade/driver.sh deleted file mode 100755 index c77528684bc..00000000000 --- a/hadoop-ozone/dist/src/main/compose/upgrade/upgrades/manual-upgrade/driver.sh +++ /dev/null @@ -1,54 +0,0 @@ -#!/usr/bin/env bash -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# This script tests upgrade from one release to a later one. Docker -# image with Ozone binaries are required for both versions. - -set -e -o pipefail - -# Fail if required vars are not set. -set -u -: "${OZONE_UPGRADE_FROM}" -: "${OZONE_UPGRADE_TO}" -: "${TEST_DIR}" -: "${OZONE_UPGRADE_CALLBACK}" -set +u - -source "$TEST_DIR"/compose/non-ha/load.sh -source "$TEST_DIR"/testlib.sh -[[ -f "$OZONE_UPGRADE_CALLBACK" ]] && source "$OZONE_UPGRADE_CALLBACK" - -echo "--- RUNNING MANUAL UPGRADE TEST FROM $OZONE_UPGRADE_FROM TO $OZONE_UPGRADE_TO ---" - -echo "--- SETTING UP OLD VERSION $OZONE_UPGRADE_FROM ---" -OUTPUT_NAME="$OZONE_UPGRADE_FROM" -prepare_for_image "$OZONE_UPGRADE_FROM" -callback setup_old_version - -echo "--- RUNNING WITH OLD VERSION $OZONE_UPGRADE_FROM ---" -start_docker_env -callback with_old_version -stop_docker_env - -echo "--- SETTING UP NEW VERSION $OZONE_UPGRADE_TO ---" -OUTPUT_NAME="$OZONE_UPGRADE_TO" -prepare_for_image "$OZONE_UPGRADE_TO" -callback setup_this_version - -echo "--- RUNNING WITH NEW VERSION $OZONE_UPGRADE_TO ---" -OZONE_KEEP_RESULTS=true start_docker_env -callback with_this_version