HDDS-4450. Cannot run ozone if HADOOP_HOME points to Hadoop install

[adoroszlai]

What changes were proposed in this pull request?

If HADOOP_HOME does not point to Ozone install, make an attempt to find it relative to the script being run. This logic is copied from start-ozone.sh, where it was added for HDDS-1912 to fix similar problem.

https://issues.apache.org/jira/browse/HDDS-4450

How was this patch tested?

Tested using ozonescripts compose environment after temporarily adding HADOOP_HOME=/usr/local/hadoop to docker-config. Verified that all three scripts (ozone and stop-ozone.sh, changed here, and start-ozone.sh, changed previously) work fine.

cd hadoop-ozone/dist/target/ozone-1.1.0-SNAPSHOT/compose/ozonescripts
docker-compose up -d
# <wait a bit>
./start.sh
...
./ps.sh
# <output with Ozone processes>
docker-compose exec scm bash
bash-4.2$ ozone freon ockg -n1 -t1 -F ONE
...
Successful executions: 1
bash-4.2$ echo $HADOOP_HOME
/usr/local/hadoop
bash-4.2$ exit
./stop.sh
...
./ps.sh
# <no more Ozone process>

[elek]

I guess, it's an incompatible change, but should be acceptable IMHO.

[avijayanhwx]

@elek Can you tell me how this is an incompatible change?

[elek]

Can you tell me how this is an incompatible change?

Based on my understanding HADOOP_HOME was usable earlier to point to any Ozone install, now it's forced to point a specific dir.

The chance is very small for this behavior is used by any cluster manager, but I would like to be sure it's safe. (earlier I accidentally committed incompatible changes.)

But as you are fine with it, let's merge it.

Thanks the patch @adoroszlai and review @avijayanhwx

[adoroszlai]

Thanks @elek for merging it and @avijayanhwx for the review.

Based on my understanding HADOOP_HOME was usable earlier to point to any Ozone install, now it's forced to point a specific dir.

The script is happy as long as it finds the ozone-config.sh file, does not care about specific directories, versions or instances. HADOOP_HOME is only forced if it does not already point to any Ozone install, rather to Hadoop or something completely different or maybe a non-existent dir. In this case, ozone (and stop-ozone.sh) did not work prior to this change.

Also, as long as Ozone scripts are executed, not sourced, updating the variables does not affect the caller.

Normally HADOOP_HOME and other environment variables can be defined for each component separately. However, in an Impala mini cluster (used for running tests), all components share the same environment. HADOOP_HOME points to Hadoop, so we need a way to override this "locally" for Ozone. This change allows that without extensive and potentially dangerous refactoring (renaming all HADOOP variables and updating Ozone's copy of hadoop functions etc.).

[elek]

❤️ Thanks for the detailed explanation.