Skip to content

YETUS-816. Improve hadoop personality to support ozone/hdds projects #46

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
aw-was-here merged 3 commits into from
Mar 7, 2019
Merged

YETUS-816. Improve hadoop personality to support ozone/hdds projects #46

Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c5940b4
adjustments to build all the ozone/hdds projects in case of an ozone …
elek Mar 5, 2019
aa1d558
disable shadedclient and mvnsite for ozone only tests
elek Mar 5, 2019
3b8f405
fixing iteration on array + ignore shellcheck errors
elek Mar 7, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 27 additions & 4 deletions precommit/src/main/shell/personality/hadoop.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -404,11 +404,34 @@ function personality_modules
fi

extra="-Ptest-patch ${extra}"

for module in $(hadoop_order ${ordering}); do
# shellcheck disable=SC2086
personality_enqueue_module ${module} ${extra}
OZONE_CHANGED=false
CORE_HADOOP_CHANGED=false
for module in $CHANGED_MODULES

@aw-was-here aw-was-here Mar 7, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CHANGED_MODULES is an array, not a string. You want "${CHANGED_MODULES[@]}" here. (Also, looks like a bug in shellcheck: it should have flagged that.)

@elek elek Mar 7, 2019

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot, fixed.

(BTW, It was not flagged, I don't know why:

precommit/src/main/shell/personality/hadoop.sh:426:46: note: Double quote to prevent globbing and word splitting. [SC2086]
precommit/src/main/shell/personality/hadoop.sh:427:47: note: Double quote to prevent globbing and word splitting. [SC2086]

I tested with a simple shell script and similar structure was flagged there. I think the type of the CHANGED_MODULES is unknown here...)

do
if [[ "$module" =~ "hdds" ]]; then
OZONE_CHANGED=true
elif [[ "$module" =~ "ozone" ]]; then
OZONE_CHANGED=true
else
CORE_HADOOP_CHANGED=true
fi
done

if [ "$OZONE_CHANGED" = true ]; then
extra="-Phdds ${extra}"
fi

if [ "$CORE_HADOOP_CHANGED" = false ] && [ "$OZONE_CHANGED" = true ]; then
if [ "$testtype" != "mvnsite" ] && [ "$testtype" != "shadedclient" ]; then
personality_enqueue_module hadoop-hdds ${extra}

@aw-was-here aw-was-here Mar 7, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shellcheck disable the message here. extra should have been an array but it wasn't when we first wrote yetus.

personality_enqueue_module hadoop-ozone ${extra}

@aw-was-here aw-was-here Mar 7, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same.

fi
else
for module in $(hadoop_order ${ordering}); do
# shellcheck disable=SC2086
personality_enqueue_module ${module} ${extra}
done
fi
}

## @description Add tests based upon personality needs
Expand Down