Skip to content

Enable java cluster test in CI. #68

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
jovany-wang wants to merge 12 commits into from
Closed

Enable java cluster test in CI. #68

jovany-wang wants to merge 12 commits into from

Conversation

@jovany-wang
Copy link
Collaborator

@jovany-wang jovany-wang commented Aug 6, 2018 •
edited
Loading

What do these changes do?

  1. Enabled cluster test in CI.
  2. Made raylet port configurable in java.

Related issue number

N/A

@jovany-wang jovany-wang requested a review from zhijunfu August 6, 2018 12:09
raulchen
raulchen reviewed Aug 6, 2018
View reviewed changes
java/cleanup.sh
pkill -9 redis
pkill -9 raylet
ps aux | grep ray | awk '{system("kill "$2);}'
rm /tmp/raylet*
Copy link

@raulchen raulchen Aug 6, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why remove this?

java/test_cluster.sh Outdated
if [ "$1" == "raylet" ]; then
sed -i 's/^use_raylet.*$/use_raylet = true/g' ray.config.ini
else
sed -i 's/^use_raylet.*$/use_raylet = false/g' ray.config.ini
Copy link

@raulchen raulchen Aug 6, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if it's a real problem. But I'd prefer not editing the file in place, because it makes the git state dirty.

Copy link
Collaborator Author

@jovany-wang jovany-wang Aug 6, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this way is not great, I will try distinguishing raylet or non raylet mode byOVERWRITE.

Copy link

@raulchen raulchen Aug 8, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using overwrite is fine, or you can create a temp config file

java/test_cluster.sh Outdated
fi

sh cleanup.sh
rm -rf local_deploy
Copy link

@raulchen raulchen Aug 6, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what will be stored in local_deploy? maybe move it to /tmp/local_deploy?

Copy link
Collaborator Author

@jovany-wang jovany-wang Aug 6, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does CURR_WORK_DIR be not suitable?
@zhijunfu What do you think?

@raulchen
Copy link

raulchen commented Aug 6, 2018

  1. I think it's better to run unit/integration tests in CI, instead of an exercise.
  2. will Java appear as a new job on the Travis build page (e.g., this one)? will raylet and non-raylet be in one job or two?

@jovany-wang
Copy link
Collaborator Author

jovany-wang commented Aug 6, 2018 •
edited
Loading

@raulchen

  1. test_cluster.sh is an integration test. Execrise02 is just an App Demo, like Hello-World Sample. Execrise02 is used for test_cluster.sh.

  2. Of cause Java tests appear as job # 5. All java tests are in job # 5, and it means that whether raylet or non raylet is in job# 5.
    This is the job # 5 script:

 "script": [
    "./java/test.sh",
    "pushd java",
    "./test_cluster.sh",
    "popd"
  ],

@jovany-wang jovany-wang mentioned this pull request Aug 17, 2018
Closed
raulchen
raulchen approved these changes Aug 20, 2018
View reviewed changes
java/test/src/main/java/org/ray/api/example/HelloExample.java
import org.ray.core.RayRuntime;

/**
* HelloWorld example for tests, like test cluster.
Copy link

@raulchen raulchen Aug 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test cluster -> test_cluster.sh

java/test_cluster.sh
[[ ${start_process_log} =~ "Started Ray head node" ]] || exit 1
echo "Check[$1]: Ray all processes started."

execution_log=$(cat "./local_deploy/ray/run/org.ray.api.example.HelloExample/0.out.txt")
Copy link

@raulchen raulchen Aug 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my PR changed the logging path to /tmp/raylogs, you should rebase and change this accordingly.

@jovany-wang
Copy link
Collaborator Author

jovany-wang commented Sep 28, 2018

Considering that we are going to get rid of RayCli, it's unnecessary to merge this PR.

@jovany-wang jovany-wang deleted the test_cluster_ci branch October 19, 2018 07:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zhijunfu zhijunfu Awaiting requested review from zhijunfu

1 more reviewer

@raulchen raulchen raulchen approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@jovany-wang @raulchen