Skip to content

Commit fb76690

Browse files
hjoliverhjoliver
committed
Tidy test-battery.
Start all tests from global-tests.rc. test_header functions to set host and owner vars. Get rid of old CYLC_TEST_TASK_(OWNER|HOST) vars.
1 parent 7c721c2 commit fb76690

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

51 files changed

+156
-269
lines changed

‎bin/cylc-cat-log

+1
Original file line numberDiff line numberDiff line change
@@ -239,6 +239,7 @@ def get_task_job_attrs(suite_name, point, task, submit_num):
239239
live_job_id = batch_sys_job_id
240240
return (task_job_data["user_at_host"], batch_sys_name, live_job_id)
241241

242+
242243
def tmpfile_edit(tmpfile, geditor=False):
243244
"""Edit a temporary read-only file containing the string filestr.
244245

‎bin/cylc-test-battery

+27-43
Original file line numberDiff line numberDiff line change
@@ -22,46 +22,32 @@ usage() {
2222
cat <<eof
2323
Usage: cylc test-battery [...]
2424
25-
Run automated cylc and parsec tests under [FILES or DIRECTORIES].
26-
Test locations default to the following directory tree:
27-
$CYLC_DIR/tests/
25+
Run automated Cylc and Parsec tests, under (by default):
26+
$CYLC_DIR/tests/.
2827
29-
Some tests (e.g. those specific to particular batch schedulers) can be
30-
configured in your site/user config file. A few others still submit jobs
31-
to a user@host account taken from the environment:
32-
\$CYLC_TEST_TASK_HOST # default localhost
33-
\$CYLC_TEST_TASK_OWNER # defaut \$USER
28+
Options and arguments are appended to "prove -j \$NPROC -s -r \${@:-tests}".
29+
NPROC is the number of concurrent processes to run, which defaults to the
30+
global config "process pool size" setting.
3431
35-
Requirements:
36-
* Passwordless ssh must be configured to task host accounts.
37-
* Some test suites submit jobs to 'at' so atd must be running.
32+
The tests ignore normal site/user global config and instead use the file:
33+
$CYLC_DIR/conf/global-tests.rc
34+
This should specify test job hosts under the [test battery] section, plus any
35+
other critical settings settings, including [hosts] configuration for test job
36+
hosts (and special batchview commands like qcat if available). Additional
37+
global config items can be added on the fly using the create_test_globalrc
38+
shell function defined in the test_header.
3839
39-
Options and arguments are appended to the "prove -j \$NPROC -s -r \${@:-tests}"
40-
command, where NPROC is the number of child processes that can be used to run
41-
the test files.
40+
Suite run directories are only cleaned up for passing tests on the suite host.
4241
43-
Some tests use a clean global config file. If some items from your site config
44-
file are needed in this, e.g. to get remote test hosts working, add them to
45-
$CYLC_DIR/conf/global-tests.rc.
42+
Set "export CYLC_TEST_DEBUG=true" to print failed-test stderr to the terminal.
4643
47-
The command normally uses the "process pool size" setting (default=4) in the
48-
site/user global configuration file to determine the number of tests to run in
49-
parallel. You can also change the amount of concurrency with the "-j N" option.
44+
To change the test file comparision command from "diff -u" do (for example):
45+
export CYLC_TEST_DIFF_CMD='xxdiff -D'
5046
51-
Suite run directories are cleaned up on the suite host for passing tests -
52-
otherwise they are left alone.
53-
54-
To output stderr from failed tests to the terminal,
55-
"export CYLC_TEST_DEBUG=true" before running this command.
56-
57-
The command normally uses "diff -u" to compare files. However, if an alternate
58-
command such as "xxdiff -D" is desirable (e.g. for debugging), "export
59-
CYLC_TEST_DIFF_CMD=xxdiff -D".
47+
Some test suites submit jobs to the 'at' so atd must be up on the job hosts.
6048
6149
Commits or Pull Requests to cylc/cylc on GitHub will trigger Travis CI to run
6250
generic (non platform-specific) tests - see $CYLC_DIR/.travis.yml.
63-
After enabling Travis CI for your own cylc fork, you can skip generic tests
64-
locally by setting CYLC_TEST_RUN_GENERIC=false.
6551
6652
By default all tests are executed. To run just a subset of them:
6753
* list individual tests or test directories to run on the comand line
@@ -71,36 +57,34 @@ By default all tests are executed. To run just a subset of them:
7157
List specific tests relative to $CYLC_DIR (i.e. starting with "test/").
7258
Some platform-specific tests are automatically skipped, depending on platform.
7359
74-
FOR DEVELOPERS:
75-
* Platform-specific tests must set "CYLC_TEST_IS_GENERIC=false" before
76-
sourcing the test_header.
77-
* Tests requiring the sqlite3 CLI must be skipped if sqlite3 is not installed
78-
(it is not otherwise a Cylc software prerequisite):
60+
Platform-specific tests must set "CYLC_TEST_IS_GENERIC=false" before sourcing
61+
the test_header.
62+
63+
Tests requiring the sqlite3 CLI must be skipped if sqlite3 is not installed (it
64+
is not otherwise a Cylc software prerequisite):
7965
| if ! which sqlite3 > /dev/null; then
8066
| # Skip the remaining 3 tests.
8167
| skip 3 "sqlite3 not installed?"
8268
| purge_suite \$SUITE_NAME
8369
| exit 0
8470
| fi
8571
86-
For more information see "Reference Tests" in the User Guide.
87-
8872
Options:
8973
-h, --help Print this help message and exit.
9074
9175
Examples:
9276
9377
Run the full test suite with the default options.
9478
cylc test-battery
95-
Run the full test suite with 12 processes.
79+
Run the full test suite with 12 processes
9680
cylc test-battery -j 12
97-
Run only tests under "tests/cyclers/" with 12 processes.
98-
cylc test-battery -j 12 tests/cyclers
81+
Run only tests under "tests/cyclers/"
82+
cylc test-battery tests/cyclers
9983
Run only "tests/cyclers/16-weekly.t" in verbose mode
10084
cylc test-battery -v tests/cyclers/16-weekly.t
101-
Run only tests under "tests/cyclers/" with 12 processes, and skip 00-daily.t
85+
Run only tests under "tests/cyclers/", and skip 00-daily.t
10286
export CYLC_TEST_SKIP=tests/cyclers/00-daily.t
103-
cylc test-battery -j 12 tests/cyclers
87+
cylc test-battery tests/cyclers
10488
eof
10589
}
10690

‎doc/src/cylc-user-guide/siterc.tex

+3-1
Original file line numberDiff line numberDiff line change
@@ -807,7 +807,9 @@ \subsection{[task events]}
807807

808808
\subsection{[test battery]}
809809

810-
Settings for the automated development tests.
810+
Settings for the automated development tests. Note the test battery reads
811+
\lstinline=$CYLC_DIR/conf/global-tests.rc= instead of the normal site/user
812+
global config files.
811813

812814
\subsubsection[remote host with shared fs]{[test battery] \textrightarrow remote host with shared fs}
813815

‎lib/cylc/scheduler.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -624,7 +624,7 @@ def info_get_task_jobfile_path(self, task_id):
624624
"""Return task job file path."""
625625
name, point = TaskID.split(task_id)
626626
return get_task_job_log(
627-
self.suite, point, name, tail=JOB_LOG_JOB)
627+
self.suite, point, name, suffix=JOB_LOG_JOB)
628628

629629
def info_get_suite_info(self):
630630
"""Return a dict containing the suite title and description."""

‎tests/authentication/08-shared-fs.t

+1-7
Original file line numberDiff line numberDiff line change
@@ -19,13 +19,7 @@
1919
# current host.
2020
CYLC_TEST_IS_GENERIC=false
2121
. "$(dirname "$0")/test_header"
22-
23-
export CYLC_TEST_HOST=$( \
24-
cylc get-global-config -i '[test battery]remote host with shared fs' \
25-
2>'/dev/null')
26-
if [[ -z "${CYLC_TEST_HOST}" ]]; then
27-
skip_all '"[test battery]remote host with shared fs": not defined'
28-
fi
22+
set_test_remote_host
2923
set_test_number 4
3024

3125
# "install_suite" does not work here because it installs suites on the TMPDIR,

‎tests/authentication/09-remote-suite-same-name.t

+1-6
Original file line numberDiff line numberDiff line change
@@ -20,12 +20,7 @@
2020
# be very confused if it is running under its ~/cylc-run/SUITE as well.)
2121
CYLC_TEST_IS_GENERIC=false
2222
. "$(dirname "$0")/test_header"
23-
24-
export CYLC_TEST_HOST=$( \
25-
cylc get-global-config -i '[test battery]remote host' 2>'/dev/null')
26-
if [[ -z "${CYLC_TEST_HOST}" ]]; then
27-
skip_all '"[test battery]remote host": not defined'
28-
fi
23+
set_test_remote_host
2924
set_test_number 2
3025

3126
install_suite "${TEST_NAME_BASE}" "${TEST_NAME_BASE}"

‎tests/authentication/10-remote-suite-passphrase-cache.t

+1-6
Original file line numberDiff line numberDiff line change
@@ -21,12 +21,7 @@
2121
# remote host.
2222
CYLC_TEST_IS_GENERIC=false
2323
. "$(dirname "$0")/test_header"
24-
25-
export CYLC_TEST_HOST=$( \
26-
cylc get-global-config -i '[test battery]remote host' 2>'/dev/null')
27-
if [[ -z "${CYLC_TEST_HOST}" ]]; then
28-
skip_all '"[test battery]remote host": not defined'
29-
fi
24+
set_test_remote_host
3025
set_test_number 5
3126

3227
SSH_OPTS='-oBatchMode=yes -oConnectTimeout=5'

‎tests/broadcast/09-remote.t

+1-6
Original file line numberDiff line numberDiff line change
@@ -18,12 +18,7 @@
1818
# Test broadcast from remote task job.
1919
CYLC_TEST_IS_GENERIC=false
2020
. "$(dirname "$0")/test_header"
21-
22-
export CYLC_TEST_HOST=$(cylc get-global-config -i '[test battery]remote host' \
23-
2>'/dev/null')
24-
if [[ -z $CYLC_TEST_HOST ]]; then
25-
skip_all '"[test battery]remote host": not defined'
26-
fi
21+
set_test_remote_host
2722
set_test_number 2
2823
install_suite "${TEST_NAME_BASE}" "${TEST_NAME_BASE}"
2924

‎tests/cylc-cat-log/01-remote.t

+2-6
Original file line numberDiff line numberDiff line change
@@ -19,15 +19,11 @@
1919
CYLC_TEST_IS_GENERIC=false
2020
. $(dirname $0)/test_header
2121
#-------------------------------------------------------------------------------
22-
export CYLC_TEST_HOST=$(cylc get-global-config -i "[test battery]remote host" 2>'/dev/null')
23-
export CYLC_TEST_OWNER=$(cylc get-global-config -i "[test battery]remote owner" 2>'/dev/null')
24-
if [[ -z ${CYLC_TEST_HOST}${CYLC_TEST_OWNER} ]]; then
25-
skip_all '"[test battery]remote host": not defined'
26-
fi
22+
set_test_remote
2723
set_test_number 14
2824
create_test_globalrc "" "
2925
[hosts]
30-
[[${CYLC_TEST_HOST:-localhost}]]
26+
[[${CYLC_TEST_HOST}]]
3127
retrieve job logs = False"
3228
install_suite $TEST_NAME_BASE $TEST_NAME_BASE
3329
#-------------------------------------------------------------------------------

‎tests/cylc-cat-log/05-remote-tail.t

+1-10
Original file line numberDiff line numberDiff line change
@@ -18,17 +18,8 @@
1818
# Test "cylc cat-log" with a custom remote tail command.
1919
CYLC_TEST_IS_GENERIC=false
2020
. $(dirname $0)/test_header
21+
set_test_remote
2122
#-------------------------------------------------------------------------------
22-
create_test_globalrc
23-
CYLC_TEST_HOST="$( \
24-
cylc get-global-config -i '[test battery]remote host' 2>'/dev/null')"
25-
CYLC_TEST_OWNER="$( \
26-
cylc get-global-config -i '[test battery]remote owner' 2>'/dev/null')"
27-
if [[ -z "${CYLC_TEST_HOST}${CYLC_TEST_OWNER}" ]]; then
28-
skip_all '"[test battery]remote host/owner": not defined'
29-
fi
30-
export CYLC_TEST_HOST=${CYLC_TEST_HOST:-"localhost"}
31-
export CYLC_TEST_OWNER=${CYLC_TEST_OWNER:-${USER}}
3223
set_test_number 4
3324
install_suite $TEST_NAME_BASE $TEST_NAME_BASE
3425
set -eu

‎tests/cylc-cat-log/08-editor-remote.t

+3-11
Original file line numberDiff line numberDiff line change
@@ -18,20 +18,12 @@
1818
# Test "cylc cat-log" open local logs in editor.
1919

2020
. "$(dirname $0)"/test_header
21-
22-
create_test_globalrc
23-
24-
HOST="$( cylc get-global-config -i '[test battery]remote host' 2>'/dev/null')"
25-
OWNER="$( cylc get-global-config -i '[test battery]remote owner' 2>'/dev/null')"
26-
if [[ -z "${OWNER}${HOST}" ]]; then
27-
skip_all '"[test battery]remote host/owner": not defined'
28-
fi
29-
CYLC_TEST_HOST=${CYLC_TEST_HOST:-"localhost"}
30-
CYLC_TEST_OWNER=${CYLC_TEST_OWNER:-${USER}}
21+
set_test_remote
3122

3223
. "${TEST_SOURCE_DIR}"/editor/bin/run_tests.sh
3324
export PATH="${TEST_SOURCE_DIR}/editor/bin/":"${PATH}"
3425

3526
install_suite "${TEST_NAME_BASE}" "editor"
36-
run_tests "${HOST}" "${OWNER}"
27+
run_tests "${CYLC_TEST_HOST}" "${CYLC_TEST_OWNER}"
3728
purge_suite "${SUITE_NAME}"
29+
purge_suite_remote "${CYLC_TEST_OWNER}@${CYLC_TEST_HOST}" "${SUITE_NAME}"

‎tests/cylc-get-site-config/00-basic.t

-1
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,6 @@
1919
. $(dirname $0)/test_header
2020
#-------------------------------------------------------------------------------
2121
set_test_number 10
22-
create_test_globalrc
2322
#-------------------------------------------------------------------------------
2423
TEST_NAME=$TEST_NAME_BASE-get-config
2524
run_ok $TEST_NAME.validate cylc get-site-config

‎tests/cylc-get-site-config/01-defaults.t

-3
Original file line numberDiff line numberDiff line change
@@ -21,9 +21,6 @@
2121

2222
set_test_number 1
2323

24-
# Configure a new site config file.
25-
create_test_globalrc
26-
2724
# Empty it (of non-default global-tests.rc items, which would then be retrieved
2825
# by "cylc get-global-config" below).
2926
echo '' > "$CYLC_CONF_PATH/global.rc"

‎tests/cylc-kill/00-multi-hosts-compat.t

+1-7
Original file line numberDiff line numberDiff line change
@@ -18,15 +18,9 @@
1818
# Test kill multiple jobs on localhost and a remote host
1919
CYLC_TEST_IS_GENERIC=false
2020
. "$(dirname "$0")/test_header"
21-
export CYLC_TEST_HOST=$(
22-
cylc get-global-config -i '[test battery]remote host' 2>'/dev/null')
23-
if [[ -z "${CYLC_TEST_HOST}" ]]; then
24-
skip_all '"[test battery]remote host": not defined'
25-
fi
26-
21+
set_test_remote_host
2722
set_test_number 3
2823

29-
create_test_globalrc
3024
install_suite "${TEST_NAME_BASE}" "${TEST_NAME_BASE}"
3125

3226
run_ok "${TEST_NAME_BASE}-validate" \

‎tests/cylc-kill/01-multi-hosts.t

+1-7
Original file line numberDiff line numberDiff line change
@@ -18,15 +18,9 @@
1818
# Test kill multiple jobs on localhost and a remote host
1919
CYLC_TEST_IS_GENERIC=false
2020
. "$(dirname "$0")/test_header"
21-
export CYLC_TEST_HOST=$(
22-
cylc get-global-config -i '[test battery]remote host' 2>'/dev/null')
23-
if [[ -z "${CYLC_TEST_HOST}" ]]; then
24-
skip_all '"[test battery]remote host": not defined'
25-
fi
26-
21+
set_test_remote_host
2722
set_test_number 3
2823

29-
create_test_globalrc
3024
install_suite "${TEST_NAME_BASE}" "${TEST_NAME_BASE}"
3125

3226
run_ok "${TEST_NAME_BASE}-validate" \

‎tests/cylc-message/00-ssh.t

+2-6
Original file line numberDiff line numberDiff line change
@@ -20,11 +20,7 @@
2020
CYLC_TEST_IS_GENERIC=false
2121
. "$(dirname "$0")/test_header"
2222
#-------------------------------------------------------------------------------
23-
CYLC_TEST_HOST="$( \
24-
cylc get-global-config -i '[test battery]remote host' 2>'/dev/null')"
25-
if [[ -z "${CYLC_TEST_HOST}" ]]; then
26-
skip_all '"[test battery]remote host": not defined'
27-
fi
23+
set_test_remote_host
2824
set_test_number 3
2925

3026
create_test_globalrc '' "
@@ -38,7 +34,7 @@ run_ok "${TEST_NAME_BASE}-validate" \
3834
cylc validate "${SUITE_NAME}" -s "CYLC_TEST_HOST=${CYLC_TEST_HOST}"
3935
suite_run_ok "${TEST_NAME_BASE}-run" \
4036
cylc run --debug --no-detach --reference-test "${SUITE_NAME}" \
41-
-s "CYLC_TEST_HOST=${CYLC_TEST_HOST}"
37+
-s "CYLC_TEST_HOST=${CYLC_TEST_HOST}"
4238

4339
run_fail "${TEST_NAME_BASE}-grep-DENIED-suite-log" \
4440
grep -q "\\[client-connect\\] DENIED .*@${CYLC_TEST_HOST}:cylc-message" \

‎tests/cylc-poll/04-poll-multi-hosts.t

+1-7
Original file line numberDiff line numberDiff line change
@@ -18,15 +18,9 @@
1818
# Test poll multiple jobs on localhost and a remote host
1919
CYLC_TEST_IS_GENERIC=false
2020
. "$(dirname "$0")/test_header"
21-
export CYLC_TEST_HOST=$( \
22-
cylc get-global-config -i '[test battery]remote host' 2>'/dev/null')
23-
if [[ -z "${CYLC_TEST_HOST}" ]]; then
24-
skip_all '"[test battery]remote host": not defined'
25-
fi
26-
21+
set_test_remote_host
2722
set_test_number 3
2823

29-
create_test_globalrc
3024
install_suite "${TEST_NAME_BASE}" "${TEST_NAME_BASE}"
3125

3226
run_ok "${TEST_NAME_BASE}-validate" \

‎tests/cylc-poll/05-poll-multi-messages.t

-1
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,6 @@
1919
. "$(dirname "$0")/test_header"
2020
set_test_number 2
2121

22-
create_test_globalrc
2322
install_suite "${TEST_NAME_BASE}" "${TEST_NAME_BASE}"
2423

2524
run_ok "${TEST_NAME_BASE}-validate" cylc validate "${SUITE_NAME}"

‎tests/cylc-scan/00-simple.t

-1
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,6 @@
1919
. $(dirname $0)/test_header
2020
#-------------------------------------------------------------------------------
2121
set_test_number 2
22-
create_test_globalrc
2322
#-------------------------------------------------------------------------------
2423
install_suite $TEST_NAME_BASE simple
2524
#-------------------------------------------------------------------------------

‎tests/cylc-scan/02-sigstop.t

-1
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,6 @@
1818
# Test "cylc scan" on suite suspended with SIGSTOP
1919
. "$(dirname "$0")/test_header"
2020
set_test_number 4
21-
create_test_globalrc
2221
install_suite "${TEST_NAME_BASE}" "${TEST_NAME_BASE}"
2322

2423
run_ok "${TEST_NAME_BASE}-validate" cylc validate "${SUITE_NAME}"

‎tests/database/03-remote.t

+1-5
Original file line numberDiff line numberDiff line change
@@ -18,11 +18,7 @@
1818
# Suite database content, "task_jobs" table with a remote job.
1919
CYLC_TEST_IS_GENERIC=false
2020
. "$(dirname "$0")/test_header"
21-
export CYLC_TEST_HOST=$( \
22-
cylc get-global-config -i '[test battery]remote host' 2>'/dev/null')
23-
if [[ -z "${CYLC_TEST_HOST}" ]]; then
24-
skip_all '"[test battery]remote host": not defined'
25-
fi
21+
set_test_remote_host
2622
set_test_number 3
2723
install_suite "${TEST_NAME_BASE}" "${TEST_NAME_BASE}"
2824

‎tests/events/01-task.t

-1
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,6 @@
2121
set_test_number 2
2222
#-------------------------------------------------------------------------------
2323
install_suite $TEST_NAME_BASE task
24-
create_test_globalrc
2524
#-------------------------------------------------------------------------------
2625
TEST_NAME=$TEST_NAME_BASE-validate
2726
run_ok $TEST_NAME cylc validate $SUITE_NAME

0 commit comments

Comments
 (0)