Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

hook up stdlib to the standard test running system #24701

Merged
merged 2 commits into from
Nov 23, 2017
Merged

hook up stdlib to the standard test running system #24701

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
660848c
hook up stdlib to the standard test running system
KristofferC Nov 22, 2017
8f197bd
fixups from review
KristofferC Nov 22, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion test/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
SRCDIR := $(abspath $(dir $(lastword $(MAKEFILE_LIST))))
JULIAHOME := $(abspath $(SRCDIR)/..)
BUILDDIR := .
STDLIBDIR := $(abspath $(JULIAHOME)/stdlib)
include $(JULIAHOME)/Make.inc
# TODO: this Makefile ignores BUILDDIR, except for computing JULIA_EXECUTABLE

TESTGROUPS = linalg sparse unicode strings dates
TESTS = all $(TESTGROUPS) \
TESTS = all stdlib $(TESTGROUPS) \
$(patsubst $(STDLIBDIR)/%/,%,$(dir $(wildcard $(STDLIBDIR)/*/.))) \
$(filter-out TestHelpers runtests testdefs, \
$(patsubst $(SRCDIR)/%.jl,%,$(wildcard $(SRCDIR)/*.jl))) \
$(foreach group,$(TESTGROUPS), \
Expand Down
17 changes: 17 additions & 0 deletions test/choosetests.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# This file is a part of Julia. License is MIT: https://julialang.org/license

const STDLIB_DIR = joinpath(JULIA_HOME, "..", "share", "julia", "site", "v$(VERSION.major).$(VERSION.minor)")
const STDLIBS = readdir(STDLIB_DIR)

@doc """

`tests, net_on, exit_on_error, seed = choosetests(choices)` selects a set of tests to be
Expand Down Expand Up @@ -176,6 +179,20 @@ function choosetests(choices = [])
filter!(x -> !(x in net_required_for), tests)
end

if "stdlib" in skip_tests
filter!(x -> (x != "stdlib" && !(x in STDLIBS)) , tests)
elseif "stdlib" in tests
filter!(x -> (x != "stdlib" && !(x in STDLIBS)) , tests)
prepend!(tests, STDLIBS)
end

if startswith(string(Sys.ARCH), "arm")
# Remove profile from default tests on ARM since it currently segfaults
# Allow explicitly adding it for testing
warn("Skipping Profile tests")
filter!(x -> (x != "Profile"), tests)
end

filter!(x -> !(x in skip_tests), tests)

tests, net_on, exit_on_error, seed
Expand Down
21 changes: 19 additions & 2 deletions test/runtests.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,30 @@ else
typemax(Csize_t)
end

function test_path(test)
if test in STDLIBS
test_file = joinpath(STDLIB_DIR, test, "test", "runtests")
if !isfile(test_file * ".jl")
error("Standard library $test did not provide a `test/runtests.jl` file")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may be beyond the scope of this PR, but given #24628, there may or may not be a nicer way to report the error? (I didn't investigate more this question, and definitely non-blocking).

end
return test_file
else
return test
end
end

const node1_tests = String[]
function move_to_node1(t)
if t in tests
splice!(tests, findfirst(equalto(t), tests))
push!(node1_tests, t)
end
end

# Base.compile only works from node 1, so compile test is handled specially
move_to_node1("compile")
move_to_node1("SharedArrays")

# In a constrained memory environment, run the "distributed" test after all other tests
# since it starts a lot of workers and can easily exceed the maximum memory
max_worker_rss != typemax(Csize_t) && move_to_node1("distributed")
Expand Down Expand Up @@ -55,7 +70,7 @@ cd(dirname(@__FILE__)) do
local resp
wrkr = p
try
resp = remotecall_fetch(runtests, wrkr, test; seed=seed)
resp = remotecall_fetch(runtests, wrkr, test, test_path(test); seed=seed)
catch e
resp = [e]
end
Expand Down Expand Up @@ -105,9 +120,11 @@ cd(dirname(@__FILE__)) do
# and either way, append the results
# to the overall aggregator
n > 1 && print("\tFrom worker 1:\t")
isolate = true
t == "SharedArrays" && (isolate = false)
local resp
try
resp = eval(Expr(:call, () -> runtests(t, seed=seed))) # runtests is defined by the include above
resp = eval(Expr(:call, () -> runtests(t, test_path(t), isolate, seed=seed))) # runtests is defined by the include above
catch e
resp = [e]
end
Expand Down
14 changes: 0 additions & 14 deletions test/stdlib.jl
View file
Open in desktop

This file was deleted.

4 changes: 2 additions & 2 deletions test/testdefs.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

using Test

function runtests(name, isolate=true; seed=nothing)
function runtests(name, path, isolate=true; seed=nothing)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As this file is include'd, doesn't this function know about test_path ? (to avoid passing the path argument here)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I prefer to keep it explicit like this to potentially ease the use of this file in other contexts.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok no problem. But maybe then as in most of the cases the name is equal to the path, there could be a default, or even better a keyword path=name. But also I find it tricky that the path is still not the real path, as ".jl" has to be added. I guess this logic comes from the use of name before, but I see no reason now that path should not contain the .jl extension. Then the default value for path would be name * ".jl".

old_print_setting = Test.TESTSET_PRINT_ENABLE[]
Test.TESTSET_PRINT_ENABLE[] = false
try
Expand All @@ -19,7 +19,7 @@ function runtests(name, isolate=true; seed=nothing)
@timed @testset $"$name" begin
# srand(nothing) will fail
$seed != nothing && srand($seed)
include($"$name.jl")
include($"$path.jl")
end
end
res_and_time_data = eval(m, ex)
Expand Down