Tests run after core explode in time and allocations

[ararslan]

When the bitarray test suite is run on its own, it finishes reasonably quickly:

$ JULIA_CPU_CORES=1 ./julia test/runtests.jl bitarray 
Test (Worker) | Time (s) | GC (s) | GC % | Alloc (MB) | RSS (MB)
bitarray (1)  |   66.34  |  0.95  |  1.4 | 2924.40    | 304.91

But if the core testsuite is run first, the time spent on bitarray and the amount of allocated memory explodes:

$ JULIA_CPU_CORES=1 ./julia test/runtests.jl core bitarray 
Test (Worker) | Time (s) | GC (s) | GC % | Alloc (MB) | RSS (MB) 
core (1)      |   71.98  |  3.29  |  4.6 | 8573.17    | 594.81
bitarray (1)  |  863.07  | 38.03  |  4.4 | 134423.97  | 594.81

This also happens with the arrayops test suite (and possibly others).

This has been observed on Linux, FreeBSD, and macOS. The timings given here were collected by @iblis17 on Linux. The extra time needed for the test suite that follows core has been causing the FreeBSD CI workers to time out, since the suite can take upwards of 20 minutes to run and the time limit for builds producing no output is 20 (maybe 30?) minutes.

[andreasnoack]

This appears to be caused just be the Array constructors in

julia/test/core.jl

Lines 5462 to 5468 in 1a3a633

	A = Array{U}(ntuple(x->0, N)...)
	@test isempty(A)
	@test sizeof(A) == 0
	len = ntuple(x->10, N)
	mxsz = maximum(sizeof, Base.uniontypes(U))
	A = Array{U}(len)

. I've commented out everything in the file except what is required for the construction of the arrays and I still see the slowdown. If I comment out the construction then the slowdown is gone.

[quinnj]

It's probably mostly due to the array construction specifically of the Union here, which was previously reported as having some super weirdness with related to performance. We could just comment out that Union for now until #22688 is fixed.

[JeffBezanson]

I can't reproduce this on current master, but FreeBSD still seems to time out fairly often so something else might be able to trigger this as well.

[iblislin]

About recent CI freezing, I attached to one of freezed process on my worker:

[venv] julia@Gaebolg:~ % lldb -p 17750
(lldb) process attach --pid 17750
threadProcess 17750 stopped

Executable module set to "/home/julia/ci/worker/11rel-amd64/build/usr/bin/julia".
Architecture set to: x86_64--freebsd11.0.
(lldb) thread list
Process 17750 stopped
* thread #1: tid = 101039, 0x0000000802188498 libc.so.7`__sys_openat + 8
  thread #2: tid = 100360, 0x00000008021882ba libc.so.7`_sigwait + 10
(lldb) bt
* thread #1: tid = 101039, 0x0000000802188498 libc.so.7`__sys_openat + 8
  * frame #0: 0x0000000802188498 libc.so.7`__sys_openat + 8
    frame #1: 0x00000008040fe586 libthr.so.3`??? + 150
    frame #2: 0x00000008020ed287 libc.so.7`open + 199
    frame #3: 0x0000000800b71c6f libjulia.so.0.7`ios_file [inlined] open_cloexec(mode=438) + 5 at ios.c:901 [opt]
    frame #4: 0x0000000800b71c6a libjulia.so.0.7`ios_file(s=<unavailable>, fname=<unavailable>, rd=1, wr=<unavailable>, cr
eate=<unavailable>, trunc=<unavailable>) + 202 at ios.c:929 [opt] 
    frame #5: 0x0000000804bae0fe sys.so`julia_open_19095 + 334 at iostream.jl:143
    frame #6: 0x0000000804c8c1db sys.so`japi1_open_23127 + 427 at iostream.jl:171
    frame #7: 0x00000008554f2ee6 
    frame #8: 0x0000000800a472c4 libjulia.so.0.7`jl_apply_generic [inlined] jl_call_fptr_internal(meth=0x00007fffffffd0d8,
 args=<unavailable>, nargs=<unavailable>) + 49 at julia_internal.h:366 [opt]
    frame #9: 0x0000000800a47293 libjulia.so.0.7`jl_apply_generic [inlined] jl_call_method_internal(meth=0x00007fffffffd0d
8, args=<unavailable>, nargs=<unavailable>) + 16 at julia_internal.h:385 [opt]
...

I think it's #24037

[iblislin]

there is a mysterious race condition

[ararslan]

I also can't reproduce the originally mentioned issue on current master with FreeBSD or macOS. When run after core I get about 1.5 minutes for bitarray on FreeBSD and about 2 on macOS.

[iblislin]

Cannot reproduce also. I'm ok with closing this issue.

[ararslan]

Okay, I'll go ahead and close this. Please open an issue to track the current FreeBSD freezes.