`=== nothing` is not `== nothing` with `--code-coverage=user` #32579

tkf · 2019-07-15T02:34:02Z

I just have a very strange bug that cannot be reproduced locally and replacing === nothing with == nothing (as below) fixes this bug.

diff --git a/examples/primes.jl b/examples/primes.jl
index 7dac9d2f..0f194e7f 100644
--- a/examples/primes.jl
+++ b/examples/primes.jl
@@ -11,7 +11,7 @@ using Transducers

 function sieve(xf, x)
     @info "sieve(xf, x=$x)"
-    if (@show mapfoldl(xf, right, (x,), init=nothing)) === nothing
+    if (@show mapfoldl(xf, right, (x,), init=nothing)) == nothing
         @info "$x is NOT prime"
         nothing, xf
     else

See: https://travis-ci.com/tkf/Transducers.jl/builds/119090143 (Ref JuliaFolds/Transducers.jl#17)

Maybe it's relevant to #32135?

The text was updated successfully, but these errors were encountered:

tkf · 2019-07-15T03:11:28Z

I tried other variants:

isa Nothing works https://travis-ci.com/tkf/Transducers.jl/builds/119091015
isnothing works https://travis-ci.com/tkf/Transducers.jl/builds/119092120
!== x where x is not a nothing works https://travis-ci.com/tkf/Transducers.jl/builds/119092147

StefanKarpinski · 2019-07-15T05:17:05Z

This seems like it shouldn’t happen.

vtjnash · 2019-07-15T19:38:41Z

Seems like someone may have deleted the build log for the OP?

Keno · 2019-07-15T19:47:44Z

I can reproduce this locally if I try using Run.jl as in the travis script for that package.

Keno · 2019-07-15T19:52:51Z

Looks like the difference is the --code-coverage=user flag.

Keno · 2019-07-15T20:20:08Z

Looking at the IR, I see:

23 ┄ %44  = φ (#21 => false)::Bool
│    %45  = φ (#21 => _3, #22 => nothing)::Union{Nothing, Int64}
└───        goto #25
24 ─        goto #25
25 ┄ %48  = φ (#23 => %44)::Bool
│    %49  = φ (#23 => %45, #24 => nothing)::Union{Nothing, Int64}
└───        goto #26
26 ─ %51  = (isa)(%49, Nothing)::Bool
└───        goto #28 if not %51
27 ─ %53  = π (%49, Nothing)
└───        goto #31
28 ─ %55  = (isa)(%49, Int64)::Bool
└───        goto #30 if not %55
29 ─ %57  = π (%49, Int64)
└───        goto #31
30 ─        Core.throw(ErrorException("fatal error in type inference (type bound)"))::Union{}
└───        $(Expr(:unreachable))::Union{}
31 ┄ %61  = φ (#27 => %48, #29 => %48)::Bool

which looks like an optimizer bug, probably in type lifting.

tkf · 2019-07-15T23:10:12Z

using Run.jl

Ah, I thought I've tried this... But yes, I can reproduce it with Run.test() now.

Edit: maybe the easiest way to run the script is julia --startup-file=no --project=test --code-coverage=user examples/primes.jl

tkoolen · 2019-07-16T05:51:53Z

Maybe related to #30872?

The bug here is a bit subtle, but perhaps best illustrated with the included test case: ``` function f32579(x::Int64, b::Bool) if b x = nothing end if isa(x, Int64) y = x else y = x end if isa(y, Nothing) z = y else z = y end return z === nothing end ``` The code just after SSA conversion looks like: ``` 2 1 ─ goto #3 if not _3 3 2 ─ %2 = Main.nothing::Core.Compiler.Const(nothing, false) 5 3 ┄ %3 = φ (#2 => %2, #1 => _2)::Union{Nothing, Int64} │ %4 = (%3 isa Main.Int64)::Bool └── goto #5 if not %4 6 4 ─ %6 = π (%3, Int64) └── goto #6 8 5 ─ %8 = π (%3, Nothing) 10 6 ┄ %9 = φ (#4 => %6, #5 => %8)::Union{Nothing, Int64} │ %10 = (%9 isa Main.Nothing)::Bool └── goto #8 if not %10 11 7 ─ %12 = π (%9, Nothing) └── goto #9 13 8 ─ %14 = π (%9, Int64) 15 9 ┄ %15 = φ (#7 => %12, #8 => %14)::Union{Nothing, Int64} │ %16 = (%15 === Main.nothing)::Bool └── return %16 ``` Now, we have special code in SROA (despite it not really being an SROA transform) that looks at `===` and replaces it by a nest of phis of booleans. The reasoning for this transform is that it eliminates a use of a value where we only care about the type and not the content, thus making it more likely that the value will subsequently be eligible for SROA. In addition, while it goes along resolving which values feed into any particular phi, it accumulates and type conditions it encounters along the way. Thus in the example above, something like the following happens: - We look at %14, which πs to %9 with an Int64 constraint, so we only consider the #4 predecessor for %9 (due to the constraint), until we get to %3, where we again only consider the #1 predecessor, where we find the argument (of type Int64) and conclude the result is always false - Now we pop the next item of the stack from our original phi, look at %12, which πs to %9 with a Nothing constraint. At this point we used to terminate the search because we already looked at %9. However, crucially, we looked at %9 only with an Int64 constraint, so we missed the fact that `nothing` was in fact a possible value for this phi. The result was a missing entry in the generated phi node: ``` 1 ─ goto #3 if not b 2 ─ %2 = Main.nothing::Core.Compiler.Const(nothing, false) 3 ┄ %3 = φ (#1 => false)::Bool │ %4 = φ (#2 => %2, #1 => _2)::Union{Nothing, Int64} │ %5 = (%4 isa Main.Int64)::Bool └── goto #5 if not %5 4 ─ %7 = π (%4, Int64) └── goto #6 5 ─ %9 = π (%4, Nothing) 6 ┄ %10 = φ (#4 => %3, #5 => %3)::Bool │ %11 = φ (#4 => %7, #5 => %9)::Union{Nothing, Int64} │ %12 = (%11 isa Main.Nothing)::Bool └── goto #8 if not %12 7 ─ goto #9 8 ─ nothing::Nothing 9 ┄ %16 = φ (#7 => %10, #8 => %10)::Bool └── return %16 ``` (note the missing #2 predecessor in phi node %3), which would result in an undefined value at runtime, though in this case LLVM would have taken advantage of that to just return 0: ``` define i8 @julia_f32579_16051(i64, i8) { top: ; @ REPL[1]:15 within `f32579' ret i8 0 } ``` Compare this now to the optimized IR with this patch: ``` 1 ─ goto #3 if not b 2 ─ %2 = Main.nothing::Core.Compiler.Const(nothing, false) 3 ┄ %3 = φ (#2 => true, #1 => false)::Bool │ %4 = φ (#2 => %2, #1 => _2)::Union{Nothing, Int64} │ %5 = (%4 isa Main.Int64)::Bool └── goto #5 if not %5 4 ─ %7 = π (%4, Int64) └── goto #6 5 ─ %9 = π (%4, Nothing) 6 ┄ %10 = φ (#4 => %3, #5 => %3)::Bool │ %11 = φ (#4 => %7, #5 => %9)::Union{Nothing, Int64} │ %12 = (%11 isa Main.Nothing)::Bool └── goto #8 if not %12 7 ─ goto #9 8 ─ nothing::Nothing 9 ┄ %16 = φ (#7 => %10, #8 => %10)::Bool └── return %16 ``` The %3 phi node has its missing entry and the generated LLVM code correctly returns `b`: ``` define i8 @julia_f32579_16112(i64, i8) { top: %2 = and i8 %1, 1 ; @ REPL[1]:15 within `f32579' ret i8 %2 } ```

The bug here is a bit subtle, but perhaps best illustrated with the included test case: ``` function f32579(x::Int64, b::Bool) if b x = nothing end if isa(x, Int64) y = x else y = x end if isa(y, Nothing) z = y else z = y end return z === nothing end ``` The code just after SSA conversion looks like: ``` 2 1 ─ goto #3 if not _3 3 2 ─ %2 = Main.nothing::Core.Compiler.Const(nothing, false) 5 3 ┄ %3 = φ (#2 => %2, #1 => _2)::Union{Nothing, Int64} │ %4 = (%3 isa Main.Int64)::Bool └── goto #5 if not %4 6 4 ─ %6 = π (%3, Int64) └── goto #6 8 5 ─ %8 = π (%3, Nothing) 10 6 ┄ %9 = φ (#4 => %6, #5 => %8)::Union{Nothing, Int64} │ %10 = (%9 isa Main.Nothing)::Bool └── goto #8 if not %10 11 7 ─ %12 = π (%9, Nothing) └── goto #9 13 8 ─ %14 = π (%9, Int64) 15 9 ┄ %15 = φ (#7 => %12, #8 => %14)::Union{Nothing, Int64} │ %16 = (%15 === Main.nothing)::Bool └── return %16 ``` Now, we have special code in SROA (despite it not really being an SROA transform) that looks at `===` and replaces it by a nest of phis of booleans. The reasoning for this transform is that it eliminates a use of a value where we only care about the type and not the content, thus making it more likely that the value will subsequently be eligible for SROA. In addition, while it goes along resolving which values feed into any particular phi, it accumulates and type conditions it encounters along the way. Thus in the example above, something like the following happens: - We look at %14, which πs to %9 with an Int64 constraint, so we only consider the #4 predecessor for %9 (due to the constraint), until we get to %3, where we again only consider the #1 predecessor, where we find the argument (of type Int64) and conclude the result is always false - Now we pop the next item of the stack from our original phi, look at %12, which πs to %9 with a Nothing constraint. At this point we used to terminate the search because we already looked at %9. However, crucially, we looked at %9 only with an Int64 constraint, so we missed the fact that `nothing` was in fact a possible value for this phi. The result was a missing entry in the generated phi node: ``` 1 ─ goto #3 if not b 2 ─ %2 = Main.nothing::Core.Compiler.Const(nothing, false) 3 ┄ %3 = φ (#1 => false)::Bool │ %4 = φ (#2 => %2, #1 => _2)::Union{Nothing, Int64} │ %5 = (%4 isa Main.Int64)::Bool └── goto #5 if not %5 4 ─ %7 = π (%4, Int64) └── goto #6 5 ─ %9 = π (%4, Nothing) 6 ┄ %10 = φ (#4 => %3, #5 => %3)::Bool │ %11 = φ (#4 => %7, #5 => %9)::Union{Nothing, Int64} │ %12 = (%11 isa Main.Nothing)::Bool └── goto #8 if not %12 7 ─ goto #9 8 ─ nothing::Nothing 9 ┄ %16 = φ (#7 => %10, #8 => %10)::Bool └── return %16 ``` (note the missing #2 predecessor in phi node %3), which would result in an undefined value at runtime, though in this case LLVM would have taken advantage of that to just return 0: ``` define i8 @julia_f32579_16051(i64, i8) { top: ; @ REPL[1]:15 within `f32579' ret i8 0 } ``` Compare this now to the optimized IR with this patch: ``` 1 ─ goto #3 if not b 2 ─ %2 = Main.nothing::Core.Compiler.Const(nothing, false) 3 ┄ %3 = φ (#2 => true, #1 => false)::Bool │ %4 = φ (#2 => %2, #1 => _2)::Union{Nothing, Int64} │ %5 = (%4 isa Main.Int64)::Bool └── goto #5 if not %5 4 ─ %7 = π (%4, Int64) └── goto #6 5 ─ %9 = π (%4, Nothing) 6 ┄ %10 = φ (#4 => %3, #5 => %3)::Bool │ %11 = φ (#4 => %7, #5 => %9)::Union{Nothing, Int64} │ %12 = (%11 isa Main.Nothing)::Bool └── goto #8 if not %12 7 ─ goto #9 8 ─ nothing::Nothing 9 ┄ %16 = φ (#7 => %10, #8 => %10)::Bool └── return %16 ``` The %3 phi node has its missing entry and the generated LLVM code correctly returns `b`: ``` define i8 @julia_f32579_16112(i64, i8) { top: %2 = and i8 %1, 1 ; @ REPL[1]:15 within `f32579' ret i8 %2 } ``` (cherry picked from commit 0a12944)

KristofferC · 2019-09-23T06:36:33Z

Ref #32135 (comment) for another instance of this

The bug here is a bit subtle, but perhaps best illustrated with the included test case: ``` function f32579(x::Int64, b::Bool) if b x = nothing end if isa(x, Int64) y = x else y = x end if isa(y, Nothing) z = y else z = y end return z === nothing end ``` The code just after SSA conversion looks like: ``` 2 1 ─ goto #3 if not _3 3 2 ─ %2 = Main.nothing::Core.Compiler.Const(nothing, false) 5 3 ┄ %3 = φ (#2 => %2, #1 => _2)::Union{Nothing, Int64} │ %4 = (%3 isa Main.Int64)::Bool └── goto #5 if not %4 6 4 ─ %6 = π (%3, Int64) └── goto #6 8 5 ─ %8 = π (%3, Nothing) 10 6 ┄ %9 = φ (#4 => %6, #5 => %8)::Union{Nothing, Int64} │ %10 = (%9 isa Main.Nothing)::Bool └── goto #8 if not %10 11 7 ─ %12 = π (%9, Nothing) └── goto #9 13 8 ─ %14 = π (%9, Int64) 15 9 ┄ %15 = φ (#7 => %12, #8 => %14)::Union{Nothing, Int64} │ %16 = (%15 === Main.nothing)::Bool └── return %16 ``` Now, we have special code in SROA (despite it not really being an SROA transform) that looks at `===` and replaces it by a nest of phis of booleans. The reasoning for this transform is that it eliminates a use of a value where we only care about the type and not the content, thus making it more likely that the value will subsequently be eligible for SROA. In addition, while it goes along resolving which values feed into any particular phi, it accumulates and type conditions it encounters along the way. Thus in the example above, something like the following happens: - We look at %14, which πs to %9 with an Int64 constraint, so we only consider the #4 predecessor for %9 (due to the constraint), until we get to %3, where we again only consider the #1 predecessor, where we find the argument (of type Int64) and conclude the result is always false - Now we pop the next item of the stack from our original phi, look at %12, which πs to %9 with a Nothing constraint. At this point we used to terminate the search because we already looked at %9. However, crucially, we looked at %9 only with an Int64 constraint, so we missed the fact that `nothing` was in fact a possible value for this phi. The result was a missing entry in the generated phi node: ``` 1 ─ goto #3 if not b 2 ─ %2 = Main.nothing::Core.Compiler.Const(nothing, false) 3 ┄ %3 = φ (#1 => false)::Bool │ %4 = φ (#2 => %2, #1 => _2)::Union{Nothing, Int64} │ %5 = (%4 isa Main.Int64)::Bool └── goto #5 if not %5 4 ─ %7 = π (%4, Int64) └── goto #6 5 ─ %9 = π (%4, Nothing) 6 ┄ %10 = φ (#4 => %3, #5 => %3)::Bool │ %11 = φ (#4 => %7, #5 => %9)::Union{Nothing, Int64} │ %12 = (%11 isa Main.Nothing)::Bool └── goto #8 if not %12 7 ─ goto #9 8 ─ nothing::Nothing 9 ┄ %16 = φ (#7 => %10, #8 => %10)::Bool └── return %16 ``` (note the missing #2 predecessor in phi node %3), which would result in an undefined value at runtime, though in this case LLVM would have taken advantage of that to just return 0: ``` define i8 @julia_f32579_16051(i64, i8) { top: ; @ REPL[1]:15 within `f32579' ret i8 0 } ``` Compare this now to the optimized IR with this patch: ``` 1 ─ goto #3 if not b 2 ─ %2 = Main.nothing::Core.Compiler.Const(nothing, false) 3 ┄ %3 = φ (#2 => true, #1 => false)::Bool │ %4 = φ (#2 => %2, #1 => _2)::Union{Nothing, Int64} │ %5 = (%4 isa Main.Int64)::Bool └── goto #5 if not %5 4 ─ %7 = π (%4, Int64) └── goto #6 5 ─ %9 = π (%4, Nothing) 6 ┄ %10 = φ (#4 => %3, #5 => %3)::Bool │ %11 = φ (#4 => %7, #5 => %9)::Union{Nothing, Int64} │ %12 = (%11 isa Main.Nothing)::Bool └── goto #8 if not %12 7 ─ goto #9 8 ─ nothing::Nothing 9 ┄ %16 = φ (#7 => %10, #8 => %10)::Bool └── return %16 ``` The %3 phi node has its missing entry and the generated LLVM code correctly returns `b`: ``` define i8 @julia_f32579_16112(i64, i8) { top: %2 = and i8 %1, 1 ; @ REPL[1]:15 within `f32579' ret i8 %2 } ``` (cherry picked from commit 0a12944)

quinnj mentioned this issue Jul 15, 2019

Illegal instruction error when handling Union types #32582

Closed

Keno added compiler:optimizer Optimization passes (mostly in base/compiler/ssair/) bug Indicates an unexpected problem or unintended behavior labels Jul 15, 2019

tkf changed the title ~~=== nothing is not == nothing (only in Travis)~~ === nothing is not == nothing with --code-coverage=user Jul 15, 2019

Keno closed this as completed in 0a12944 Jul 17, 2019

KristofferC mentioned this issue Jul 20, 2019

Backports for 1.2.0 release #32592

Merged

14 tasks

KristofferC mentioned this issue Aug 26, 2019

WIP: Backports for 1.0.5 #33075

Merged

55 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`=== nothing` is not `== nothing` with `--code-coverage=user` #32579

`=== nothing` is not `== nothing` with `--code-coverage=user` #32579

tkf commented Jul 15, 2019

tkf commented Jul 15, 2019

StefanKarpinski commented Jul 15, 2019

vtjnash commented Jul 15, 2019

Keno commented Jul 15, 2019

Keno commented Jul 15, 2019

Keno commented Jul 15, 2019

tkf commented Jul 15, 2019 •

edited

Loading

tkoolen commented Jul 16, 2019

KristofferC commented Sep 23, 2019

=== nothing is not == nothing with --code-coverage=user #32579

=== nothing is not == nothing with --code-coverage=user #32579

Comments

tkf commented Jul 15, 2019

tkf commented Jul 15, 2019

StefanKarpinski commented Jul 15, 2019

vtjnash commented Jul 15, 2019

Keno commented Jul 15, 2019

Keno commented Jul 15, 2019

Keno commented Jul 15, 2019

tkf commented Jul 15, 2019 • edited Loading

tkoolen commented Jul 16, 2019

KristofferC commented Sep 23, 2019

`=== nothing` is not `== nothing` with `--code-coverage=user` #32579

`=== nothing` is not `== nothing` with `--code-coverage=user` #32579

tkf commented Jul 15, 2019 •

edited

Loading