Zero is a real number (Flux.Nil)

[mcabbott]

To see what CI thinks, this gives the special nil used for outputsize supertype Real.

The reason to do so is that really all the activation functions should be defined for ::Real, not ::Any in the name of sanity, and not ::Number to exclude complex numbers, since they often test x>0 etc.

Maybe this has consequences I don't see yet. They won't work in a function which with signature ::Complex, but nor did they before -- and most complex-valued things accept real numbers too.

[darsnack]

Right now, if something accepted only Complex and not Real, then Nil would fail. But it seems like we would want to pass it in, and instead define the things that need to be defined for Complex numbers (e.g. conj etc.).

Nil is more like a Missing that subtypes Number. "Zero" is a complex number too 😉.

[mcabbott]

I'm not sure I follow your thinking, but let me explain mine, with apologies if this is too basic.

For functions, normally want the widest signature for which they will give correct results. If you call C then perhaps that might be f(::Float64), if you're pretty generic but assume that multiplication is commutative, then ideally you want f(::Union{Real, Complex}), JuliaDiff/ChainRules.jl#540. If you only accept positive numbers, or if you divide up the real line into different implementations, then you do exclude complex numbers (and quaternions) and f(::Real) seems like a sensible way to communicate this.

Of course Flux & friends have many f(::Any) which actually only accept real numbers, but (IMO) this is part of why error messages are so confusing, as something then goes wrong 10 levels deeper.

For types, I think instead you want the narrowest signature which doesn't cause problems? Clearly 0 == false but you can't subtype Bool. You can subtype Integer but it's plausible that'll run into problems -- some functions dispatch on ::Integer to do something categorical, and we want this not to take such paths.. If we make nil isa Real then it will still be accepted by all f(::Number), but also by f(::Union{Real, Complex}).

[darsnack]

Looking at my comment, I don't know what I meant either. I think my logic was backwards in a bit of a drive-by comment. I think this is a good PR to merge.