Move BigFloat to stdlib

[simonbyrne]

BigFloat could be moved to a stdlib package (BigFloats? MPFR?), though there are a few places it is coupled to Base:

• widen(::Float64) and big(::AbstractFloat)
  • Should we just leave these undefined, and type-pirate them in the package?
• array hashing (which calls widen)
  • Should be doable via TwicePrecision tricks I misunderstood what the widen was for.
• @irrational macro, which defines a BigFloat method
• Float64(::Irrational,::RoundingMode) (which is used in floating point inequalities)
• Rational{T}(::Irrational) and rationalize(::T,::Irrational)

c.f. https://discourse.julialang.org/t/moving-bigfloat-to-stdlib/8545

[ViralBShah]

We should definitely move BigFloat to stdlib.

[simonbyrne]

One possible solution to the Irrational ones is that we manually define the necessary methods for the built-in ones (pi, e, etc.), and then move the @irrational macro itself to the new package?

[JeffBezanson]

Yes, that sounds ok. The type piracy here also does not bother me. Initially you have no way to widen a Float64, but if you load the BigFloats package then you do. Makes sense to me.

[nalimilan]

As I noted on Discourse, I'm afraid we can't define widen(::Float64) in the stdlib, or you wouldn't be able to hash heterogeneous arrays containing Float64 values without loading the BigFloats stdlib module. The reasons for this are complex, and so far we haven't found any alternative (#16401). However, we could move all user-facing APIs to the stdlib and only use BigFloat internally.

[StefanKarpinski]

I don't see a ton of advantage to this. We basically need to have BigInt in Base since we have syntax for it (long integer literals). We only export BigFloat itself and setprecision from the MPFR module, so it's not like it's a big contributor to Base API bloat. We already have the big function which we would like to have continue to work on floats without needing to load BigFloats so we need to have the BigFloat functionality loaded in the system image no matter what. So it seems like the only real change would be that if people want to change the default BigFloat precision they would have to write using BigFloats before doing setprecision. However, the setprecision function seems to aim to be generic – although it also only seems to currently support changing the precision of BigFloat and I'm not sure what else it could even change the precision of.

[pkofod]

nd I'm not sure what else it could even change the precision of.

other arbitrary precision numbers? (I mean, packages might extend the function with a method for arbfloats, ...)

[StefanKarpinski]

Right, so we want a shared extension point anyway, which means to me that setprecision doesn't even really belong to BigFloats – it just happens to be defined there because BigFloat is the only Base type that it applies to. So really the only thing that would be exported from a BigFloats standard library would be the BigFloat type itself.

[simonbyrne]

The main advantage as I see it is that the MPFR dependency could eventually be made optional.

[JeffBezanson]

Agreed, I see this as a step towards making MPFR fully optional.

[ViralBShah]

Is this still something we can accomplish? Assuming the list that @simonbyrne posted is roughly accurate, it seems quite doable.

[ViralBShah]

A good first step is to remove from Base and make an stdlib. Can we do both BigFloat and BigInt as stdlibs? That paves the way for eventual separation.

[JeffreySarnoff]

BigInt and BigFloat share their first three letters, and that is their most alike quality.
stdlib BigNums.jl could export both. Where one goes, in a tree-like sense, the other should follow.

[KristofferC]

The first step is to figure out how to deal with the fact that e.g. big integers are a part of the syntax of the language:

julia> Meta.parse("10000000000000000000") |> dump
Expr
  head: Symbol macrocall
  args: Array{Any}((3,))
    1: GlobalRef
      mod: Module Core
      name: Symbol @int128_str
    2: Nothing nothing
    3: String "10000000000000000000"

What should this mean when BigInt is an stdlib and might not be available? Do you need to add a dependency to BigInts if you write big number literals in your code?

[ViralBShah]

@KristofferC Aren't stdlibs always available?

[KristofferC]

Aren't stdlibs always available?

Not really, for example, you can filter them out with PackageCompiler if the project file shows that you do not use them. You are also required to declare them as a dependency if you intend to use them to be able to load them in packages. It seems weird that the presence of a standard library can influence what is correct syntax in the language.

[ViralBShah]

So maybe close this issue then?

[KristofferC]

I was only talking about BigInt here. BigFloat should be movable I think.

[ViralBShah]

Wouldn't it be a bit odd though to have one as a stdlib and not the other, since then you have to know which one needs to be declared as a depencency and which one is not?

[KristofferC]

One difference is that you can get a dependency on BigInt by just writing a big integer literal but to get a dependency on BigFloat you need to use something explicit e.g. the big"" string macro.

[simonbyrne]

One difference is that you can get a dependency on BigInt by just writing a big integer literal

It would be a breaking change, but it could parse as a big"" string macro?

[PallHaraldsson]

I don't see a ton of advantage to this. We basically need to have BigInt in Base since we have syntax for it (long integer literals). We only export BigFloat itself and setprecision from the MPFR module, so it's not like it's a big contributor to Base API bloat. We already have the big function which we would like to have continue to work on floats

I don't want big to work on floats. Before we discuss BigInt which is a separate issue, I would simply like BigFloats gone to an external MPFR.jl library but without it being a stdlib.

What are the pros and cons of it as a stdlib, or included at all? I want to suggest users use ArbNumerics.jl OR MPFR.jl but not usually at the same time so I could do:

const BigFloat = ArbNumerics.ArbFloat

i.e. add that to that package, and it would work as a faster drop-in replacement, for most users.

[PallHaraldsson]

widen(::Float64) and big(::AbstractFloat)
Should we just leave these undefined, and type-pirate them in the package?

undefined, or I guess you rather mean:

widen(Float64) = error("Install ArbNumerics.jl (or MPFR.jl) for this to work)"

@simonbyrne

array hashing (which calls widen)

couldn't that, and all other cases you mention be handled by that erroring definition and/or defining in a package, pirating?

E.g.

@irrational macro, which defines a BigFloat method

it's not clear it's best to get BigFloat out (with its default 256 bits), the next step for widen should be to 128 bits I recall you get for ArbNumerics.ArbFloat, its default is settable, to e.g. 256 bits, hypothetically you could widen to higher and higher or to MPFR.

The first step is to figure out how to deal with the fact that e.g. big integers are a part of the syntax of the language:

julia> Meta.parse("10000000000000000000") |> dump
[..]
      name: Symbol @int128_str

It seems you made an invalid point since you get Int128 up to:

Meta.parse("170141183460469231731687303715884105728") # On higher than the typemax, there's also UInt128.

I think we could just error in the parser (for a slight technically breaking change, who depends on that large integer literals, in the parser? Note if you add .0 the this turns into a Float64, not BigFloat) and it wouldn't be awfully bad. And support for BigInt can be dynamically added to the parser(?), Julia doesn't need BigInt for itself (in the parser). It's possibly easier to add to the parser at runtime with JuliaSyntax.jl now used, if that's actually advised...

@JeffreySarnoff

BigInt and BigFloat share their first three letters, and that is their most alike quality.
stdlib BigNums.jl could export both.

The reason I don't like this much is that the users that want BigInt may not need or want BigFloat and vice versa. Eventually we should have a BigInt (external or stdlib) library (or name it GMP.jl?), in addition to for BigFloats. It only seems to make sense to add a BigNums.jl (stdlib) to include both, in case you want to actually use both, and for for sure nor ArbNumerics.jl inseaed of MPFR.jl for the BigFloat implementation.

We could have using Julia1 to restore full compatibility.

julia> widen(1.0f0)  # ok
1.0

julia> widen(1.0)  # I know intentional to not make the type obvious, still arguable bad (for new users). By requiring a package it's opt-in and plausibly should be known:
1.0

julia> typeof(ans)
BigFloat

[nalimilan]

FWIW array hashing no longer uses widen since #26022.