[API Proposal]: BFloat16

[iamcarbon]

Background and motivation

The bfloat16 type provides the same number range as the 32-bit IEEE 754 single-precision floating point type, but with a reduced precision (24 bits -> 8 bits). This is useful for machine learning to improve memory utilization, and can be used to accelerate AI workloads via AVC-512 BP / and ARMv8.6-A instructions.

Adding this type would allow us to implement these new instructions sets, and provide a common base type for various machine learning libraries.

API Proposal

namespace System.Numerics
{
    public readonly struct BFloat16
      : IComparable,
        IComparable<BFloat16>,
        IEquatable<BFloat16>
    {
        public static BFloat16 Epsilon { get; }
        public static BFloat16 MinValue { get; }
        public static BFloat16 MaxValue { get; }
                
        // Casting
        public static explicit operator BFloat16(float value);
        public static explicit operator BFloat16(double value);
        public static explicit operator float(BFloat16 value);
        public static explicit operator double(BFloat16 value);

        // Comparison
        public int CompareTo(object value);
        public int CompareTo(BFloat16 value);
        public static bool operator ==(BFloat16 left, BFloat16 right);
        public static bool operator !=(BFloat16 left, BFloat16 right);
        public static bool operator <(BFloat16 left, BFloat16 right);
        public static bool operator >(BFloat16 left, BFloat16 right);
        public static bool operator <=(BFloat16 left, BFloat16 right);
        public static bool operator >=(BFloat16 left, BFloat16 right);

        // Equality
        public bool Equals(BFloat16 obj);
        public override bool Equals(object? obj);
        public override int GetHashCode();
        
        // ToString override
        public override string ToString();
    }
}

API Usage

BFloat16 bf16 = (BFloat16)1.0f; 

Alternative Designs

No response

Risks

No response

Tagging subscribers to this area: @dotnet/area-system-numerics
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Background and motivation

The bfloat16 type provides the same number range as the 32-bit IEEE 754 single-precision floating point type, but with a reduced precision (24 bits -> 8 bits). This is useful for machine learning to improve memory utilization, and can be used to accelerate AI workloads via AVC-512 BP / and ARMv8.6-A instructions.

Adding this type would allow us to implement these new instructions sets, and provide a common base type for various machine learning libraries.

API Proposal

namespace System
{
    public readonly struct BFloat16 : IComparable, IFormattable, IComparable<BFloat16>, IEquatable<BFloat16>, IConvertible, ISpanFormattable, IUtf8SpanFormattable
    {
        public static readonly BFloat16 MinValue;
        public static readonly BFloat16 MaxValue;
                
        public static bool IsNegative(BFloat16 h);
        public static BFloat16 Parse(string s);
        public static BFloat16 Parse(string s, NumberStyles style);
        public static BFloat16 Parse(string s, NumberStyles style, IFormatProvider provider);
        public static BFloat16 Parse(string s, IFormatProvider provider);
        public static BFloat16 Parse(ReadOnlySpan<char> s);
        public static BFloat16 Parse(ReadOnlySpan<char> s, NumberStyles style);
        public static BFloat16 Parse(ReadOnlySpan<char> s, IFormatProvider provider);
        public static BFloat16 Parse(ReadOnlySpan<char> s, NumberStyles style, IFormatProvider provider);
        public bool TryFormat(Span<char> destination, out int charsWritten, ReadOnlySpan<char> format, IFormatProvider provider);
        public static bool TryParse(string s, out BFloat16 result);
        public static bool TryParse(string s, NumberStyles style, IFormatProvider provider, out BFloat16 result);
        public static bool TryParse(ReadOnlySpan<char> s, out BFloat16 result);
        public static bool TryParse(ReadOnlySpan<char> s, NumberStyles style, IFormatProvider provider, out BFloat16 result);
        public int CompareTo(object value);
        public int CompareTo(BFloat16 value);
        public bool Equals(BFloat16 obj);
        public override bool Equals(object obj);
        public override int GetHashCode();
        public TypeCode GetTypeCode();
        public string ToString(IFormatProvider provider);
        public string ToString(string format);
        public string ToString(string format, IFormatProvider provider);
        public override string ToString();
        public static explicit operator BFloat16(float value);
        public static explicit operator float(BFloat16 value);
        public static bool operator ==(BFloat16 left, BFloat16 right);
        public static bool operator !=(BFloat16 left, BFloat16 right);
        public static bool operator <(BFloat16 left, BFloat16 right);
        public static bool operator >(BFloat16 left, BFloat16 right);
        public static bool operator <=(BFloat16 left, BFloat16 right);
        public static bool operator >=(BFloat16 left, BFloat16 right);
    }
}

API Usage

BFloat16 bf16 = 1.0f; 

Alternative Designs

No response

Risks

No response

Author:	iamcarbon
Assignees:	-
Labels:	api-suggestion, area-System.Numerics, untriaged
Milestone:	-

[MichalPetryka]

This should probably expose the whole API surface that Half has, including all the operators like addition and such even if they're not accelerated by most hardware.

[iamcarbon]

@MichalPetryka Updated to implement the IFloatingPoint interface, along with its operators. These can likely also forward to MathF / float, like Half by default.

[MichalPetryka]

Updated to implement the IFloatingPoint interface, along with its operators

You've missed IMinMaxValue<BFloat16>, which Half has.

[iamcarbon]

The proposal has been updated to include the IMinMaxValue interface. Note: the API is limited to public members. There are various INumber and IFloatingPoint members that are not listed, but will need explicit implementations to participate in the generic math system. @MichalPetryka Let me know if you spot any other missing public members.

[MichalPetryka]

The proposal has been updated to include the IMinMaxValue interface. Note: the API is limited to public members. There are various INumber and IFloatingPoint members that are not listed, but will need explicit implementations to participate in the generic math system. @MichalPetryka Let me know if you spot any other missing public members.

Unary Negation Operators seems to have the unary plus.

[iamcarbon]

Unary Negation Operators seems to have the unary plus.

Fixed.

[huoyaoyuan]

This should probably expose the whole API surface that Half has, including all the operators like addition and such even if they're not accelerated by most hardware.

I don't think mathematic functions should be implemented. They are likely not supported by hardware, nor required by any specification.
The first version of Half in .NET 5 is only a transport type, with no IEEE754 function implemented.

I'd expect it to implement only conversion operators, and basic arithmetic operators only:

// comparable, equatable, parsing and formatting omitted
IMinMaxValue
IBinaryNumber
IFloatingPoint

[iamcarbon]

I believe there's still value implementing the Trigonometric & Hyperbolic functions as this type maintains the full Float32 range.

Converting a BFloat16 to a Single can also be done in a few shift operations. This operation is much slower on the Half type.

public unsafe static float BFloat16ToSingle(ushort bfloat16)
{
    int f32Value  = 
        (bfloat16 & 0x8000) << 16 |                      // sign bit
        ((bfloat16 & 0x7FFF) + 0x1C000) << 13; // exponent and mantissa

    return *(float*)&f32Value;
}

ARM also provides the accelerated BFCVT function to convert a Single back to a Float16.

However, I agree they are non-essential.

[MichalPetryka]

I don't think mathematic functions should be implemented. They are likely not supported by hardware, nor required by any specification.
The first version of Half in .NET 5 is only a transport type, with no IEEE754 function implemented.

I think it's worth noting that proposed API surface isn't necessarily the one that's initially implemented as it was noted in #81376.
As such, I think that unless the decision would be to never add the full set of operations (which seems unlikely since hardware is already starting to expose them), API review should see the final surface during review, even if its implementation would be partial initially.

Let me know if you spot any other missing public members.

Diffing with Half seems to still show some missing members.

[colejohnson66]

public unsafe static float BFloat16ToSingle(ushort bfloat16)
{
    int f32Value  = 
        (bfloat16 & 0x8000) << 16 |                      // sign bit
        ((bfloat16 & 0x7FFF) + 0x1C000) << 13; // exponent and mantissa

    return *(float*)&f32Value;
}

This seems like needlessly complicated to read, and generates worse codegen than is needed. A bfloat16 is just a truncated binary32:

public static float BFloat16ToBinary32(ushort value)
{
    uint temp = (uint)value << 16;
    return Unsafe.As<uint, float>(ref temp);
}

[tannergooding]

API review should see the final surface during review

This isn't important to API review. The potential for operators to be added later is generally not a major consideration in the exposure of a type. We almost never know the "full" surface area, and while it might be relevant to consider whether additional APIs are planned, they really only limit the ability to cleanly implement/expose the initial surface.

---

This type is not really a core/common type and isn't even strictly "well spec'd" in the same way the IEEE 754 types are. It likely should exist in the System.Numerics namespace (much as the new Decimal32/64/128 types will be).

It should initially only cover itself as a minimal interchange type with the relevant conversion APIs. That is going to be the 99% use case and is the only case that will be hardware accelerated for the near future. I'm fine with separately considering the expansion of this to support the full set of IBinaryFloatingPointIeee754<T> members, but that should be split out and separate from the mainline consideration. Such members would only be convenience APIs for upcast to float, do the operation, downcast to bfloat after all and in many cases would be the less efficient way to operate on the data (typical usage in AI/ML/GPU is to upcast a vector's worth of these values, operate on them as float end to end, and then downcast when storing back to memory/disk).

A bfloat16 is just a truncated binary32

Notably this is not universally true. It was initially introduced using truncation, but there are a number of different hardware implementations nowadays and some use ties to even (IEEE 754 default, which Google TPU uses) or round to odd (ARM), etc.

We should likely default to truncation, but its possible we need additional APIs to support other rounding modes.