proposal: spec: add support for int128 and uint128

[mei-rune]

No description provided.

[ianlancetaylor]

Can you provide a real-life use case?

[twoleds]

it's good for UUID, IPv6, hashing (MD5) etc, we can store IPv6 into uint128 instead byte slice and do some mathematics with subnetworks, checking range of IP addresses

[minux]

These use cases are not strong enough to justify adding 128-bit types,
which is a big task to emulate it on all targets.

1. MD5 is not secure anymore, so there is little benefit adding types to
   store its result.
2. How often do you need to manipulate a UUID as a number rather than a
   byte slice (or a string)?
3. The other use cases can be done with math/big just as easy.

Also note that GCC doesn't support __int128 on 32-bit targets and Go do
want consistent language features across all supported architectures.

[twoleds]

I agree with you there aren't a lot of benefits for int128/uint128, maybe a little better performance for comparing and hashing in maps when we use uint128 for storing UUID/IPv6 because for byte slices or string we need do some loops and extra memory but it isn't important I think

[mei-rune]

I stat all interface flux of a device in one day.

[the80srobot]

In addition to crypto, UUID and IPv6, int128 would be enormously helpful for volatile memory analysis, by giving you a safe uintptr diff type.

[iMartyn]

It also just makes code that much more readable if you have to deal with large IDs e.g. those you get back from google directory API amongst others (effectively they're uuids encoded as uint128).
Obviously you can use math/big but it makes the code much harder to reason about because you have to parse the code mentally first, distracting you from reading the code.

[ericlagergren]

Adding a data point: ran into a situation with a current project where I need to compute (x * y) % m where x*y can possibly overflow and require a 128-bit integer. Doing the modulus by hand for the high and low halves is needlessly complicated.

[jfesler]

Another +1 for both IPv6 and UUID cases.

[ianlancetaylor]

The examples of UUID and IPv6 are not convincing to me. Those types can be done as a struct just as easily.

It's not clear that this is worth doing if processors do not have hardware support for the type; are there processors with 128-bit integer multiply and divide instructions?

See also #19623.

[ericlagergren]

@ianlancetaylor I do not think so. GCC seems to use the obvious 6 instructions for mul, 4 for add and sub, and a more involved routine for quo. I'm not how anybody could emulate mul, add, or sub that precisely (in Go) without assembly, but that prohibits inlining and adds function call overhead.

[ianlancetaylor]

The fact that the current tools can't yet inline asm code is not in itself an argument for changing the language. We would additionally need to see a significant need for efficient int128 arithmetic.

If there were hardware support, that in itself would suggest a need, since presumably the processor manufacturers would only add such instructions if people wanted them.

[ericlagergren]

If there were hardware support, that in itself would suggest a need

A need that—presumably—compilers couldn't meet by adding their own 128-bit types, which they have. I mean, for all but division it's a couple extra instructions. For most cases that's been sufficient.

I confess I'm not an expert on CPU characteristics, but my understanding is much of the driving force behind adding larger sizes was the ability to address more memory. That makes me think general 128-bit support is rather unlikely.

Yet major compilers have added support (GCC, Clang, ICC, ...) for C and C++. Rust has them because of LLVM. Julia has them as well.

Other languages and compilers having support isn't sufficient reason to make a language change, sure. But it's evidence there exists a need other than simply UUIDs.

Their domain seems to lie in cryptography and arbitrary-precision calculations, for now.

[FlorianUekermann]

Additional usecases are timestamps, cryptographic nonces and database keys.

Examples like database keys, nonces and UUID represent a pretty large collection of applications where keys/handles can't ever be reused or number ranges can't overlap.

[ianlancetaylor]

@FlorianUekermann People keep saying UUID, but I see no reason that a UUID could not be implemented using a struct. It's not like people use arithmetic on a UUID once it has been created. The only reason to add int128 to the language is if people are going to use arithmetic on values of that type.

[FlorianUekermann]

It's not like people use arithmetic on a UUID once it has been created

They do. UUIDs don't have to be random. Sequential UUIDs are common in databases for example. Combine sequential UUIDs with some range partitioning and you'll wish for integer ops in practice.

Still, timestamps seem like the most obvious example to me, where 64bit is not sufficient and the full range of arithmetic operations is obviously meaningful. Had it been available, I would expect that the time package contained some examples.

How big of an undertaking is the implementation of div? The rest seems rather straightforward.

[ericlagergren]

How big of an undertaking is the implementation of div?

The code for naïve 128-bit division exists in the stdlib already (math/big). The PowerPC Compiler Writer’s Guide has a 32-bit implementation of 64-bit division (https://cr.yp.to/2005-590/powerpc-cwg.pdf, page 82) that can be translated upwards.

[josharian]

Use case: [u]int128 can be used to check for overflow of [u]int64 operations in a natural way. Yes, this could make you want int256, but since int64 is the word size of many machines, this particular overflow matters a lot. See e.g. #21588. Other obvious options to address this use case are math/bits and
#19623.

Somewhat related use case: #21835 (comment).

[lfaoro]

I don't understand why it's such a big deal to just do it, you have 128 comments on this issue, it's been open since 2014, other mainstream languages already complied w/ the users need.

What else do you need to consider support for int128? We don't want to use big.Int anymore.

[gophun]

@lfaoro It's been explained why it would be a big deal: #9455 (comment)

[phuclv90]

We have a path where we represent the size of an IP range, and IPv6 ranges can be larger than 64 bits. uint128 would have fit the bill.

@thockin an IPv6 address isn't a number and you don't do arithmetic on it. Only bitwise operations are needed. Therefore that's not a good reason to add the type. There are already lots of comments regarding IPv6 above

[thockin]

Regretfully, we find ourselves in a place where the number of IPs in a given CIDR is used, and changing that is harder that just useing a bignum.

…

On Thu, Mar 16, 2023, 6:58 AM Lưu Vĩnh Phúc ***@***.***> wrote: We have a path where we represent the size of an IP range, and IPv6 ranges can be larger than 64 bits. uint128 would have fit the bill. @thockin <https://github.com/thockin> an IPv6 address isn't a number and you don't do arithmetic on it. Only bitwise operations are needed. Therefore that's not a good reason to add the type. There are already lots of comments regarding IPv6 above — Reply to this email directly, view it on GitHub <#9455 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABKWAVHPJJ64OQBK77NSSJTW4MMAHANCNFSM4AZR5JSA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[chiro-hiro]

If anyone try to build a WebAssembly runtime with Go, int128 would help.

[JesseCoretta]

Hi!

I have read the comments on this thread, and for the most part I see both points of view. But one thing occurs to me, and while it is somewhat related to UUIDs, unfortunately this can of worms is a little deeper when you consider ASN.1 Object Identifiers.

Per [X.667], there is precedent for UUIDs to be interpreted as actual integers in certain Object Identifiers. One that exists in the wild, just for those who are curious, is:

// {joint-iso-itu-t(2) uuid(25) ans(987895962269883002155146617097157934)}
2.25.987895962269883002155146617097157934

Now, many of you can look at that leaf arc and see quite clearly that it will overflow not just int64, but uint64 as well. By quite a bit, actually:

• 340,282,366,920,938,463,463,374,607,431,768,211,455 for uint128
• 18,446,744,073,709,551,615 for uint64
• 9,223,372,036,854,775,807 for int64

Now, I know some of you might be biting your tongues, thinking "oh no ... if she's right, that would mean the encoding/asn1-provided ObjectIdentifier type is basically invalid too, right?".

Well, in a way it always been invalid (at least in my point of view). No offense to the Go team, its just that because int is used as the slice type, and int allows negative values -- something that should never, ever appear in an OID -- that's something else I need to account for manually in my code. And this isn't even taking the overflow issue into account.

That said, I'm not here to bash the asn1 package, nor am I directly requesting any changes to it. I know how the Go team feels about this package, but quite frankly I love ASN.1 and I am just grateful it is supported via my favorite language.

But I am voting in favor of uint128 support (and, I suppose, int128 by necessity), just from my obscure PoV. I can easily make my own OID type (e.g.: []uint128) and do what I need to do, never worrying about overflows involving legal UUID-based OIDs. So I guess, from where I stand, its sort of a compromise.

One counter-argument that I can foresee:

"OIDs have no set limit on the magnitude of individual arcs, so conceivably this would be an issue when someone registers an OID that has some ungodly long number that would overflow the largest supported uintXXX-type in the universe."

This is true. One can only do so much. I'm also certain that OID RAs (Registration Authorities) look for such questionable moves and perhaps mitigate them. But you're not wrong. And yet, [X.667] exists for a reason, so I would cite that as motivation for supporting uint128. At least those registrations are legitimate, if a little odd ...

Thank you for listening

Jesse 💜❤️

[Bjohnson131]

Is the team still looking for "strong use cases" for uint128?

[josharian]

@Bjohnson131 more strong use cases are welcome. Please "load hidden" on the comments on this issue first, though, and take a pass through--it's easy to miss things that have already been discussed above.

I suspect that the most helpful thing at this moment might be to write a complete doc, paying particular attention to what packages might be impacted (search for the text "strconv" and "generics" in comments after loading all hidden), and what the answer there is. But that's a lot of work, with no guarantee of impact.

(It'd also be helpful to update the original post with a list of use cases proposed in the comments.)

[phuclv90]

FWIW .NET added support for 128-bit int not so long ago and you may want to check out the discussions

• Add an (U)Int128 type dotnet/runtime#20640
• Add support for Int128 and UInt128 data types dotnet/runtime#67151

[josharian]

@phuclv90 thanks! That conversation is strikingly similar to this one, including (perhaps) the denouement of using generics once they were available to solve the strconv problem.

[Bjohnson131]

I think that there's 2 things that people have not mentioned. (u)int128s allow us to write cleaner, more error-free software. period.

There's an air among communities that this isn't a valid reason, as people should write good code. Often though, (and we all know we're all guilty of this) our code isn't good or completely readable. (u)int128s would help everybody in this regaurd every time they're used to avoid messy code as well as code with bugs.

Intrinsically, this value-add should be obvious. more people writing more better bug-free code is good for the entire ecosystem.

[c-robinson]

It looks like the Go source indicates that it might be useful to have at least a uint128. There are multiple references to uint128 peppered around the crypto/aes package, as well as a type in crypto/internal/edwards25519 for uses that are waaaaay over my head:

// uint128 holds a 128-bit number as two 64-bit limbs, for use with the
// bits.Mul64 and bits.Add64 intrinsics.
type uint128 struct {
	lo, hi uint64
}

and then net/netip defines the same type with the same name, but adds a bunch of methods... so much so that the netip implementation seems like it would be entirely usable as a replacement for, say, a uint128-sized value stored in big.Int.

@minux's initial counter-argument to this proposal hinged on the statement that

The other use cases can be done with math/big just as easy.

But that would seem to be in conflict with the netip approach.

[Eisenwave]

I have written a proposal P3140: std::int_least128_t for C++.

The Motivation in that proposal is largely language-agnostic, so it might be helpful in this discussion.

[andreyvit]

+1. My use cases: arithmetics — 128-bit fixed-point numerics (high-precision monetary values), bitwise ops — hashing, identifiers, larger bitmasks. Like others have noted, code simplification for bitwise operations is a noticeable benefit.

However, if the team wants an alternative proposal, we could allow arithmetics and bitwise ops on [n]uint64 types. This would take care of uint128, uint256 and similar, without extending the list of primary types. Then extending fmt, strconv, json and other packages would be an optional effort that doesn't have to happen in lockstep with compiler changes (or maybe doesn't need to happen at all).

[sylr]

+1. My use cases: arithmetics — 128-bit fixed-point numerics (high-precision monetary values)

Same here, an int128 based fixed-point numeric could allow to be precise enough for cryptocurrencies such as bitcoin and be "large" enough to be also used for market capitalizations with fiat currencies.

[andreyvit]

I went through my codebase, and I'm more in favor of [2]uint64 proposal now (i.e. allow treating [n]uint64 as integers for arithmetics, bitwise ops, comparisons, maybe even literal initializers) than simply an uint128 type. This would be assigning new semantics to something that is already 99% used for bignums, won't break any existing code, and won't require any library changes (we can add extra strconv functions for these, but, most importantly perhaps, no sweeping reflection changes across all serialization libs).

People can add MarshalText etc on top of these as appropriate for their domains (which will probably produce hex, base64 or even dashed UUID formatting).

The reasoning for [n]uint64 over int128 etc is:

• in at least two cases, I really want an uint256, not uint128
• formatting and parsing is domain-specific (and materially differs in different places in my code)
• I bet at some point in the future people will start asking for uint512, uint1024, etc
• really the annoying part making code hard to read is operators; if these are solved by the compiler, everything else can be added trivially

Should I open a separate ticket for discussion, or is this a non-starter @ianlancetaylor @robpike?

[gaby]

Why isnt https://github.com/golang/go/blob/master/src/net/netip/uint128.go exported ? It would solve part of this issue.

[Bjohnson131]

I'm more in favor of [2]uint64 proposal now

Allow me to play the consistency questioner.

1. Will a float128 then be called a [2]float64?
2. How will you express generic functions that accept []uint64s as well as []uint128s?

Let me say this now, if [2]uint64 worked for my use-case, I would not be here. What you're proposing is something which is already taught as a work-around for the lack of u128 support, and it is not a solution to many use-cases named here.

[andreyvit]

@Bjohnson131 Could you please clarify about the use cases not covered by [n]uint64? I feel like I've read the entire thread, but cannot recall any that fit, would love to consider those.

Let's make sure we're talking about the same thing. I'm proposing that the following code should be legal (for any operator in place of +):

var a, b, c [2]uint64
a = b + c

Re: generics overall, I'd imagine initially there'd be no way to write generic code that works for [n]uint64 with any n, but that's not any worse than having an uint128 type. You can probably write generic code over something like ~[2]uint64 | ~[4]uint64 if needed.

But perhaps I misunderstand your question about generics. If we imagine that type uint128 = [2]uint64, then accepting either []uint64 or []uint128 looks like func xxx[T uint64 | [2]uint64](v []T).

Re: float128, I don't imagine this being applicable for floats. Long uints are truly often represented as [n]uint64, but that's not true about floats at all.

[dylan1951]

Why does it take 10+ years to add 128 bit ints despite enormous demand for it?

[DmitriyMV]

@dylan1951 please refer to #9455 (comment)

[dylan1951]

@dylan1951 please refer to #9455 (comment)

The comment says it's not a zero change thing to do, and says it isn't arguing against the change. What's your point?