-
Notifications
You must be signed in to change notification settings - Fork 17.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
errors: add support for wrapping multiple errors #53435
Comments
How does walking up the tree work in |
Depth-first. Unwrap to an |
This would also subsume #50819 and I imagine any other similar proposals. Maybe it should wait for a generic iteration standard, but should there be some errors.Each function that goes through each link in the chain? My suspicion is there’s not much to do besides As-ing and Is-ing, but it does seem like an absence, since there will be some internal function that works like that, which could potentially be exported. |
We should probably wait and see what happens with generic iteration, however. |
Does If not, and it returns a copy, then it would allocate, which also seems unfortunate (but probably not a dealbreaker). |
Unspecified, but the contract for it says that the caller must not modify the slice, so it may. Perhaps |
This strikes me as a significant step forward in figuring out the multi-error question. Thank you! The only part of this that doesn't seem entirely spot-on is What is the use case for Figuring Walking an arbitrary error tree using (The details about On the topic of slice copies, a minor question. Does |
I find that the main reason I unwrap my own multierrors is so that I can report them separately to my logging service or display them as a list in a CLI. The original xerrors formatting proposal was dropped for being too complicated but I think that an easy to use error walking mechanism might make it easy to let users figure out how they want to display and format all the suberrors on their own. |
This proposal has been added to the active column of the proposals project |
If I hadn't read the comments, I might have thought that after the "Join" error, I could use "Split" to parse it out. "errors.Split" does the same thing as "errors.Unwrap", but deals with multiple errors. So instead of "Split" which is the opposite of "Join", it should be "UnwrapMultiple" or "Walk" which is more appropriate. |
Since there is no reaction to multiple error conditions, the naming should also be differentiated, such as ”UnwrapMultiple() []error “. |
We need to provide some reasonably convenient way to get at the underlying errors of a multierr. Perhaps
Perhaps there's a case for such an API, but I think it's distinct from this proposal. Accessing the component errors of a multierr is different from a full tree walk.
Yes. |
I'm 100% on
If those were the only things accepted I'd be happy and it would be more than enough to allow external multierrors to interoperate with each other and std. I do think there 100% does need to be a tree walking API—but it's too early to decide what that looks like. Once the protocol above is enshrined in I'm not 100% sold on the name for I'm not really sold on |
I'm not so sure that we do. For uses (testing?) in which you really specifically want to unwrap from an error that has an But I suspect that almost nobody wants specifically to unwrap a multi-error exactly one layer deep. Rather they want to ask the general question "what errors are in this tree?". So I think that's the right form of API to add. And I think it'll get created internally anyway to implement Is and As. Maybe this proposal should simply omit |
The fact that I'm not certain how much of a use case there is for a tree walk that doesn't preserve the structure of the tree. It would be nice to see real-world examples of this in practice. |
I grepping in my own dependencies and grabbed all the uses. I ignored the standard library, static analyzers, linters, and then pulled all the uses that were not in tests. Here they are: x/tools uses errors.Unwrap to get to the innermost error, without regard for the structure of what is above it. I suspect that this could would be better served by being able to evaluate a predicate on every error in the tree. aws-sdk-go uses errors.Unwrap to recursively evaluate a predicate on every error in the tree, with early stopping. containerd uses errors.Unwrap in a way that I find confusing and might be buggy. It uses it to peel back exactly one layer of wrapping and then compares this to a particular error using string matching. This is fragile (if another wrapping layer gets introduced) and would probably be better served by a way to check every error in the tree. containerd uses errors.Unwrap again to peel back exactly one layer of wrapping. It looks like errors.As would be a better fit here; I don't see any reason why exactly one layer is the right answer here. jwt uses errors.Unwrap to implement Is. In this case, errors.Unwrap is unnecessary, because e is known to have a concrete type that implements Unwrap; the code could call e.Unwrap directly instead. I then grabbed two uses in tests at random: errwrap uses errors.Unwrap specifically to test that it is compatible with how the standard library defines errors.Unwrap. It has no actual use for errors.Unwrap or opinion about its behavior. pgconn uses errors.Unwrap to test its own error's compability with package errors. It looks like it would be equally well served by errors.As. So far, I see no evidence that anyone cares much about the structure of the error tree. It looks to me like they are all: misusing the API (evidence that the API is wrong); using the wrong API (evidence that the API is wrong); using the API pointlessly; or using the API to evaluate a predicate "anywhere along the error chain". |
That's useful data; thanks. We don't have error trees right now; we have an error chain. We can't tell whether people will care about the structure of trees by looking at how they work with linear chains. It'd be interesting to look at use of existing multierr packages to see how often multierrs are converted back to lists of errors. |
I agree. Can I leave that for you to do? I've exceeded my time allotment for this for a while. :) |
I agree with you very much, we should focus on the original core API supporting the multiple error model first, and then expose the complexity to the user. Before that, we can experiment with Join, Split, Walk, etc. Most importantly, decide the definition of |
I'm going to go look, but probably not until next week. Had a few days of meetings, and I'm now exhausted and behind on everything. :) |
Given the size of these error trees, where even "a dozen" would be huge, I would suggest the solution to iteration is simply to define that the traversal used by the rest of the library will be used to return a []error populated in the traversal order used by the rest of the code, or a simple tree struct. Such a small set of values doesn't rate having the iteration proposal placed as a blocker in front of it. It's rarely used functionality (as observed above for Unwrap in general), on the error path (not the happy path where performance is generally a bigger deal), for a rarely-used use case where Is and As is not enough. Code that is generating millions or billions of complex error trees per second that it then somehow needs to deal with in a complicated manner is too broken to be saved by even zero-performance error tree iteration. |
I'd like to help encourage my team to use https://gist.github.com/BenjamenMeyer/ef9926913dcc3b165da8f25a459442a9 I was quite surprised that Love the proposal here but would certainly be happy with something that would allow NOTE: The code in the gist is just a sample for this explicit behavior. Consider the list of errors representing the layers of the code with the singular error what the code actually sees in the tests; yet it fails to be able to detect the lower level errors. HTH Follow-up: This seems to be something that has gone back to 1.13-betas with properly wrapping and detecting the various layered errors: https://groups.google.com/g/golang-nuts/c/SwKZ-qJmZl0?pli=1 |
Errors in Go are no longer viewed as a linear chain, but a tree. See golang/go#53435. Add a Range function that iterates through an error in a pre-order, depth-first order. This matches the iteration order of errors.As in Go 1.20. This adds the logic (but currently commented out) for having Error implement the multi-error version of Unwrap in Go 1.20. It is commented out currently since it causes "go vet" to complain about having the "wrong" signature. Signed-off-by: Joe Tsai <[email protected]>
Go 1.20 includes native support for wrapping multiple errors. Errors which wrap multiple other errors must implement, Unwrap() []error If an error implements this method, `errors.Is` and `errors.As` will descend into the list and continue matching. Versions of Go prior to 1.20, however, still need the old `Is` and `As` method implementations on the error object to get a similar behavior. This change adds the `Unwrap() []error` method gated by a build constraint requiring Go 1.20 or higher. It similarly moves the existing `Is` and `As` methods to a file that is ignored on Go 1.20 or higher. Once Go 1.21 is released and 1.19 is no longer supported, the pre-go1.20 file may be deleted and the build constraints removed. For details, see also the section, "How should existing multierror types adopt the new interface?" of the [multiple errors proposal][1]. [1]: golang/go#53435
Go 1.20 includes native support for wrapping multiple errors. Errors which wrap multiple other errors must implement, Unwrap() []error If an error implements this method, `errors.Is` and `errors.As` will descend into the list and continue matching. Versions of Go prior to 1.20, however, still need the old `Is` and `As` method implementations on the error object to get a similar behavior. This change adds the `Unwrap() []error` method gated by a build constraint requiring Go 1.20 or higher. It similarly moves the existing `Is` and `As` methods to a file that is ignored on Go 1.20 or higher. Once Go 1.21 is released and 1.19 is no longer supported, the pre-go1.20 file may be deleted and the build constraints removed. For details, see also the section, "How should existing multierror types adopt the new interface?" of the [multiple errors proposal][1]. [1]: golang/go#53435 Refs #64
Looks great and can't wait to apply it to engineering! |
An error which implements an "Unwrap() []error" method wraps all the non-nil errors in the returned []error. We replace the concept of the "error chain" inspected by errors.Is and errors.As with the "error tree". Is and As perform a pre-order, depth-first traversal of an error's tree. As returns the first matching result, if any. The new errors.Join function returns an error wrapping a list of errors. The fmt.Errorf function now supports multiple instances of the %w verb. For golang#53435. Change-Id: Ib7402e70b68e28af8f201d2b66bd8e87ccfb5283 Reviewed-on: https://go-review.googlesource.com/c/go/+/432898 Reviewed-by: Cherry Mui <[email protected]> Reviewed-by: Rob Pike <[email protected]> Run-TryBot: Damien Neil <[email protected]> Reviewed-by: Joseph Tsai <[email protected]> (cherry picked from commit 4a0a2b3)
Referring to bokwoon95 #53435 (comment)
and kortschak #53435 (comment)
This muliterror feature is incomplete, because I am required to make |
@dimandzhi This issue is closed. I created #65428 as a new issue for follow up. You can propose new ideas there. |
I'm late to the party, but see https://tip.golang.org/doc/comment#func:
|
For the most recent version of this proposal, see: #53435 (comment) below.
This is a variation on the rejected proposal #47811 (and perhaps that proposal should just be reopened), and an expansion on a comment in it.
Background
Since Go 1.13, an error may wrap another by providing an
Unwrap
method returning the wrapped error. Theerrors.Is
anderrors.As
functions operate on chains of wrapped errors.A common request is for a way to combine a list of errors into a single error.
Proposal
An error wraps multiple errors if its type has the method
Reusing the name
Unwrap
avoids ambiguity with the existing singular Unwrap method. Returning a 0-length list fromUnwrap
means the error doesn't wrap anything. Callers must not modify the list returned byUnwrap
. The list returned byUnwrap
must not contain anynil
errors.We replace the term "error chain" with "error tree".
The
errors.Is
anderrors.As
functions are updated to unwrap multiple errors.Is
reports a match if any error in the tree matches.As
finds the first matching error in a preorder traversal of the tree.The
errors.Join
function provides a simple implementation of a multierr. It does not flatten errors.The
fmt.Errorf
function permits multiple instances of the%w
formatting verb.The
errors.Split
function retrieves the original errors from a combined error.The
errors.Unwrap
function is unaffected: It returnsnil
when called on an error with anUnwrap() []error
method.Questions
Prior proposals have been declined on the grounds that this functionality can be implemented outside the standard library, and there was no good single answer to several important questions.
Why should this be in the standard library?
This proposal adds something which cannot be provided outside the standard library: Direct support for error trees in
errors.Is
anderrors.As
. Existing combining errors operate by providingIs
andAs
methods which inspect the contained errors, requiring each implementation to duplicate this logic, possibly in incompatible ways. This is best handled inerrors.Is
anderrors.As
, for the same reason those functions handle singular unwrapping.In addition, this proposal provides a common method for the ecosystem to use to represent combined errors, permitting interoperation between third-party implementations.
How are multiple errors formatted?
A principle of the
errors
package is that error formatting is up to the user. This proposal upholds that principle: Theerrors.Join
function combines error text with a user-provided separator, andfmt.Errorf
wraps multiple errors in a user-defined layout. If users have other formatting requirements, they can still create their own error implementations.How do
Is
andAs
interact with combined errors?Every major multierror package that I looked at (see "Prior art" below) implements the same behavior for
Is
andAs
:Is
reports true if any error in the combined error matches, andAs
returns the first matching error. This proposal follows common practice.Does creating a combined error flatten combined errors in the input?
The
errors.Join
function does not flatten errors. This is simple and comprehensible. Third-party packages can easily provide flattening if desired.Should
Split
unwrap errors that wrap a single error?The
errors.Split
function could call the single-wrappingUnwrap() error
method when present, converting a non-nil result into a single-element slice. This would allow traversing an error tree with only calls toSplit
.This might allow for a small improvement in the convenience of code which manually traverses an error tree, but it is rare for programs to manually traverse error chains today. Keeping
Split
as the inverse ofJoin
is simpler.Why does the name of the
Split
function not match theUnwrap
method it calls?Giving the single- and multiple-error wrapping methods the same name neatly avoids any questions of how to handle errors that implement both.
Split
is a natural name for the function that undoes aJoin
.While we could call the method
Split
, or the functionUnwrapMultiple
, or some variation on these options, the benefits of the above points outweigh the value in aligning the method name with the function name.Prior art
There have been several previous proposals to add some form of combining error, including:
https://go.dev/issue/47811: add Errors as a standard way to represent multiple errors as a single error
https://go.dev/issue/48831: add NewJoinedErrors
https://go.dev/issue/20984: composite errors
https://go.dev/issue/52607: add With(err, other error) error
fmt.Errorf("%w: %w", err1, err2) is largely equivalent to With(err1, err2).
Credit to @jimmyfrasche for suggesting the method name
Unwrap
.There are many implementations of combining errors in the world, including:
https://pkg.go.dev/github.com/hashicorp/go-multierror (8720 imports)
https://pkg.go.dev/go.uber.org/multierr (1513 imports)
https://pkg.go.dev/tailscale.com/util/multierr (2 imports)
The text was updated successfully, but these errors were encountered: