Proposal: generic *.Result types

[jedevc]

This PR requires #2935 - to see only the commits that differ, see here.

Summary

The new generic Result struct in this package is intended to replace the various Result and related Attestation structs across the codebase, including:

• frontend.Result
• client.Result/gateway.Result
• exporter.Source

Motivation

These Results are all fundamentally the same, and share the same logic, with the only difference being the type used in the Ref/Refs/Attestations properties.

Since these were all separate, we easily encounter the following problems:

• Repetition of struct and method declarations.

  Each separate Result type required it's own properties and methods to be redeclared. Sometimes these were consistent with each other, and other times they were not. Unifying all Results together ensures a lack of duplication, and means that changing the structure of the Result will not require extensive changes throughout every single declaration.

• Complex conversion between Result types.

  At gateway and api boundaries, we are required to convert each Result into a Result of a different type. Previously, this has been a lengthly and error prone conversion, requiring knowledge of the exact structure of the Result, so as to allow traversal. By unifying Results, we can implement a ConvertResult function to convert from types of Result[A] to types of Result[B] given a function from A to B, which cleanly encapsulates the structure of the Result.

• Complex conversion to the protobuf gateway api types.

  At the gateway api, we have to convert to pb.Results. Previously, we had to include logic to convert to and from both frontend.Result and client.Result. With a generic Result, we can simple convert to a Result[pb.Ref] and then have one single set of methods to convert Result[pb.Ref] to a pb.Result.

These problems have been an issue for some time, however, recently with the addition of attestations, the problem has grown, since the amount of logic required at conversion boundaries has massively increased.

How

This patch intends to simplify all of the above, and reduce line-count and code complexity by using the new-ish go generics features. Care has been taken to ensure that this patch is as small as possible, and affects as few Go APIs as possible. This is ensured by using type aliases for the Result type to re-create the various previous Results, e.g. frontend.Result/client.Result/exporter.Source, so that little code in packages that utilize these Results need to be changed. The only major changes take place at conversion boundaries, where, for the most part, the code is simplified.

One major caveat is that the result package is made significantly more complex, due to the generic code, and the need for reflection to perform comparisons of any types to nil, as interfaces such as solver.ResultProxy (used as a Ref type) do not implement the comparable interface, which is a limitation of the current go language spec (hopefully to be resolved in the future).

This is marked as draft for now, as the final implementation isn't fixed, or even certain, as I know that generics have been a controversial feature in the past. Thoughts welcome! 🎉

[jedevc]

Looks like this might also depend on #3022 for the linter to support the updated version of Go.

[tonistiigi]

I'd like to first see (and potentially merge) the version that normalizes the refs to a single map inside Result. This should mean the generic types that this PR adds in

result.Attestation[T]
result.InTotoAttestation[T]

are not needed at all.

That would also simplify the proto layer that this builds on and where it is more important to make changes early because we can't do backward-incompatible changes.

[jedevc]

Have rebased onto #3038, which means that the Attestation interface isn't generic anymore 🎉

The diff affects fewer lines now, though I still think it's a worthwhile patch to pull in.

[jedevc]

Woops, just noticed this little section during review of this, this block isn't needed anymore - I've opened a PR to clear it up in #3086, apologies 😄