IPNS: allow lean V2-only records

[lidel]

Where we are today (2023 Q1)

We are still living with the legacy of decisions made in 2021 and require creation of hybrid V1+V2 records, but we only care about V2 fields during the validation.

I've researched and documented the current state in #319, and obviously it is not the best.

Gist:

• Created records have both v1 and V2 signatures, and we have duplicated values in both top level protobuf AND data CBOR field
• Record Validation only cares about V2 signature, but still requires fields related to V1 to be always present in a record and match values from CBOR in data field, for the record to be considered valid
  • This means many fields MUST be duplicated, even when both ends use a modern client that only cares about signatureV2 that guards CBOR field.

We've been producing and expecting these hybrid V1+V2 records since 2021.

Proposed move to V2-only IPNS

It is time to clean up IPNS ecosystem and stop requiring V1 fields when there is no signatureV1.

For modern IPNS, the outer IpnsEntry protobuf should effectively only have two required fields: data and its signatureV2, and such record, as long signature is valid, should be accepted as valid.

We can get to that future in two steps:

1. Keep producing V1+V2 records as backward-compatible default, but relax validation
   • Only check/require fields to be duplicated in top level protobuf IF signatureV1 is present. IF there is no signatureV1 and only signatureV2 and data fields are present and valid, the record should be considered valid.
   • This will allow people to build V2-only systems that produce records that are considered valid.
2. (at some point in the future) stop producing V1+V2 and switch to publishing V2-only records that are protobuf with only two fields: Data CBOR+signatureV2
   • we have to do this in two steps, because we've rushed the way V2 was introduced in 2021, and STILL require field copying, even when signatureV1 is missing 🙃 and we could only care about CBOR versions from the start :(

Migration plan for V2-only IPNS

Beta Give feedback

1. Create IPIP that updates record Creation and Validation to be V2-only. (move Backward Compatibility section to the end, move current V1+V2 creation/validation to "Backward Compatibility" and then relax legacy record Validation to be compatible with new default) → IPIP-428: Allowing V2-Only Records in IPNS #428
2. Implement relaxed validation in boxo/ipns (see this old PR) → feat(ipns): refactored IPNS package with lean records boxo#339
3. Implement relaxed validation in js-ipns
4. Kubo: ipfs name publish: add a flag --v1compat that is true by default, but allows publisher/tests to opt-out and produce V2-only records → feat(ipns): records with V2-only signatures kubo#9932
5. Iroh: Reach out to Iroh for parity in Rust. Ideally, they should start clean and publish V2-only from the start (cc IPNS support rs-ipfs/rust-ipfs#6, IPNS publishing n0-computer/beetle#58, IPNS resolution support n0-computer/beetle#82, [WIP] feat: Basic IPNS support n0-computer/iroh#539)
6. Make sure gateway-conformance tests are updated and test V2-only records (https://github.com/ipfs/kubo/pull/9667/files#diff-d4f4dee0e70225e2de23ea521dc782efc5438d721194aa0882f8a40a0e288656R83 - this needs version with and without V1 fields)

cc #205 @aschmahmann for sanity check

[ianopolous]

Yes please! This is long overdue. I've just finished implementing IPNS in Java and was wondering when we would do this. I've gone with only including V2 sig already, but as you say the matching v1 protobuf fields are still required. If this is likely to happen, then I will just relax the validation now.

[aschmahmann]

Yes @lidel we should absolutely do this. I feel bad that this is the current state of things, at the time it was basically the only option if we wanted to implement the improvements associated with v2 records which were even longer overdue.

Given both the proliferation of more IPFS implementations and the ongoing refactor work in the Go IPFS libraries it seems like something that both should be done, and should be easier to do than previously.

[aschmahmann]

For modern IPNS, the outer IpnsEntry protobuf should effectively only have two required fields: data and its signatureV2, and such record, as long signature is valid, should be accepted as valid.

There's also the public key field (i.e. for RSA keys) which could still live in the protobuf as it doesn't need to be signed and isn't currently duplicated in the data field.

[hacdias]

I would be happy to take this if it doesn't have any dependencies that we have to wait for.

[lidel]

For anyone following, this work continues in IPIP-428