Experimental: Cache in client certificate verification (lib and wasm)

[Alenar]

Content

This PR add a experimental cache system to the certificate chain verification of the client library.

This cache have two implementation:

• One purely in-memory
• One only for the client-wasm that store its value in the browser local storage.

Client library

• Add a CertificateVerifierCache trait which define how to: get one cached value, add one value to a cache, reset all cached values.
  • Only available using the unstable feature
  • This cache only contains the certificate hash (key) and the certificate parent hash (value).
• Refactor the certificate chain validation process to use the cache:
  • Only available using the unstable feature
  • If a certificate hash is cached it's considered valid and the verification continue with the parent hash
  • Certificate are downloaded only if not cached
  • The certificate given as a starting point and the genesis certificate are always validated without the cache
  • If validated without the cache the certificate hash and parent hash are then stored in the cache
  • Genesis certificate are never stored in the cache
  • Providing a cache system is optional
• Add the MemoryCertificateVerifierCache struct:
  • Only available using the unstable feature
  • implement CertificateVerifierCache using a in-memory HashMap
  • have configurable expiration delay for its cached values
• Change the certificate_client module from a single file to a folder with the following organization:
  • api.rs: define the main CertificateClient struct and the public traits that extends it (such as the CertificateVerifierCache).
  • fetch.rs: implementation and tests of Certificate data fetching (get, list)
  • verify.rs: implementation and tests of Certificate chain verification
  • verify-cache: a submodule directory to regroup all cache implementations that are part of the client library
• Add a new feedback event: CertificateValidatedFromCache

Client Wasm library

• Add the LocalStorageCertificateVerifierCache struct:
  • implement CertificateVerifierCache using browsers local storage
  • for simplicity each value are stored in a separate key in the local storage (this have the side-effect of flooding the local storage with values, before it only had the 'Explorer_state' value).
    We could instead store them in a single key but that would mean serializing and/or de-serializing the whole cache at each access, alternatively it could be stored in-memory with a synchronization system.
• Add two new clients options:
  • enable_certificate_chain_verification_cache: default false, when used conjointly with unstable it load the LocalStorageCertificateVerifierCache.
  • certificate_chain_verification_cache_duration_in_seconds: modify cache item expiration duration, default to 604800 (one week).
• When trying to use the LocalStorageCertificateVerifierCache and the local storage is not available on the current platform (such as with nodejs) a warning is logged in the console and the cache system remain disabled.

Explorer

• Enable LocalStorageCertificateVerifierCache
• Certificate validated using the cache use a specific label
• Add a clickable link at the end of the verification process to clear the cache.

Demo

Here's a screencast of the cache in action:

Screencast.from.2024-12-12.16-30-18.webm

Pre-submit checklist

• Branch
  • Tests are provided (if possible)
  • Crates versions are updated (if relevant)
  • Commit sequence broadly makes sense
  • Key commits have useful messages
• PR
  • No clippy warnings in the CI
  • Self-reviewed the diff
  • Useful pull request description
  • Reviewer requested
• Documentation
  • Update README file (if relevant)
  • Update documentation website (if relevant)
  • Add dev blog post (if relevant)

Comments

As the security of this cache system is quite low (see comment below), this should be only see as experimental and not used in production.

Issue(s)

Relates to #1484

[github-actions]

Test Results

    4 files  ± 0     52 suites  ±0   10m 11s ⏱️ +16s
1 448 tests +16  1 448 ✅ +16  0 💤 ±0  0 ❌ ±0 
1 704 runs  +61  1 704 ✅ +61  0 💤 ±0  0 ❌ ±0 

Results for commit 052de8f. ± Comparison against base commit d106a3f.

This pull request removes 8 and adds 24 tests. Note that renamed tests count towards both.

mithril-client ‑ certificate_client::tests::get_certificate_empty_list
mithril-client ‑ certificate_client::tests::get_certificate_list
mithril-client ‑ certificate_client::tests::test_show_ko
mithril-client ‑ certificate_client::tests::test_show_ok_none
mithril-client ‑ certificate_client::tests::test_show_ok_some
mithril-client ‑ certificate_client::tests::validating_chain_send_feedbacks
mithril-client ‑ certificate_client::tests::verify_chain_return_certificate_with_given_hash
mithril-common ‑ test_utils::certificate_chain_builder::test::builds_certificate_chain_correctly_chained

mithril-client ‑ certificate_client::fetch::tests::get_certificate_empty_list
mithril-client ‑ certificate_client::fetch::tests::get_certificate_list
mithril-client ‑ certificate_client::fetch::tests::test_show_ko
mithril-client ‑ certificate_client::fetch::tests::test_show_ok_none
mithril-client ‑ certificate_client::fetch::tests::test_show_ok_some
mithril-client ‑ certificate_client::verify::tests::cache::genesis_certificates_verification_result_is_not_cached
mithril-client ‑ certificate_client::verify::tests::cache::non_genesis_certificates_verification_result_is_cached
mithril-client ‑ certificate_client::verify::tests::cache::verification_of_certificates_should_not_use_cache_until_crossing_an_epoch_boundary
mithril-client ‑ certificate_client::verify::tests::cache::verification_of_first_certificate_of_a_chain_should_always_fetch_it_from_network
mithril-client ‑ certificate_client::verify::tests::cache::verify_chain_return_certificate_with_cache
…

♻️ This comment has been updated with latest results.

[Alenar]

Security assessments

System overview

Without cache

The operations that takes the most time and resources are verifying the signature and downloading each certificate from the aggregator.

Without cache the following checks are done:

• integrity:
  • does recomputing the certificate hash give the stored value
  • does recomputing the signed message based from the protocol message give the stored value
• cryptography: does the multi signature, or genesis signature (depending on the certificate type), sign the certificate signed_message
• chaining:
  • light circularity check: is the certificate chain to itself ?
  • does the certificate hash match the parent certificate hash
  • does the certificate avk match the parent certificate avk

With cache

• The cache only consist of a map of certificate hash associated with the parent certificate hash
• The certificate given as a starting point is never validated from the cache
• The genesis certificate is never stored in the cache, but it's easy for third party to add it manually (this would lead to an error since the cache system is not designed to return genesis certificate)
• If the hash of a certificate is in the cache, it's believed valid
• Limited additional metadata can be added by implementations, but nothing can be added to add extra checks when validating the chain
• No check are done against cached entries except for expiration
• The data stored is trivial and can be easily forged
• The system won't use the cache until crossing an epoch boundary, this ensure that the AVK of the last epoch was signed in a non-cached certificate.

Recap table

	Normal validation	Cached verification
Integrity - certificate hash	✅	❌
Integrity - protocol message	✅	❌
Cryptography - signature	✅	❌
Chaining - circularity	✅	❌
Chaining - hash	✅	✅
Chaining - avk	✅	✅

Risks

With those characteristics the cache system could easily be compromised (non-exhaustive list) :

• by inserting the genesis certificate in the cache to make all verification fails
• by introducing a circularity making the computation never end
• by linking a certificate to another one (or even directly to the genesis) to avoid checking part of the chain

Potential enhancements

We could do almost of the checks by storing almost all metadata from the certificate, except the signature and the signers list (to limit the amount of data to store).
This would make the forging of a fake cached value much harder at the cost of storing a signifiant supplemental amount of data.
Storage tight implementations such as local storage could have to replaced by more complex, but capable, systems (probably indexed db for browsers).

[sfauvel]

LGTM

[dlachaume]

LGTM 👍
Just some minor suggestions.