feat: add fetching cardano database snapshots by epoch in client and cli #2744

Alenar · 2025-10-17T14:43:06Z

Content

This PR add new capabilities to the client library and the client-cli

Client library:
- cardano_database_v2: add new methods to fetch snapshots for a specific epoch, for the latest epoch, or for the latest epoch with an offset
- cardano_stake_distribution: add new methods to fetch snapshots for the latest epoch, or for the latest epoch with an offset
Client CLI - cardano-database command: add a new --epoch parameter to fetch snapshots for a specific epoch, for the latest epoch, or for the latest epoch with an offset (i.e. --epoch 4, --epoch latest, --epoch latest-4)

Details

doc website: update documentation for the client cli cardano-database list parameters table.
end to end: add a new step that fetch the list of cardano db v2 snapshots for the latest-5

`mithril-common`

Epoch:
- add new type EpochSpecifier to store user inputs in the form of <number>, latest, latest-<number> coming for either http routes path or the cli.
- add Epoch::parse_specifier
- implement FromStr to Epoch to allow child crates to do "87".parse::<Epoch>() instead of "87".parse::<u64>().map(Epoch)

`mithril-client` library

cardano_stake_distribution client:
- add get_for_latest_epoch and get_for_latest_epoch_with_offset
- isolate the error logic for list retrieval in order to simplify the tests
cardano_database_v2 client:
- add list_by_epoch, list_for_latest_epoch, and list_for_latest_epoch_with_offset
aggregator client: add supports for the latest aggregator routes that fetch cardano database snapshots and cardano stake distribution for an epoch or latest or latest-<offset>

`mithril-client-cli`

cardano-database commands:
- generalize to the module the warn_unused_parameter_with_v1_backend logic so it can be reused in other commands than download
- list command: add --epoch parameter which is parsed into a EpochSpecifier and is only honored if the backend is set to v2 (else a warning is print)

`mithril-client-wasm`

add get_cardano_stake_distribution_for_latest_epoch
add unstable list_cardano_database_v2_per_epoch, list_cardano_database_v2_for_latest_epoch
both ***_for_latest_epoch functions allow to pass a json object which contains an optional offset, i.e in js:

const cardano_database_snapshots_for_latest_epoch_with_offset =
      await client.list_cardano_database_v2_for_latest_epoch({ offset: 3 });

update ci-test node project to check the new functions

`mithril-aggregator-fake`

add /artifact/cardano-database/epoch/{epoch} route with support of latest and latest-{offset}
/artifact/cardano-stake-distribution/epoch/{epoch} route: add support of latest and latest-{offset}

`mithril-build-script`

new generated hashmap (indexed by epoch number): cardano_database_snapshot_list_per_epoch, cardano_stake_distributions_per_epoch
new generated id list: cardano_database_snapshot_epochs (indexed by epoch number)
changed cardano_stake_distribution_epochs key from &str to an epoch number

Pre-submit checklist

Branch
- Tests are provided (if possible)
- Crates versions are updated (if relevant)
- CHANGELOG file is updated (if relevant)
- Commit sequence broadly makes sense
- Key commits have useful messages
PR
- All check jobs of the CI have succeeded
- Self-reviewed the diff
- Useful pull request description
- Reviewer requested
Documentation
- Update documentation website (if relevant)
- No new TODOs introduced

Issue(s)

Relates to #2704

mithril-client-cli/src/commands/cardano_db/list.rs

github-actions · 2025-10-17T14:50:00Z

Test Results

4 files ± 0 168 suites ±0 23m 16s ⏱️ -33s
2 261 tests +20 2 261 ✅ +20 0 💤 ±0 0 ❌ ±0
6 987 runs +80 6 987 ✅ +80 0 💤 ±0 0 ❌ ±0

Results for commit 18dfe19. ± Comparison against base commit 29375b2.

This pull request removes 11 and adds 31 tests. Note that renamed tests count towards both.

mithril-build-script ‑ fake_aggregator::tests::extract_csd_epochs_with_invalid_json
mithril-build-script ‑ fake_aggregator::tests::extract_csd_epochs_with_valid_data
mithril-build-script ‑ fake_aggregator::tests::test_extract_csd_epochs_with_empty_map
mithril-build-script ‑ fake_aggregator::tests::test_extract_csd_epochs_with_missing_epoch
mithril-client ‑ cardano_stake_distribution_client::tests::get_cardano_stake_distribution_by_epoch_returns_error
mithril-client ‑ cardano_stake_distribution_client::tests::get_cardano_stake_distribution_by_epoch_returns_error_when_invalid_json_structure_in_response
mithril-client ‑ cardano_stake_distribution_client::tests::get_cardano_stake_distribution_by_epoch_returns_none_when_not_found_or_remote_server_logical_error
mithril-client ‑ cardano_stake_distribution_client::tests::get_cardano_stake_distribution_returns_error
mithril-client ‑ cardano_stake_distribution_client::tests::get_cardano_stake_distribution_returns_error_when_invalid_json_structure_in_response
mithril-client ‑ cardano_stake_distribution_client::tests::get_cardano_stake_distribution_returns_none_when_not_found_or_remote_server_logical_error
…

mithril-build-script ‑ fake_aggregator::tests::test_extract_item_by_epoch_by_epoch_with_invalid_json
mithril-build-script ‑ fake_aggregator::tests::test_extract_item_by_epoch_by_epoch_with_valid_data
mithril-build-script ‑ fake_aggregator::tests::test_extract_item_by_epoch_with_empty_map
mithril-build-script ‑ fake_aggregator::tests::test_extract_item_by_epoch_with_missing_epoch
mithril-build-script ‑ fake_aggregator::tests::test_extract_item_list_per_epoch_for_epoch
mithril-build-script ‑ fake_aggregator::tests::test_extract_item_list_per_epoch_with_invalid_json
mithril-build-script ‑ fake_aggregator::tests::test_extract_item_list_per_epoch_with_list
mithril-build-script ‑ fake_aggregator::tests::test_extract_item_list_per_epoch_with_missing_epoch
mithril-client ‑ cardano_database_client::fetch::tests::list::list_cardano_database_snapshots_by_epoch_returns_messages
mithril-client ‑ cardano_database_client::fetch::tests::list::list_cardano_database_snapshots_for_latest_epoch_returns_messages
…

♻️ This comment has been updated with latest results.

jpraynaud

LGTM 👍

mithril-common/src/entities/epoch.rs

mithril-test-lab/mithril-aggregator-fake/src/shared_state.rs

mithril-client-cli/src/commands/cardano_db/list.rs

.github/workflows/test-client.yml

turmelclem

LGTM 👍

…h/without offset)

- unify error handlings tests by testing the inner function that handle them directly instead of copying those tests for each - use dummy directly instead of modifying them since it does not bring much value here

…h/without offset

So it can be re-used accross crates. - `EpochSpecifier` enum to represents the cases between <{uint}, 'latest', latest-{uint}> - add `Epoch::parse_specifier` - impl `FromStr` to epoch to avoid having downstream code specifying the inner type of the epoch when parsing from str - update aggregator `ExpandedEpoch` logic

- move it from the cardano db download module to its parent module so all cardano db commmands can use it. - make the unused parameters list configurable

With `{epoch}` either a number, `latest`, or `latest-x` where X is a number + Add support for `latest` / `latest-x` on the route `cardano-stake-distribution/epoch/{epoch}`

+ add epoch lists for cardano database (cdb), indexed by epoch number + refactor cardano stake distribution (csd) epoch list gen: - use a common mechanism with cdb - change index from str to epoch number

... to generate cardano stake distribution & cardano database lists per epoch. Note: this means adding `mithril-build-script` as a straight dependency for `mithril-aggregator-fake`, this is fine since it's a test only tool.

…cdb and csd

The idea was to avoid import of `EpochSpecifier` in callers but since the import is needed anyway to match the result it's more clear for it to be part of the specifier api.

* mithril-build-script from `0.2.27` to `0.2.28` * mithril-aggregator from `0.7.88` to `0.7.89` * mithril-client-cli from `0.12.32` to `0.12.33` * mithril-client-wasm from `0.9.6` to `0.9.7` * mithril-client from `0.12.33` to `0.12.34` * mithril-common from `0.6.23` to `0.6.24` * mithril-aggregator-fake from `0.4.14` to `0.4.15` * mithril-end-to-end from `0.4.105` to `0.4.106` * [js] client-wasm-ci-test from `0.3.11` to `0.3.12` * [js] mithril-client-wasm from `0.9.6` to `0.9.7`

Alenar requested review from jpraynaud and turmelclem October 17, 2025 14:43

Alenar self-assigned this Oct 17, 2025

github-advanced-security bot found potential problems Oct 17, 2025

View reviewed changes

mithril-client-cli/src/commands/cardano_db/list.rs Fixed Show fixed Hide fixed

mithril-client-cli/src/commands/cardano_db/list.rs Fixed Show fixed Hide fixed

Alenar force-pushed the djo/2704/cdb-by-epoch-in-client branch from 6200f62 to 24af55f Compare October 17, 2025 17:14

Alenar temporarily deployed to testing-preview October 17, 2025 17:36 — with GitHub Actions Inactive

jpraynaud approved these changes Oct 20, 2025

View reviewed changes

mithril-common/src/entities/epoch.rs Outdated Show resolved Hide resolved

mithril-test-lab/mithril-aggregator-fake/src/shared_state.rs Show resolved Hide resolved

Alenar force-pushed the djo/2704/cdb-by-epoch-in-client branch from 24af55f to 40f0677 Compare October 20, 2025 16:33

github-advanced-security bot found potential problems Oct 20, 2025

View reviewed changes

mithril-client-cli/src/commands/cardano_db/list.rs Fixed Show fixed Hide fixed

mithril-client-cli/src/commands/cardano_db/list.rs Fixed Show fixed Hide fixed

Alenar force-pushed the djo/2704/cdb-by-epoch-in-client branch from 40f0677 to 83ebcc0 Compare October 21, 2025 09:58

Alenar temporarily deployed to testing-preview October 21, 2025 10:08 — with GitHub Actions Inactive

jpraynaud approved these changes Oct 21, 2025

View reviewed changes

.github/workflows/test-client.yml Outdated Show resolved Hide resolved

turmelclem approved these changes Oct 21, 2025

View reviewed changes

Alenar force-pushed the djo/2704/cdb-by-epoch-in-client branch from 83ebcc0 to d711d4d Compare October 21, 2025 13:19

Alenar added 13 commits October 21, 2025 15:19

feat(client-lib): list cardano db snapshot by epoch/latest epoch (wit…

dee0d33

…h/without offset)

test(client-lib): simplify cardano stake distribution tests

c5e00ae

- unify error handlings tests by testing the inner function that handle them directly instead of copying those tests for each - use dummy directly instead of modifying them since it does not bring much value here

feat(client-lib): get cardano stake distribution for latest epoch wit…

6b0f330

…h/without offset

refactor(client-cli): make unused param in v2 warning reusable

786c9e7

- move it from the cardano db download module to its parent module so all cardano db commmands can use it. - make the unused parameters list configurable

feat(client-cli): add --epoch parameter to cardano db v2 list command

87662dd

test(e2): add run of cdb snapshot list --epoch latest-5

2f0b6e7

doc(website): document --epoch param in cdb snapshot list params table

66541f2

test(aggregator-fake): add route cardano-database/epoch/{epoch}

6ede592

With `{epoch}` either a number, `latest`, or `latest-x` where X is a number + Add support for `latest` / `latest-x` on the route `cardano-stake-distribution/epoch/{epoch}`

feat(build-script): add generation of cdb list per epoch & csd per epoch

a2bd270

+ add epoch lists for cardano database (cdb), indexed by epoch number + refactor cardano stake distribution (csd) epoch list gen: - use a common mechanism with cdb - change index from str to epoch number

feat(client-wasm): add list/get per epoch/latest/latest(-offset) for …

7d4445c

…cdb and csd

refactor(common): move parsing of EpochSpecifier from Epoch to self

edf6dc3

The idea was to avoid import of `EpochSpecifier` in callers but since the import is needed anyway to match the result it's more clear for it to be part of the specifier api.

Alenar force-pushed the djo/2704/cdb-by-epoch-in-client branch from d711d4d to a03655f Compare October 21, 2025 13:19

Alenar temporarily deployed to testing-preview October 21, 2025 13:30 — with GitHub Actions Inactive

Alenar force-pushed the djo/2704/cdb-by-epoch-in-client branch from 6560a47 to fc11abe Compare October 21, 2025 14:03

Alenar temporarily deployed to testing-preview October 21, 2025 14:16 — with GitHub Actions Inactive

Alenar added 4 commits October 21, 2025 16:47

ci(client-manual-workflow): check list of cardano db v2 for latest epoch

7ad29c9

ci(client-manual-workflow): fix missing prepare steps for wasm test

c8f0927

chore: update changelog

18dfe19

Alenar force-pushed the djo/2704/cdb-by-epoch-in-client branch from fc11abe to 18dfe19 Compare October 21, 2025 14:47

Alenar temporarily deployed to testing-preview October 21, 2025 15:05 — with GitHub Actions Inactive

Alenar merged commit f98f9d2 into main Oct 21, 2025
41 of 52 checks passed

Alenar deleted the djo/2704/cdb-by-epoch-in-client branch October 21, 2025 15:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: add fetching cardano database snapshots by epoch in client and cli #2744

feat: add fetching cardano database snapshots by epoch in client and cli #2744

Uh oh!

Alenar commented Oct 17, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Oct 17, 2025 •

edited

Loading

Uh oh!

jpraynaud left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

turmelclem left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

feat: add fetching cardano database snapshots by epoch in client and cli #2744

feat: add fetching cardano database snapshots by epoch in client and cli #2744

Uh oh!

Conversation

Alenar commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Content

Details

mithril-common

mithril-client library

mithril-client-cli

mithril-client-wasm

mithril-aggregator-fake

mithril-build-script

Pre-submit checklist

Issue(s)

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Uh oh!

jpraynaud left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

turmelclem left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Alenar commented Oct 17, 2025 •

edited

Loading

`mithril-common`

`mithril-client` library

`mithril-client-cli`

`mithril-client-wasm`

`mithril-aggregator-fake`

`mithril-build-script`

github-actions bot commented Oct 17, 2025 •

edited

Loading