Push, pull, compact, verify commands e2e tests
#893
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…from a TTY (#881) * Interact: fix if is_tty * CHANGELOG.md: update * Release (patch): 0.204.5 * Updated yanked crate (futures-util to 0.3.31)
* Images, kamu-base-with-data-mt: add "kamu" to predefined users * Images, kamu-base-with-data-mt: init-workspace.py use .kamuconfig * CHANGELOG.md: update
rmn-boiko
force-pushed
the
chore/push-and-pul-command-e2e-tests
branch
from
9795293 to
ff8281d
Compare
rmn-boiko
changed the title
Push, pull, compact, verify commands e2e tests
s373r
reviewed
Oct 11, 2024
src/e2e/app/cli/repo-tests/src/commands/test_compact_command.rs
Outdated
Show resolved
Hide resolved
src/e2e/app/cli/repo-tests/src/commands/test_compact_command.rs
Outdated
Show resolved
Hide resolved
s373r
reviewed
Oct 14, 2024
| &self, | ||
| token: &AccessToken, | ||
| account_name_maybe: Option<AccountName>, | ||
| ) -> DatasetId { | ||
| let dataset_id = self.create_player_scores_dataset(token).await; | ||
|
|
||
| // TODO: Use the alias from the reply, after fixing the bug: | ||
| // https://github.com/kamu-data/kamu-cli/issues/891 | ||
|
|
||
| // At the moment, only single-tenant |
There was a problem hiding this comment.
// At the moment, only single-tenan
it doesn't seem to be any more relevant
s373r
approved these changes
Oct 14, 2024
s373r
added a commit
that referenced
this pull request
Oct 14, 2024
* Add query commitments example * Release (patch) `0.204.5`: fix `--yes / -y` flag: fixed when working from a TTY (#881) * Interact: fix if is_tty * CHANGELOG.md: update * Release (patch): 0.204.5 * Updated yanked crate (futures-util to 0.3.31) * CI: Fixes `kamu-base-with-data-mt` image builds (#885) * Images, kamu-base-with-data-mt: add "kamu" to predefined users * Images, kamu-base-with-data-mt: init-workspace.py use .kamuconfig * CHANGELOG.md: update * Add push pull tests * Add s3 tests * Remove unused code * Test transform engine error * Revert check * Add container extra groups * Add verify and compact commands * Fix grammar * Fix review commets * Revert "Merge branch 'master' into chore/push-and-pul-command-e2e-tests" This reverts commit a4ec39c, reversing changes made to 1130fa5. * Clean code * Fix review comments. Iter 2 * Revert ToDo comment --------- Co-authored-by: Sergii Mikhtoniuk <[email protected]> Co-authored-by: Dima Pristupa <[email protected]>
s373r
added a commit
that referenced
this pull request
Oct 15, 2024
* Add query commitments example * Release (patch) `0.204.5`: fix `--yes / -y` flag: fixed when working from a TTY (#881) * Interact: fix if is_tty * CHANGELOG.md: update * Release (patch): 0.204.5 * Updated yanked crate (futures-util to 0.3.31) * CI: Fixes `kamu-base-with-data-mt` image builds (#885) * Images, kamu-base-with-data-mt: add "kamu" to predefined users * Images, kamu-base-with-data-mt: init-workspace.py use .kamuconfig * CHANGELOG.md: update * Add push pull tests * Add s3 tests * Remove unused code * Test transform engine error * Revert check * Add container extra groups * Add verify and compact commands * Fix grammar * Fix review commets * Revert "Merge branch 'master' into chore/push-and-pul-command-e2e-tests" This reverts commit a4ec39c, reversing changes made to 1130fa5. * Clean code * Fix review comments. Iter 2 * Revert ToDo comment --------- Co-authored-by: Sergii Mikhtoniuk <[email protected]> Co-authored-by: Dima Pristupa <[email protected]>
zaychenko-sergei
pushed a commit
that referenced
this pull request
Oct 16, 2024
* E2E: add (or expand) `config`/`init`/`add`/`rename` command tests (#878) * KamuCliApiServerHarnessOptions::with_kamu_config(): introduce * E2E: cover "kamu config" command * E2E, test_init_in_an_existing_workspace(): add * E2E, test_config_set_value(): respect potential CI agent podman configuration * Updated yanked crate: futures-channel * E2E, test_config_get_with_default(): respect potential CI agent podman configuration * kamu-cli-e2e-common: extract "player-scores" & "leaderboard" datasets into statics * assert_cmd: add patch * E2E, test_add_dataset_from_stdin(): add * E2E, test_add_dataset_from_stdin(): add * AddCommand::validate_args(): fix "name" arg processing * E2E, test_add_dataset_with_name(): mix add commands * E2E, test_add_dataset_with_replace(): add * Interact: fix if is_tty * CHANGELOG.md: update * E2E, test_delete_dataset(): add * E2E, test_delete_dataset_recursive(): add * E2E, test_delete_dataset_all(): add * E2E, test_rename_dataset(): add * assert_cmd: use patch (now from the kamu-data org) * kamu-cli-puppet: remove unused dep * Extracted to the separate PR * E2E: add (or expand) `ingest`/`inspect`/`log`/`new`/`reset`/`search`/etc command tests (#883) * E2E, ingest_data_to_player_scores_from_stdio(): add * clippy fixes * E2E, test_ingest_from_stdin(): also check data via the tail command * E2E, ingest_data_to_player_scores_from_stdio(): check data for all ingest steps * E2E, test_ingest_recursive(): add... a part of the test * E2E, test_ingest_with_source_name(): add * E2E, test_inspect_lineage(): add * IngestCommand: add a todo * DATASET_ROOT_PLAYER_SCORES_INGEST_DATA_NDJSON_CHUNK_{1,2,3}: stabilize tests * Rename: LineageCommand -> InspectLineageCommand * E2E: stabilize ingest command tests * KamuCliPuppetExt::list_blocks(): add * E2E, test_inspect_query(): add * E2E, test_inspect_schema(): add * kamu-cli, Login: fix a doc string typo * E2E, test_log(): add * E2E, test_new_{root,derivative}(): add * E2E, test_reset(): add * cli-reference.md: update * test_repo_alias_command -> test_repo_command * E2E, KamuApiServerClientExt::ingest_data(): add * E2E, test_search_multi_user(): add * E2E, test_search_by_name(): add * E2E, test_search_by_repo(): add * E2E, test_sql_command(): add * E2E, test_gc(): add * E2E, test_system_info(): add * E2E, test_system_diagnose(): add * CI: test fixes * assert_ingest_data_to_player_scores_from_stdio(): stabilize rows order * assert_ingest_data_to_player_scores_from_stdio(): stabilize rows order [2] * test_sql_command(): use the default name * assert_ingest_data_to_player_scores_from_stdio(): stabilize rows order [3] * CHANGELOG.md: update * CI: windows build fixes * E2E, test_tail(): add * assert_ingest_data_to_player_scores_from_stdio(): stabilize rows order [4] * assert_ingest_data_to_player_scores_from_stdio(): stabilize rows order [5] * pretty_assertions::assert_eq!(): expected first, actual after * Remove extra files * E2E: cover "kamu login" / "kamu logout" commands * `Push, pull, compact, verify` commands e2e tests (#893) * Add query commitments example * Release (patch) `0.204.5`: fix `--yes / -y` flag: fixed when working from a TTY (#881) * Interact: fix if is_tty * CHANGELOG.md: update * Release (patch): 0.204.5 * Updated yanked crate (futures-util to 0.3.31) * CI: Fixes `kamu-base-with-data-mt` image builds (#885) * Images, kamu-base-with-data-mt: add "kamu" to predefined users * Images, kamu-base-with-data-mt: init-workspace.py use .kamuconfig * CHANGELOG.md: update * Add push pull tests * Add s3 tests * Remove unused code * Test transform engine error * Revert check * Add container extra groups * Add verify and compact commands * Fix grammar * Fix review commets * Revert "Merge branch 'master' into chore/push-and-pul-command-e2e-tests" This reverts commit a4ec39c, reversing changes made to 1130fa5. * Clean code * Fix review comments. Iter 2 * Revert ToDo comment --------- Co-authored-by: Sergii Mikhtoniuk <[email protected]> Co-authored-by: Dima Pristupa <[email protected]> * E2E: add Outbox processing middleware (#901) * E2E, e2e_middleware_fn(): implement * kamu-adapter-http: hide E2E things behind the "e2e" feature gate * query_handler_post_v2(): fix a typo * kamu-cli: integrate the E2E middleware * CHANGELOG.md: update * Updates after rebasing * CHANGELOG.md: add missed commands * CHANGELOG.md: add a note about a "kamu add" fix * KamuCliPuppetExt::ingest_data(): simplify * E2E: activate new tests for SQLite * E2E: activate new tests for PostgreSQL * E2E: remove "kamu config" tests for SQLite & PostgreSQL * E2E: test_smart_pull_derivative(): use the tail command + stabilization (#903) * E2E: test_smart_pull_derivative(): use the tail command * E2E: run postgres tests one by one * E2E: test_smart_pull_derivative(): stabilize order * E2E: wait_for_flows_to_finish(): increase takes count * E2E, test_ingest_command: stabilize order without the "match_time" changing * E2E, test_log_command: stabilize order without the "match_time" changing * E2E, test_sql_command: stabilize order without the "match_time" changing * E2E, test_tail_command: stabilize order without the "match_time" changing * E2E, test_dataset_trigger_flow(): stabilize order without the "match_time" changing * E2E, test_smart_transfer_protocol: stabilize order without the "match_time" changing * KamuCliPuppetExt: absorb helper methods (#906) * E2E, test_rest_api_request_dataset_tail(): use a const instead of a hardcoded value * E2E, test_verify_command: use a consts instead of hardcoded values * E2E: KamuApiServerClient::get_node_url(): absorb * CI: mark postgres as flaky but run with 8 threads * Tests: add flaky "risingwave" group * E2E, test_dataset_trigger_flow(): add "risingwave" test-group * CI: go not specify threads for postgres * E2E, KamuCliPuppetExt::assert_player_scores_dataset_data(): extract * E2E, KamuCliPuppetExt::assert_success_command_execution(): absorb * E2E, KamuCliPuppetExt::assert_failure_command_execution(): absorb * E2E, assert_ingest_data_to_player_scores_from_stdio(): use assert_success_command_execution_with_input() * KamuCliPuppetExt::assert_player_scores_dataset_data(): use assert_success_command_execution() * E2E, test_config_command: use new methods * E2E, test_inspect_command: use new methods * E2E, test_search_command: use new methods * E2E, test_sql_command: use new methods * E2E, test_tail_command: use new methods * E2E, test_system_api_server_gql_query: use new methods * E2E, test_add_command: use new methods * KamuCliPuppetExt: use iter for maybe_expected_stderr * E2E, test_compact_command: use new methods * E2E, test_delete_command: use new methods * E2E, test_ingest_command: use new methods * E2E, test_init_command: use new methods * E2E, test_login_command: use new methods * E2E, test_new_command: use new methods * E2E, test_rename_command: use new methods * E2E, test_reset_command: use new methods * E2E, test_smart_transfer_protocol: use new methods * E2E, test_system_gc_command: use new methods * E2E, test_verify_command: use new methods * E2E, test_workspace_svc: use new methods * KamuCliPuppetExt::assert_failure_command_execution_with_input(): implement --------- Co-authored-by: Roman Boiko <[email protected]> Co-authored-by: Sergii Mikhtoniuk <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Closes: #issue
Checklist before requesting a review