Release v0.204.4

[0.204.4] - 2024-09-30

Changed

• CLI command tweaks:
  • Make --yes / -y flag global
  • Add confirmation step to system compact command
  • Add support for patterns to system compact to process multiple datasets at once
  • Fixed argument parsing error in kamu system compact command
• Simplified organization of startup initialization code over different components

Fixed

• Broken catalog issue for server and transactional modes
  • Added several E2E tests (happy paths) covering the Flows tab in the UI
• Corrected behavior of MySqlAccountRepository::get_accounts_by_ids(), for the case of empty IDs collection

Release v0.204.3

[0.204.3] - 2024-09-26

Fixed

• Dataset creation with unique alias but with existing id for FS dataset storage mode
• kamu init: fixed regression in case of using exists_ok flag... finally

Release v0.204.2

[0.204.2] - 2024-09-26

Fixed

• kamu init: fixed regression in case of using exists_ok flag

Release v0.204.1

[0.204.1] - 2024-09-25

Fixed

• Fixed build regression, in case web-ui feature flag is used

Release v0.203.1

[0.203.1] - 2024-09-24

Added

• Added database migration & scripting to create an application user with restricted permissions
• kamu delete command will respect dependency graph ordering allowing to delete multiple datasets without encountering dangling reference

Release v0.203.0

[0.203.0] - 2024-09-22

Added

• Support List and Struct arrow types in json and json-aoa encodings

Release v0.202.1

[0.202.1] - 2024-09-20

Fixed

• Open Telemetry integration fixes

Release v0.202.0

[0.202.0] - 2024-09-20

Changed

• Major dependency upgrades:
  • DataFusion 42
  • HTTP stack v.1
  • Axum 0.7
  • latest AWS SDK
  • latest versions of all remaining libs we depend on
• Outbox refactoring towards true parallelism via Tokio spaned tasks instead of futures

Fixed

• Failed flows should still propagate finishedAt time
• Eliminate span.enter, replaced with instrument everywhere

Release v0.201.0

[0.201.0] - 2024-09-18

Added

• REST API: New /verify endpoint allows verification of query commitment as per documentation (#831)

Changed

• Outbox main loop was revised to minimize the number of transactions:
  • split outbox into planner and consumption jobs components
  • planner analyzes current state and loads bunch of unprocessed messages within a 1 transaction only
  • consumption jobs invoke consumers and detect their failures
• Detecting concurrent modifications in flow and task event stores
• Improved and cleaned handling of flow abortions at different stages of processing
• Revised implementation of flow scheduling to avoid in-memory time wheel:
  • recording FlowEventScheduledForActivation event (previously, placement moment into the time wheel)
  • replaced binary heap based time wheel operations with event store queries
  • Postgres/SQlite event stores additionally track activation time for the waiting flows
  • in-memory event store keeps prepared map-based lookup structures for activation time

Release v0.200.0

[0.200.0] - 2024-09-13

Added

• Added first integration of Prometheus metrics starting with Outbox
• Added --metrics CLI flag that will dump metrics into a file after command execution

Changed

• Telemetry improvements:
  • Improved data collected around transactional code
  • Revised associating span objects with large JSON structures such as messages
  • Suppressed several noisy, but not very useful events
• Improved Outbox stability when message consumers fail
• Similarly, Task Executor keeps executing next tasks in case running a task results in an internal error