Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support persist cache task when scheduler replicates task #953

Merged
merged 1 commit into from
Jan 26, 2025

Conversation

gaius-qi
Copy link
Member

@gaius-qi gaius-qi commented Jan 26, 2025

Description

This pull request includes various changes to the dragonfly-client project, focusing on version updates, error handling improvements, and disk space management enhancements. The key changes are listed below.

Version Updates:

  • Updated the version of dragonfly-client and its dependencies from 0.2.5 to 0.2.6 in Cargo.toml. [1] [2]

Error Handling Improvements:

  • Added a new error variant NoSpace to the DFError enum in dragonfly-client-core/src/error/mod.rs to handle cases where there is no space left on the device.

Disk Space Management Enhancements:

  • Introduced methods in dragonfly-client-storage/src/content.rs and dragonfly-client-storage/src/lib.rs to check available and total disk space, and to verify if there is enough space to store content. [1] [2]
  • Modified the GC implementation in dragonfly-client/src/gc/mod.rs to use the new disk space methods for evicting tasks based on disk usage. [1] [2]

Configuration Adjustments:

  • Changed the default scheduler schedule timeout from 10 seconds to 180 seconds in dragonfly-client-config/src/dfdaemon.rs.

Code Cleanup:

  • Removed unused imports and redundant code from various files, including dragonfly-client/src/gc/mod.rs and dragonfly-client/src/grpc/dfdaemon_download.rs. [1] [2]

These changes collectively enhance the project's robustness and efficiency, particularly in handling disk space and improving error management.

Related Issue

dragonflyoss/dragonfly#3811

Motivation and Context

Screenshots (if appropriate)

@gaius-qi gaius-qi added the enhancement New feature or request label Jan 26, 2025
@gaius-qi gaius-qi added this to the v2.3.0 milestone Jan 26, 2025
@gaius-qi gaius-qi self-assigned this Jan 26, 2025
@gaius-qi gaius-qi requested a review from a team as a code owner January 26, 2025 14:19
Copy link

codecov bot commented Jan 26, 2025

Codecov Report

Attention: Patch coverage is 0.78125% with 127 lines in your changes missing coverage. Please review.

Project coverage is 33.19%. Comparing base (e787afe) to head (98e6af0).
Report is 25 commits behind head on main.

Files with missing lines Patch % Lines
dragonfly-client/src/grpc/dfdaemon_upload.rs 0.00% 39 Missing ⚠️
dragonfly-client/src/grpc/dfdaemon_download.rs 0.00% 21 Missing ⚠️
dragonfly-client-storage/src/content.rs 0.00% 18 Missing ⚠️
dragonfly-client-storage/src/lib.rs 0.00% 10 Missing ⚠️
dragonfly-client/src/metrics/mod.rs 0.00% 10 Missing ⚠️
dragonfly-client/src/bin/dfcache/import.rs 0.00% 9 Missing ⚠️
...onfly-client/src/resource/persistent_cache_task.rs 0.00% 7 Missing ⚠️
dragonfly-client/src/resource/piece_collector.rs 0.00% 7 Missing ⚠️
dragonfly-client/src/resource/task.rs 0.00% 4 Missing ⚠️
dragonfly-client-storage/src/metadata.rs 0.00% 1 Missing ⚠️
... and 1 more
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #953      +/-   ##
==========================================
+ Coverage   32.72%   33.19%   +0.46%     
==========================================
  Files          60       59       -1     
  Lines        9776     9638     -138     
==========================================
  Hits         3199     3199              
+ Misses       6577     6439     -138     
Files with missing lines Coverage Δ
dragonfly-client-config/src/dfdaemon.rs 49.07% <100.00%> (ø)
dragonfly-client-core/src/error/mod.rs 59.09% <ø> (ø)
...onfly-client-storage/src/storage_engine/rocksdb.rs 65.85% <ø> (+15.85%) ⬆️
dragonfly-client/src/bin/dfcache/main.rs 0.00% <ø> (ø)
dragonfly-client/src/gc/mod.rs 0.00% <ø> (ø)
dragonfly-client-storage/src/metadata.rs 57.83% <0.00%> (-0.24%) ⬇️
dragonfly-client/src/proxy/cache.rs 55.47% <0.00%> (ø)
dragonfly-client/src/resource/task.rs 0.00% <0.00%> (ø)
...onfly-client/src/resource/persistent_cache_task.rs 0.00% <0.00%> (ø)
dragonfly-client/src/resource/piece_collector.rs 0.00% <0.00%> (ø)
... and 6 more

@gaius-qi gaius-qi merged commit 58da38d into main Jan 26, 2025
6 checks passed
@gaius-qi gaius-qi deleted the feature/update branch January 26, 2025 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant