Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add load_to_cache field in preheat request using Open API #3838

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

SouthWest7
Copy link

Description

A new load_to_cache field has been added to the OpenAPI format to indicate whether to write the downloaded piece content to the user-space cache. This field is a Boolean type. Users can set it to true to cache the content locally to improve subsequent access efficiency. If it is set to false, no caching will be performed.

curl --location --request POST 'http://dragonfly-manager:8080/oapi/v1/jobs' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer your_dragonfly_personal_access_token' \
--header 'Authorization: token your_example.com_personal_access_token' \
--data-raw '{
    "type": "preheat",
    "args": {
        "type": "file",
        "url": "https://example.com",
        "scope": "single_seed_peer",
        "scheduler_cluster_ids":[1],
        "load_to_cache": true
    }
}'

Related Issue

#3742

Motivation and Context

Purpose

This change enhances control over the caching behavior during preheating, allowing for optimization of data retrieval. By using the LoadToCache field, users can decide whether to store the downloaded content in cache memory, improving performance for repeated access.

Changes:

  1. Added LoadToCache field:
  • In internal/job/types.go, a new boolean field LoadToCache has been added to the PreheatRequest struct, indicating whether the content should be cached.
  • In manager/types/job.go, the PreheatArgs struct now includes the LoadToCache field to handle the preheating caching behavior.
  • In scheduler/job/job.go, the LoadToCache field is integrated into the preheating job execution, ensuring that the preheating behavior respects the cache flag.
  1. Updated Preheat Request Handling:
  • In manager/job/preheat.go, the CreatePreheat method now passes the LoadToCache flag in the request, ensuring that the cache option is handled during the preheating process.
  1. Modified Preheat Job Processing:
  • The preheating job functions (preheatAllSeedPeers, preheatAllPeers, and preheatV2) in scheduler/job/job.go have been updated to incorporate the LoadToCache field when executing preheating operations.

Screenshots (if appropriate)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation Update (if none of the other choices apply)

Checklist

  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.

@SouthWest7 SouthWest7 requested a review from a team as a code owner February 17, 2025 05:25
Copy link

codecov bot commented Feb 20, 2025

Codecov Report

Attention: Patch coverage is 0% with 4 lines in your changes missing coverage. Please review.

Project coverage is 35.77%. Comparing base (6a879ab) to head (abce59c).
Report is 8 commits behind head on main.

Files with missing lines Patch % Lines
scheduler/job/job.go 0.00% 3 Missing ⚠️
manager/job/preheat.go 0.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3838      +/-   ##
==========================================
- Coverage   35.85%   35.77%   -0.08%     
==========================================
  Files         337      337              
  Lines       38991    38995       +4     
==========================================
- Hits        13979    13950      -29     
- Misses      24095    24124      +29     
- Partials      917      921       +4     
Flag Coverage Δ
unittests 35.77% <0.00%> (-0.08%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
manager/job/preheat.go 45.89% <0.00%> (-0.18%) ⬇️
scheduler/job/job.go 0.00% <0.00%> (ø)

... and 4 files with indirect coverage changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant