feat: support to set piece length for preheat #3848
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This pull request introduces the
PieceLength
parameter to various job management and scheduling functions, allowing for more granular control over piece lengths during file download tasks. This change affects multiple files and functions across the codebase, ensuring that thePieceLength
parameter is properly handled and propagated through the system.Key changes include:
Introduction of
PieceLength
Parameter:internal/job/types.go
: AddedPieceLength
to thePreheatRequest
struct.manager/types/job.go
: AddedPieceLength
toPreheatArgs
,GetTaskArgs
, andDeleteTaskArgs
. [1] [2] [3]Modification of TaskID Generation:
pkg/idgen/task_id.go
: UpdatedTaskIDV2
to includePieceLength
in the task ID generation process.pkg/idgen/task_id_test.go
: Added tests forTaskIDV2
to handlePieceLength
. [1] [2] [3] [4]Job Management Updates:
manager/job/preheat.go
: IncorporatedPieceLength
inCreatePreheat
andparseLayers
functions. [1] [2]manager/job/task.go
: UpdatedCreateGetTask
andCreateDeleteTask
to includePieceLength
in the task ID generation. [1] [2]Scheduler and Resource Updates:
scheduler/job/job.go
: Updatedpreheat
,preheatAllSeedPeers
,preheatAllPeers
, andpreheatV2
to handlePieceLength
. [1] [2] [3] [4]scheduler/resource/standard/task.go
: ChangedPieceLength
type inTask
struct and related functions. [1] [2]Test and Utility Updates:
scheduler/resource/standard/task_test.go
: Updated mock task ID generation to includePieceLength
.scheduler/scheduling/evaluator/evaluator_base_test.go
: Updated mock task ID generation to includePieceLength
.scheduler/scheduling/scheduling_test.go
: Updated mock task ID generation to includePieceLength
.scheduler/service/service_v1_test.go
: Updated mock task ID generation to includePieceLength
.scheduler/service/service_v2.go
: Updated task options to handlePieceLength
. [1] [2]test/e2e/v2/util/file.go
: UpdatedGetTaskID
to includePieceLength
.test/e2e/v2/util/task.go
: AddedPieceLength
to task ID struct and related functions. [1] [2]Related Issue
Motivation and Context
Screenshots (if appropriate)
Types of changes
Checklist