Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat:add new api DagstoreInitializeStorage to load shard from piece storage #5032

Merged
merged 2 commits into from
Jul 8, 2022
Merged

feat:add new api DagstoreInitializeStorage to load shard from piece storage #5032

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3c42130
feat:add api DagstoreInitializeStorage
hunjixin Jul 5, 2022
7e4508e
fix:change dag cmd api permission to admin
hunjixin Jul 6, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions venus-shared/api/market/api.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ type IMarket interface {

// DagstoreListShards returns information about all shards known to the
// DAG store. Only available on nodes running the markets subsystem.
DagstoreListShards(ctx context.Context) ([]market.DagstoreShardInfo, error) //perm:read
DagstoreListShards(ctx context.Context) ([]market.DagstoreShardInfo, error) //perm:admin

// DagstoreInitializeShard initializes an uninitialized shard.
//
Expand All @@ -107,14 +107,14 @@ type IMarket interface {
//
// This operation fails if the shard is not in ShardStateNew state.
// It blocks until initialization finishes.
DagstoreInitializeShard(ctx context.Context, key string) error //perm:write
DagstoreInitializeShard(ctx context.Context, key string) error //perm:admin

// DagstoreRecoverShard attempts to recover a failed shard.
//
// This operation fails if the shard is not in ShardStateErrored state.
// It blocks until recovery finishes. If recovery failed, it returns the
// error.
DagstoreRecoverShard(ctx context.Context, key string) error //perm:write
DagstoreRecoverShard(ctx context.Context, key string) error //perm:admin

// DagstoreInitializeAll initializes all uninitialized shards in bulk,
// according to the policy passed in the parameters.
Expand All @@ -123,7 +123,10 @@ type IMarket interface {
// IO pressure if the storage subsystem has a large amount of deals.
//
// It returns a stream of events to report progress.
DagstoreInitializeAll(ctx context.Context, params market.DagstoreInitializeAllParams) (<-chan market.DagstoreInitializeAllEvent, error) //perm:write
DagstoreInitializeAll(ctx context.Context, params market.DagstoreInitializeAllParams) (<-chan market.DagstoreInitializeAllEvent, error) //perm:admin

//DagstoreInitializeStorage initializes all pieces in specify storage
DagstoreInitializeStorage(context.Context, string, market.DagstoreInitializeAllParams) (<-chan market.DagstoreInitializeAllEvent, error) //perm:admin

// DagstoreGC runs garbage collection on the DAG store.
DagstoreGC(ctx context.Context) ([]market.DagstoreShardResult, error) //perm:admin
Expand Down
38 changes: 34 additions & 4 deletions venus-shared/api/market/method.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
* [DagstoreGC](#DagstoreGC)
* [DagstoreInitializeAll](#DagstoreInitializeAll)
* [DagstoreInitializeShard](#DagstoreInitializeShard)
* [DagstoreInitializeStorage](#DagstoreInitializeStorage)
* [DagstoreListShards](#DagstoreListShards)
* [DagstoreRecoverShard](#DagstoreRecoverShard)
* [DealsConsiderOfflineRetrievalDeals](#DealsConsiderOfflineRetrievalDeals)
Expand Down Expand Up @@ -242,7 +243,7 @@ IO pressure if the storage subsystem has a large amount of deals.
It returns a stream of events to report progress.


Perms: write
Perms: admin

Inputs:
```json
Expand Down Expand Up @@ -286,7 +287,7 @@ This operation fails if the shard is not in ShardStateNew state.
It blocks until initialization finishes.


Perms: write
Perms: admin

Inputs:
```json
Expand All @@ -297,12 +298,41 @@ Inputs:

Response: `{}`

### DagstoreInitializeStorage
DagstoreInitializeStorage initializes all pieces in specify storage


Perms: admin

Inputs:
```json
[
"string value",
{
"MaxConcurrency": 123,
"IncludeSealed": true
}
]
```

Response:
```json
{
"Key": "string value",
"Event": "string value",
"Success": true,
"Error": "string value",
"Total": 123,
"Current": 123
}
```

### DagstoreListShards
DagstoreListShards returns information about all shards known to the
DAG store. Only available on nodes running the markets subsystem.


Perms: read
Perms: admin

Inputs: `[]`

Expand All @@ -325,7 +355,7 @@ It blocks until recovery finishes. If recovery failed, it returns the
error.


Perms: write
Perms: admin

Inputs:
```json
Expand Down
15 changes: 15 additions & 0 deletions venus-shared/api/market/mock/mock_imarket.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 8 additions & 4 deletions venus-shared/api/market/proxy_gen.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.