Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: #4910 venus-market,venus : add sub command to mannage piece storage #153

Merged
merged 23 commits into from
Jul 5, 2022

Conversation

LinZexiao
Copy link
Collaborator

@LinZexiao LinZexiao commented Jun 23, 2022

Related Issues

close filecoin-project/venus#4910

Proposed Changes

add subaommand to set piece storage when market is running
AddFsPieceStorage
ListPieceStorage
RemovePieceStorage

Additional Info

Checklist

Before you mark the PR ready for review, please make sure that:

  • All commits have a clear commit message.
  • The PR title is in the form of of <PR type>: <#issue number> <area>: <change being made>
    • example: fix: #1234 mempool: Introduce a cache for valid signatures
    • PR type: fix, feat, INTERFACE BREAKING CHANGE, CONSENSUS BREAKING, build, chore, ci, docs, misc, perf, refactor, revert, style, test
    • area: venus, venus-messager, venus-miner, venus-gateway, venus-auth, venus-market, venus-sealer, venus-wallet, venus-cluster, api, chain, state, vm, data transfer, mempool, message, block production, multisig, networking, paychan, proving, sealing, wallet
  • This PR has tests for new functionality or change in behaviour
  • If new user-facing features are introduced, clear usage guidelines and / or documentation updates should be included in venus docs or Discussion Tutorials.
  • CI is green

@LinZexiao
Copy link
Collaborator Author

lint fail because it rely on another unmerged PR in venus repo

@LinZexiao LinZexiao requested a review from hunjixin June 23, 2022 04:59
api/impl/venus_market.go Outdated Show resolved Hide resolved
api/impl/venus_market.go Outdated Show resolved Hide resolved
api/impl/venus_market.go Outdated Show resolved Hide resolved
piecestorage/storagemgr.go Outdated Show resolved Hide resolved
piecestorage/storagemgr_test.go Outdated Show resolved Hide resolved
piecestorage/type.go Outdated Show resolved Hide resolved
Copy link
Contributor

@diwufeiwen diwufeiwen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

有些类型应该移到venus-shared仓库了,需要更新

cli/piece-storage.go Outdated Show resolved Hide resolved
cli/piece-storage.go Show resolved Hide resolved
cli/piece-storage.go Outdated Show resolved Hide resolved
cli/piece-storage.go Show resolved Hide resolved
cli/piece-storage.go Show resolved Hide resolved
config/config.go Show resolved Hide resolved
config/config.go Outdated Show resolved Hide resolved
config/config.go Outdated Show resolved Hide resolved
piecestorage/storagemgr.go Show resolved Hide resolved
piecestorage/storagemgr.go Outdated Show resolved Hide resolved
cli/piece-storage.go Show resolved Hide resolved
cli/piece-storage.go Show resolved Hide resolved
go.mod Outdated Show resolved Hide resolved
LinZexiao added 3 commits July 4, 2022 14:10
unified the name of sub command
check flag by cctx.IsSet
add ArgsUsage comment
@LinZexiao LinZexiao force-pushed the feat/tanlang/addPieceStorage branch from 46d976f to 2f48692 Compare July 4, 2022 06:17
Copy link
Contributor

@diwufeiwen diwufeiwen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

api/impl/venus_market.go Outdated Show resolved Hide resolved
api/impl/venus_market.go Show resolved Hide resolved
Aliases: []string{"e"},
Usage: "endpoint of the S3 bucket",
},
// access key
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

下面三个量最好使用交互式的输入方式,避免在history里面留下记录

config/config.go Outdated Show resolved Hide resolved
@@ -14,8 +15,20 @@ type PieceStorageManager struct {

func NewPieceStorageManager(cfg *config.PieceStorage) (*PieceStorageManager, error) {
var storages []IPieceStorage
exist := make(map[string]bool)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

建议使用map[string]struct{}

return nil
}

func (p *PieceStorageManager) ListStorages() types.PieceStorageList {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

建议名字是ListStorageInfos, 类型同理

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PieceStorageInfos

piecestorage/storagemgr.go Show resolved Hide resolved
@@ -83,3 +114,45 @@ func randStorageSelector(storages []IPieceStorage) (IPieceStorage, error) {
return storages[rand.Intn(len(storages))], nil
}
}

func (p *PieceStorageManager) GetStorages() map[string]IPieceStorage {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

删除

cs.Close() // nolint:errcheck
}()

rl := bufio.NewReader(cs)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

做到类似密码的效果

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1

@codecov-commenter
Copy link

codecov-commenter commented Jul 5, 2022

Codecov Report

Merging #153 (4b91c39) into main (587bd0a) will increase coverage by 0.28%.
The diff coverage is 20.75%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #153      +/-   ##
==========================================
+ Coverage   18.81%   19.09%   +0.28%     
==========================================
  Files         170      172       +2     
  Lines       17804    18213     +409     
==========================================
+ Hits         3349     3478     +129     
- Misses      14023    14292     +269     
- Partials      432      443      +11     
Flag Coverage Δ
unittests 19.09% <20.75%> (+0.28%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
api/impl/venus_market.go 0.00% <0.00%> (ø)
cli/piece-storage.go 0.00% <0.00%> (ø)
cli/util.go 0.00% <0.00%> (ø)
cmd/venus-market/main.go 0.00% <0.00%> (ø)
cmd/venus-market/pool-run.go 0.00% <ø> (ø)
cmd/venus-market/solo-run.go 0.00% <ø> (ø)
config/config.go 0.00% <0.00%> (ø)
piecestorage/s3.go 0.00% <0.00%> (ø)
piecestorage/filestore.go 20.73% <40.00%> (+3.84%) ⬆️
piecestorage/storagemgr.go 61.11% <64.55%> (+6.20%) ⬆️
... and 10 more

@LinZexiao LinZexiao requested a review from hunjixin July 5, 2022 05:53
piecestorage/storagemgr.go Outdated Show resolved Hide resolved
piecestorage/storagemgr.go Show resolved Hide resolved
@LinZexiao LinZexiao force-pushed the feat/tanlang/addPieceStorage branch from 22a9ad9 to e45787c Compare July 5, 2022 06:55
@LinZexiao LinZexiao force-pushed the feat/tanlang/addPieceStorage branch from e45787c to 29f241e Compare July 5, 2022 07:11
Copy link
Contributor

@hunjixin hunjixin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

piecestorage/storagemgr_test.go Outdated Show resolved Hide resolved
@hunjixin hunjixin merged commit f047bfd into main Jul 5, 2022
@zl03jsj zl03jsj deleted the feat/tanlang/addPieceStorage branch July 14, 2022 09:55
litai686 pushed a commit to litai686/venus-market that referenced this pull request Jul 30, 2022
…/feat/dtynn/cgroup_cleanup

chore: add a script for cleaning up orphan sub-groups in cgroup:cpuset
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[venus-market] attach new piece store
4 participants