Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: introduce SyntheticPoRep #11057

Closed
wants to merge 92 commits into from
Closed

feat: introduce SyntheticPoRep #11057

wants to merge 92 commits into from

Conversation

rjan90
Copy link
Contributor

@rjan90 rjan90 commented Jul 7, 2023

Related Issues

Proposed Changes

Documentation for steps taken: https://hackmd.io/@fiYJFp3GQGuKE-C0ABEqkA/rke2MmNYh

Introduce nv21 skeleton for local testing:

Additional Info

Testing plan:

  • Check that network version at start of local devnet is correct
  • Check that the Actor CIDs/Actor version for the mock v12-bundle is visible through lotus state actors-cids --network-version 21
  • Successful pre-migration at epoch 80
  • Check that network upgrade works locally with local GST
  • Wire in alpha GST
  • Check that network upgrade works on a local network with alpha GSP

Checklist

Before you mark the PR ready for review, please make sure that:

  • Commits have a clear commit message.
  • PR title is in the form of of <PR type>: <area>: <change being made>
    • example: fix: mempool: Introduce a cache for valid signatures
    • PR type: fix, feat, build, chore, ci, docs, perf, refactor, revert, style, test
    • area, e.g. api, chain, state, market, mempool, multisig, networking, paych, proving, sealing, wallet, deps
  • New features have usage guidelines and / or documentation updates in
  • Tests exist for new functionality or change in behavior
  • CI is green

@rjan90 rjan90 mentioned this pull request Jul 7, 2023
7 tasks
Comment on lines 1830 to 1833
// Use half the CPUs for pre-migration, but leave at least 3.
workerCount := MigrationMaxWorkerCount
if workerCount <= 4 {
workerCount = 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes it "at least 1"

@magik6k magik6k mentioned this pull request Jul 12, 2023
7 tasks
@magik6k magik6k changed the title [WIP] feat: introduce local nv21 skeleton feat: introduce nv21, SyntheticPoRep Jul 13, 2023
@magik6k magik6k changed the base branch from master to release/v1.23.3 July 13, 2023 13:43
rjan90 and others added 23 commits July 13, 2023 15:47
Introduce nv21 skeleton for local testing:

- Use local go-state-types with actor_version_checklist changes: https://github.com/filecoin-project/go-state-types/blob/master/actors_version_checklist.md
- Imports mock v12-actors bundle
- Define upgrade heights
- Generate adapters
- Add upgrade schedule and migration
- Add actorstype to the NewActorRegistry in /chain/consensus/computestate.go
- Add upgrade field to api/types.go/ForkUpgradeParams
- Add upgrade to node/impl/full/state.go
- Add network version to chain/state/statetree.go
- make jen
- make docsgen-cli
Set local devnet-genesis to nv20
@magik6k magik6k force-pushed the feat/nv21-skeleton branch from 5ca10e4 to 85b78f9 Compare July 13, 2023 13:49
magik6k and others added 27 commits September 7, 2023 12:44
We've now lost 2 recursive calls, but that really shouldn't make a
difference.
Update butterfly-schedule
Add butterfly artifacts
Update butterfly-bootstraps
Updated Butterfly-assets as the butterfly-network has been reset.
Set upgrade epoch on Butterfly earlier
Add new butterfly-assets
This commit includes the following updates:

- Imported v12 of the filecoin state types in migrations.go. This allows the system to interpret and operate on the latest structure of filecoin state information, ensuring compatibility with the latest version of the Filecoin network.

- Updated the 'getMigrationFuncsForNetwork' function to include a case for network.Version21, enabling the use of the UpgradeActorsV12 and PreUpgradeActorsV12 functions for network migration. These changes ensure the system correctly handles migrations when the network upgrades to version 21.

- Implemented a new 'checkNv21Invariants' function to check the invariants for version 21 of the Filecoin network. This function:

  1. Loads the new state root from the actor store.
  2. Retrieves the actor code IDs for the current state version (v12).
  3. Loads the actor tree from the state root and checks state invariants using the actor code IDs.
  4. Logs any error messages generated during the invariant check.
  5. Finally, prints a message with the time taken to complete the checks.

By checking these invariants, we can ensure the system's state is consistent and valid after migrating to network version 21.
This commit includes the following updates:

- Added a new 'pprof' flag to the command-line arguments in main.go. This flag allows the user to specify a file name for writing CPU profile information, enabling performance analysis.

- Implemented functionality to start CPU profiling in the 'Before' function. If a file name is provided via the 'pprof' flag, the function creates the specified file and begins CPU profiling.

- Added an 'After' function to stop CPU profiling once the program execution is finished.

These changes provide a way to analyze the CPU usage of the 'lotus-shed' application, which can be valuable for performance tuning and optimization.
Register SynthPoRep in ResourceTable such that task-resource limitations are also applied to SynthPoRep-sectors (i.e PC1_32G_MAX_CONCURRENT)
@rjan90
Copy link
Contributor Author

rjan90 commented Sep 20, 2023

Closing as this was superseded by: #11258

@rjan90 rjan90 closed this Sep 20, 2023
@rjan90 rjan90 deleted the feat/nv21-skeleton branch September 20, 2023 07:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Introduce nv21 skeleton in Lotus Implement SynthPoRep
7 participants